# Loading llama3.21B in quantized config shows no change in size

**URL:** <https://discuss.huggingface.co/t/loading-llama3-21b-in-quantized-config-shows-no-change-in-size/130062>\
**Category:** Beginners\
**Created:** [December 9, 2024, 4:11am UTC](https://discuss.huggingface.co/t/loading-llama3-21b-in-quantized-config-shows-no-change-in-size/130062 "2024-12-09T04:11:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![suruti94](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/suruti94/32/37030_2.png) [@suruti94](https://discuss.huggingface.co/u/suruti94)\
**Post date:** [December 9, 2024, 4:11am UTC](https://discuss.huggingface.co/t/loading-llama3-21b-in-quantized-config-shows-no-change-in-size/130062/1 "2024-12-09T04:11:49Z")

</div>

Hi,

This is what I have:

bnb\_config = BitsAndBytesConfig(  
load\_in\_4bit=True,  
bnb\_4bit\_use\_double\_quant=True,  
bnb\_4bit\_quant\_type=“nf4”,  
bnb\_4bit\_compute\_dtype=torch.bfloat16  
)  
model\_id = “meta-llama/Llama-3.2-1B”  
model = AutoModelForCausalLM.from\_pretrained(model\_id, quantization\_config=bnb\_config, device\_map={“”:0})  
print(model.get\_memory\_footprint())

The result is : 1012013184

why don’t I see a reduction in memory footprint ? When I use “facebook/opt-350m” I see the footprint is 207m. I have a GTX 1080 on my system. What am i missing ?

Thanks  
Mohan

---

<div class="post-metadata">

**Author:** ![suruti94](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/suruti94/32/37030_2.png) [@suruti94](https://discuss.huggingface.co/u/suruti94)\
**Post date:** [December 10, 2024, 2:34am UTC](https://discuss.huggingface.co/t/loading-llama3-21b-in-quantized-config-shows-no-change-in-size/130062/2 "2024-12-10T02:34:31Z")

</div>

I measured it again with this code. It measures both the model footprint as reported by pytorch and gpu memory allocated.

torch.cuda.reset\_peak\_memory\_stats(device=None)  
model\_id = “meta-llama/Llama-3.2-1B”  
model = AutoModelForCausalLM.from\_pretrained(model\_id, load\_in\_4bit=True, device\_map={“”:0})  
#model = AutoModelForCausalLM.from\_pretrained(model\_id, device\_map={“”:0})  
print(model.get\_memory\_footprint())  
print(f"gpu used {torch.cuda.max\_memory\_allocated(device=None)}")

It was 4.9b vs 1gb (footprint and cuda memory was pretty close) between original vs 4bit. Sorry for the confusion.

-thanks  
Mohan