# Error saving quantized model **URL:** \ **Category:** Intermediate\ **Created:** [June 29, 2022, 4:31pm UTC](https://discuss.huggingface.co/t/error-saving-quantized-model/19801 "2022-06-29T16:31:52Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![omoekan](https://avatars.discourse-cdn.com/v4/letter/o/45deac/32.png) [@omoekan](https://discuss.huggingface.co/u/omoekan)\ **Post date:** [June 29, 2022, 4:31pm UTC](https://discuss.huggingface.co/t/error-saving-quantized-model/19801/1 "2022-06-29T16:31:52Z")
I get the following error trying to save a quantized model. Can anyone help? Thanks. # quantize model quantized\_model = torch.quantization.quantize\_dynamic( model, {torch.nn.Linear}, dtype=torch.qint8 ) quantized\_model.save\_pretrained(quantized\_dir) * * * AttributeError Traceback (most recent call last) in () 3 model, {torch.nn.Linear}, dtype=torch.qint8 4 ) ----\> 5 quantized\_model.save\_pretrained(quantized\_dir) 1 frames /usr/local/lib/python3.7/dist-packages/transformers/modeling\_utils.py in shard\_checkpoint(state\_dict, max\_shard\_size) 291 292 for key, weight in state\_dict.items(): → 293 weight\_size = weight.numel() \* dtype\_byte\_size(weight.dtype) 294 295 # If this weight is going to tip up over the maximal size, we split. AttributeError: ‘torch.dtype’ object has no attribute ‘numel’ ---
**Author:** ![omoekan](https://avatars.discourse-cdn.com/v4/letter/o/45deac/32.png) [@omoekan](https://discuss.huggingface.co/u/omoekan)\ **Post date:** [June 30, 2022, 9:59pm UTC](https://discuss.huggingface.co/t/error-saving-quantized-model/19801/2 "2022-06-30T21:59:45Z")
I found the solution here > [@Pegasus Model Weights Compression/Pruning](https://discuss.huggingface.co/t/pegasus-model-weights-compression-pruning/6381/8): > > @lewtun I chose dynamic quantization approach. And I dont think from\_pretrained support loading quantised models. I’d be happy if you can take a look. Can we hop on a google meet call and you can help me out. Hopefully. Thanks ---
**Author:** ![rohanj](https://avatars.discourse-cdn.com/v4/letter/r/e0b2c6/32.png) [@rohanj](https://discuss.huggingface.co/u/rohanj)\ **Post date:** [November 24, 2022, 9:59am UTC](https://discuss.huggingface.co/t/error-saving-quantized-model/19801/3 "2022-11-24T09:59:56Z")
How did this issue got resolved? Am still getting the same error. ---
**Author:** ![dquan](https://avatars.discourse-cdn.com/v4/letter/d/ce7236/32.png) [@dquan](https://discuss.huggingface.co/u/dquan)\ **Post date:** [February 14, 2023, 3:31am UTC](https://discuss.huggingface.co/t/error-saving-quantized-model/19801/4 "2023-02-14T03:31:54Z")
Im getting the same error. Im trying to save a quantized model using the following line torch.quantization.quantize\_dynamic(model, {torch.nn.Linear}, dtype=torch.float16). When running quantized\_model.save\_pretrained(path\_dir), I’m running into the error AttributeError: ‘torch.dtype’ object has no attribute ‘numel’. Anyone have a solution? ---
**Author:** ![dquan](https://avatars.discourse-cdn.com/v4/letter/d/ce7236/32.png) [@dquan](https://discuss.huggingface.co/u/dquan)\ **Post date:** [February 16, 2023, 6:10pm UTC](https://discuss.huggingface.co/t/error-saving-quantized-model/19801/5 "2023-02-16T18:10:00Z")
I was actually able to resolve the error from the Pegasus Model Weights Compression/Pruning post above. Specifically, I was using Pytorch version 1.11.0 and HuggingFace version 4.20.1.