# Problem with pushing quantized model to hub

**URL:** <https://discuss.huggingface.co/t/problem-with-pushing-quantized-model-to-hub/84255>\
**Category:** 🤗Transformers\
**Created:** [April 29, 2024, 4:00am UTC](https://discuss.huggingface.co/t/problem-with-pushing-quantized-model-to-hub/84255 "2024-04-29T04:00:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![gospacedev](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/gospacedev/32/26312_2.png) [@gospacedev](https://discuss.huggingface.co/u/gospacedev)\
**Post date:** [April 29, 2024, 4:00am UTC](https://discuss.huggingface.co/t/problem-with-pushing-quantized-model-to-hub/84255/1 "2024-04-29T04:00:54Z")

</div>

Hi, everyone! I have been trying to upload a quantized model to huggingface using the push\_to\_hub function but I always receive this error:

```auto
AttributeError: 'str' object has no attribute 'data_ptr'

```

Here’s the code:

```python
from transformers import T5Tokenizer, T5ForConditionalGeneration
from quanto import quantize, freeze, qint8

model_id = "google/flan-t5-base"

quantized_model = T5ForConditionalGeneration.from_pretrained(model_id, low_cpu_mem_usage=True, use_safetensors=True)

quantize(quantized_model, weights=qint8, activaions=None)

freeze(quantized_model)

tokenizer.push_to_hub("flan-t5-base-8bit")
quantized_model.push_to_hub("flan-t5-base-8bit")

```

Thank you in advance!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [April 29, 2024, 7:39am UTC](https://discuss.huggingface.co/t/problem-with-pushing-quantized-model-to-hub/84255/2 "2024-04-29T07:39:23Z")

</div>

Hi,

Feel free to open an issue on the Quanto repository: [GitHub - huggingface/quanto: A pytorch Quantization Toolkit](https://github.com/huggingface/quanto)

---

<div class="post-metadata">

**Author:** ![Dcolinmorgan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dcolinmorgan/32/32886_2.png) [@Dcolinmorgan](https://discuss.huggingface.co/u/Dcolinmorgan)\
**Post date:** [October 14, 2024, 4:29am UTC](https://discuss.huggingface.co/t/problem-with-pushing-quantized-model-to-hub/84255/3 "2024-10-14T04:29:33Z")

</div>

> [@gospacedev](#):
>
> ```auto
> AttributeError: 'str' object has no attribute 'data_ptr'
> 
> ```

did you ever solve this? i also get it when trying to `.save_pretrained` after the freeze

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [October 14, 2024, 4:33am UTC](https://discuss.huggingface.co/t/problem-with-pushing-quantized-model-to-hub/84255/4 "2024-10-14T04:33:27Z")

</div>

There are too many causes of AttributeError to pinpoint, but it is said that it can be caused by too old or too new a version of the transformers library.

> <https://github.com/huggingface/transformers/issues/24778>
>
> \### System Info
> 
> ===================================BUG REPORT================…===================
> Welcome to bitsandbytes. For bug reports, please run
> 
> python -m bitsandbytes
> 
> 
> ================================================================================
> bin /opt/conda/envs/pytorch/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes\_cuda118.so
> CUDA SETUP: CUDA runtime path found: /opt/conda/envs/pytorch/lib/libcudart.so.11.0
> CUDA SETUP: Highest compute capability among GPUs detected: 7.5
> CUDA SETUP: Detected CUDA version 118
> CUDA SETUP: Loading binary /opt/conda/envs/pytorch/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes\_cuda118.so...
> \[2023-07-12 13:52:54,626\] \[INFO\] \[real\_accelerator.py:110:get\_accelerator\] Setting ds\_accelerator to cuda (auto detect)
> 
> Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.
> 
> \- \`transformers\` version: 4.30.2
> \- Platform: Linux-5.15.0-1038-aws-x86\_64-with-glibc2.31
> \- Python version: 3.10.12
> \- Huggingface\_hub version: 0.16.2
> \- Safetensors version: 0.3.1
> \- PyTorch version (GPU?): 2.0.1 (True)
> \- Tensorflow version (GPU?): not installed (NA)
> \- Flax version (CPU?/GPU?/TPU?): not installed (NA)
> \- Jax version: not installed
> \- JaxLib version: not installed
> \- Using GPU in script?: \<fill in\>
> \- Using distributed or parallel set-up in script?: \<fill in\>
> 
> \### Who can help?
> 
> \_No response\_
> 
> \### Information
> 
> \- \[X\] The official example scripts
> \- \[\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> Hi,
> 
> I'm trying to save quantized model. First attempt didn't work. (I also opened an issue, https://github.com/huggingface/accelerate/issues/1713, to clarify it). 
> 
> I opened this issue because I'm receiving an error message when I run following code. I'm not sure I'm following the right instructions written on https://huggingface.co/docs/transformers/main\_classes/quantization. Because model is pushed to hub in documentation. But I expect to save it to local filesystem. Thanks for your help in advance. 
> 
> \`\`\`
> \### load packages ###
> import transformers
> import textwrap
> from transformers import LlamaTokenizer, LlamaForCausalLM
> import os
> import sys
> from typing import List
> import accelerate
>  
> from peft import (
> LoraConfig,
> get\_peft\_model,
> get\_peft\_model\_state\_dict,
> prepare\_model\_for\_int8\_training,
> )
>  
> \#import fire
> import torch
> from datasets import load\_dataset
> import pandas as pd
> import deepspeed
> 
> DEVICE = "cuda" if torch.cuda.is\_available() else "cpu"
> DEVICE
> 
> \### load model ###
> BASE\_MODEL = "decapoda-research/llama-7b-hf"
>  
> model = LlamaForCausalLM.from\_pretrained(
> BASE\_MODEL,
> load\_in\_8bit=True,
> torch\_dtype=torch.float16,
> device\_map="auto",
> )
>  
> model.save\_pretrained(save\_directory="quantized\_decapoda-research\_llama-7b-hf\_v2")
> 
> \`\`\`
> 
> Error Message:
> \`\`\`
> /opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/modeling\_utils.py:1709: UserWarning: You are calling \`save\_pretrained\` to a 8-bit converted model you may likely encounter unexepected behaviors. If you want to save 8-bit models, make sure to have \`bitsandbytes\>0.37.2\` installed.
> warnings.warn(
> \---------------------------------------------------------------------------
> AttributeError Traceback (most recent call last)
> Cell In\[3\], line 1
> \----\> 1 model.save\_pretrained(save\_directory="quantized\_decapoda-research\_llama-7b-hf\_v2")
> 
> File /opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/modeling\_utils.py:1820, in PreTrainedModel.save\_pretrained(self, save\_directory, is\_main\_process, state\_dict, save\_function, push\_to\_hub, max\_shard\_size, safe\_serialization, variant, \*\*kwargs)
> 1817 weights\_name = SAFE\_WEIGHTS\_NAME if safe\_serialization else WEIGHTS\_NAME
> 1818 weights\_name = \_add\_variant(weights\_name, variant)
> \-\> 1820 shards, index = shard\_checkpoint(state\_dict, max\_shard\_size=max\_shard\_size, weights\_name=weights\_name)
> 1822 # Clean the folder from a previous save
> 1823 for filename in os.listdir(save\_directory):
> 
> File /opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/modeling\_utils.py:318, in shard\_checkpoint(state\_dict, max\_shard\_size, weights\_name)
> 315 storage\_id\_to\_block = {}
> 317 for key, weight in state\_dict.items():
> \--\> 318 storage\_id = id\_tensor\_storage(weight)
> 320 # If a \`weight\` shares the same underlying storage as another tensor, we put \`weight\` in the same \`block\`
> 321 if storage\_id in storage\_id\_to\_block:
> 
> File /opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/pytorch\_utils.py:290, in id\_tensor\_storage(tensor)
> 283 def id\_tensor\_storage(tensor: torch.Tensor) -\> Tuple\[torch.device, int, int\]:
> 284 """
> 285 Unique identifier to a tensor storage. Multiple different tensors can share the same underlying storage. For
> 286 example, "meta" tensors all share the same storage, and thus their identifier will all be equal. This identifier is
> 287 guaranteed to be unique and constant for this tensor's storage during its lifetime. Two tensor storages with
> 288 non-overlapping lifetimes may have the same id.
> 289 """
> \--\> 290 return tensor.device, storage\_ptr(tensor), storage\_size(tensor)
> 
> AttributeError: 'str' object has no attribute 'device'
> \`\`\`
> 
> \### Expected behavior
> 
> Save quantized model to local filesystem.