# Pushing a quantized (4bit) model on the Hub **URL:** \ **Category:** 🤗Transformers\ **Created:** [June 6, 2023, 8:53am UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228 "2023-06-06T08:53:29Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![sooolee](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sooolee/32/15821_2.png) [@sooolee](https://discuss.huggingface.co/u/sooolee)\ **Post date:** [June 6, 2023, 8:53am UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/1 "2023-06-06T08:53:29Z")
Hi. I trained a model in 4bit. After the training, trying to push the model to the Hub, and I get this error message: _ **NotImplementedError: You are calling `save_pretrained` on a 4-bit converted model. This is currently not supported.** _ Although the error message says ‘save\_pretrained’ my code is using ‘push\_to\_hub’: base\_model.push\_to\_hub(/) When is this function be supported? Thank you very much!! ---
**Author:** ![aplamhden](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@aplamhden](https://discuss.huggingface.co/u/aplamhden)\ **Post date:** [July 11, 2023, 10:43am UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/2 "2023-07-11T10:43:36Z")
push\_to\_hub uses save\_pretrained in order to save the model in physical format (i suspect) and then upload to cloud (HG hub) ---
**Author:** ![sooolee](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sooolee/32/15821_2.png) [@sooolee](https://discuss.huggingface.co/u/sooolee)\ **Post date:** [July 28, 2023, 5:58am UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/3 "2023-07-28T05:58:49Z")
Thanks for the comment, but that still doesn’t solve the issue. I tried again, same error message. I’m not sure (and doubt) if 4-bit model is still not supported? If so, what am I doing wrong? Can someone please provide answer or insight to this issue? Thanks! ---
**Author:** ![aplamhden](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@aplamhden](https://discuss.huggingface.co/u/aplamhden)\ **Post date:** [July 28, 2023, 6:13am UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/4 "2023-07-28T06:13:51Z")
It is supported for sure. My problem was solved after I used peft 0.4.0 I think. Cheers ---
**Author:** ![eloaf](https://avatars.discourse-cdn.com/v4/letter/e/58956e/32.png) [@eloaf](https://discuss.huggingface.co/u/eloaf)\ **Post date:** [July 28, 2023, 2:15pm UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/5 "2023-07-28T14:15:45Z")
I have peft==0.4.0 and I get the same error message when trying to save a 4-bit converted model. ---
**Author:** ![RonanMcGovern](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ronanmcgovern/32/56518_2.png) [@RonanMcGovern](https://discuss.huggingface.co/u/RonanMcGovern)\ **Post date:** [July 28, 2023, 3:58pm UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/6 "2023-07-28T15:58:26Z")
The docs aren’t entirely clear, but my read is that 8-bit is possible but 4-bit is not: > **[Quantize 🤗 Transformers models](https://huggingface.co/docs/transformers/main/main_classes/quantization)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. `Note that once a model has been loaded in 4-bit it is currently not possible to push the quantized weights on the Hub. Note also that you cannot train 4-bit weights as this is not supported yet. However you can use 4-bit models to train extra parameters, this will be covered in the next section.` ---
**Author:** ![danielpark](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/danielpark/32/20143_2.png) [@danielpark](https://discuss.huggingface.co/u/danielpark)\ **Post date:** [October 7, 2023, 10:14am UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/7 "2023-10-07T10:14:03Z")
Is this issue is closed??? I have a same issue with github version package. (peft and transfomers, etc.) ```auto NotImplementedError: You are calling save_pretrained on a 4-bit converted model. This is currently not supported. ``` ---
**Author:** ![RonanMcGovern](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ronanmcgovern/32/56518_2.png) [@RonanMcGovern](https://discuss.huggingface.co/u/RonanMcGovern)\ **Post date:** [January 8, 2024, 12:53pm UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/8 "2024-01-08T12:53:45Z")
This issue can probably be closed. Here’s the issue to follow on this: > > > Purpose: enable saving and loading transformers models in 4bit formats. > Enables… this PR in transformers: https://github.com/huggingface/transformers/pull/26037 > > addresses feature request #603 and other similar ones elsewhere. > > > tested with Bloom-560 and Llama-2-7b. > tested quantization, saving, loading, -- matched tensors and quant\_states -- matched inference results. > tested both nf4 and fp4 > test added ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [January 8, 2024, 2:49pm UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/9 "2024-01-08T14:49:14Z")
This is now possible, see Github gist: [push bnb 4 bit models on the hub · GitHub](https://gist.github.com/younesbelkada/89fd3984a2992fdbb408fa8e3bf44101) Tweet: [https://twitter.com/younesbelkada/status/1739244971905966380](https://twitter.com/younesbelkada/status/1739244971905966380) ---
**Author:** ![RonanMcGovern](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ronanmcgovern/32/56518_2.png) [@RonanMcGovern](https://discuss.huggingface.co/u/RonanMcGovern)\ **Post date:** [January 8, 2024, 6:28pm UTC](https://discuss.huggingface.co/t/pushing-a-quantized-4bit-model-on-the-hub/42228/10 "2024-01-08T18:28:20Z")
Yeah, correct, with the caveat that you still need to install transformers from source as the latest stable release doesn’t support it. Hence why I tagged the issue above, cheers.