# 4 Bit quantization

**URL:** <https://discuss.huggingface.co/t/4-bit-quantization/49864>\
**Category:** 🤗Optimum\
**Created:** [August 7, 2023, 7:51pm UTC](https://discuss.huggingface.co/t/4-bit-quantization/49864 "2023-08-07T19:51:18Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sameearif88](https://avatars.discourse-cdn.com/v4/letter/s/b4bc9f/32.png) [@sameearif88](https://discuss.huggingface.co/u/sameearif88)\
**Post date:** [August 7, 2023, 7:51pm UTC](https://discuss.huggingface.co/t/4-bit-quantization/49864/1 "2023-08-07T19:51:18Z")

</div>

Is there a way to do 4 bit quantization using optimum?

---

<div class="post-metadata">

**Author:** ![regisss](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/regisss/32/22467_2.png) [@regisss](https://discuss.huggingface.co/u/regisss)\
**Post date:** [August 9, 2023, 4:38pm UTC](https://discuss.huggingface.co/t/4-bit-quantization/49864/2 "2023-08-09T16:38:43Z")

</div>

Hi @sameearif88, GPTQ will soon be available in Optimum to enable 4-bit quantization. You can follow the ongoing PR here: [https://github.com/huggingface/optimum/pull/1216](https://github.com/huggingface/optimum/pull/1216)

---

<div class="post-metadata">

**Author:** ![sameearif88](https://avatars.discourse-cdn.com/v4/letter/s/b4bc9f/32.png) [@sameearif88](https://discuss.huggingface.co/u/sameearif88)\
**Post date:** [August 9, 2023, 5:16pm UTC](https://discuss.huggingface.co/t/4-bit-quantization/49864/4 "2023-08-09T17:16:41Z")

</div>

Hello,

Is it possible to 4 bit quantize OpenAI Whisper and Facebook MMS audio models using Optimum?

Thanks

---

<div class="post-metadata">

**Author:** ![regisss](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/regisss/32/22467_2.png) [@regisss](https://discuss.huggingface.co/u/regisss)\
**Post date:** [August 9, 2023, 5:22pm UTC](https://discuss.huggingface.co/t/4-bit-quantization/49864/5 "2023-08-09T17:22:23Z")

</div>

@sameearif88 GPTQ only works for text models at the moment so it won’t be possible to perform 4-bit quantization of speech models right away.

---

<div class="post-metadata">

**Author:** ![fxmarty](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/fxmarty/32/23782_2.png) [@fxmarty](https://discuss.huggingface.co/u/fxmarty)\
**Post date:** [August 11, 2023, 3:17pm UTC](https://discuss.huggingface.co/t/4-bit-quantization/49864/6 "2023-08-11T15:17:24Z")

</div>

@sameearif88 Feel free to open a feature request for it on GitHub.