# Quntisation LLM models

**URL:** <https://discuss.huggingface.co/t/quntisation-llm-models/125583>\
**Category:** Beginners\
**Created:** [November 18, 2024, 7:32am UTC](https://discuss.huggingface.co/t/quntisation-llm-models/125583 "2024-11-18T07:32:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tesemnikov-av](https://avatars.discourse-cdn.com/v4/letter/t/bc79bd/32.png) [@tesemnikov-av](https://discuss.huggingface.co/u/tesemnikov-av)\
**Post date:** [November 18, 2024, 7:32am UTC](https://discuss.huggingface.co/t/quntisation-llm-models/125583/1 "2024-11-18T07:32:36Z")

</div>

Am I correct in understanding that Q\_4\_K\_S quantization means 4-bit quantization with k-quantization(K) and a small(S) block size? What does a small block size mean? Q\_3 means 3 bit quantization? Thanks!

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [November 18, 2024, 8:51am UTC](https://discuss.huggingface.co/t/quntisation-llm-models/125583/2 "2024-11-18T08:51:49Z")

</div>

I can’t say I understand it correctly either, so I tried searching for it. Well, if you’re in trouble, use Q4\_K\_M.  
[https://www.reddit.com/r/LocalLLaMA/comments/1d1sc50/gguf\_weight\_encoding\_suffixes\_is\_there\_a\_guide/](https://www.reddit.com/r/LocalLLaMA/comments/1d1sc50/gguf_weight_encoding_suffixes_is_there_a_guide/)

> <https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9>

> **[Understanding the GGUF Format: A Comprehensive Guide](https://medium.com/@vimalkansal/understanding-the-gguf-format-a-comprehensive-guide-67de48848256)**
>
> Introduction

> **[A Visual Guide to Quantization - Maarten Grootendorst](https://www.maartengrootendorst.com/blog/quantization/)**
>
> My Personal Website