# Error with GPTQ for distilbert/distilbert-base-cased

**URL:** <https://discuss.huggingface.co/t/error-with-gptq-for-distilbert-distilbert-base-cased/106955>\
**Category:** 🤗Transformers\
**Created:** [September 16, 2024, 5:30pm UTC](https://discuss.huggingface.co/t/error-with-gptq-for-distilbert-distilbert-base-cased/106955 "2024-09-16T17:30:05Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![vrathi](https://avatars.discourse-cdn.com/v4/letter/v/8e8cbc/32.png) [@vrathi](https://discuss.huggingface.co/u/vrathi)\
**Post date:** [September 16, 2024, 5:30pm UTC](https://discuss.huggingface.co/t/error-with-gptq-for-distilbert-distilbert-base-cased/106955/1 "2024-09-16T17:30:06Z")

</div>

Hi,

I tried to quantize distilbert/distilbert-base-cased using the following code (instructions taken from Huggingface):

from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig  
model\_id = “distilbert/distilbert-base-cased”  
tokenizer = AutoTokenizer.from\_pretrained(model\_id)  
gptq\_config = GPTQConfig(bits=4, dataset=“c4”, tokenizer=tokenizer)  
quantized\_model = AutoModelForCausalLM.from\_pretrained(model\_id, device\_map=“auto”, quantization\_config=gptq\_config)

I got the following error. How can I quantize this distilbert model using GPTQ?

Thanks.

* * *

ValueError Traceback (most recent call last)  
 in \<cell line: 1\>()  
----\> 1 quantized\_model = AutoModelForCausalLM.from\_pretrained(model\_id, device\_map=“auto”, quantization\_config=gptq\_config)

/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto\_factory.py in from\_pretrained(cls, pretrained\_model\_name\_or\_path, \*model\_args, \*\*kwargs)  
565 pretrained\_model\_name\_or\_path, \*model\_args, config=config, \*\*hub\_kwargs, \*\*kwargs  
566 )  
 → 567 raise ValueError(  
568 f"Unrecognized configuration class {config. **class** } for this kind of AutoModel: {cls. **name** }.\n"  
569 f"Model type should be one of {', '.join(c. **name** for c in cls.\_model\_mapping.keys())}."

ValueError: Unrecognized configuration class \<class ‘transformers.models.distilbert.configuration\_distilbert.DistilBertConfig’\> for this kind of AutoModel: AutoModelForCausalLM.  
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, ElectraConfig, ErnieConfig, FalconConfig, FuyuConfig, GemmaConfig, Gemma2Config, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, JambaConfig, JetMoeConfig, LlamaConfig, MambaConfig, Mamba2Config, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, NemotronConfig, OlmoConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, Phi3Config, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig.