# Issue loading quantised model

**URL:** <https://discuss.huggingface.co/t/issue-loading-quantised-model/75087>\
**Category:** 🤗Transformers\
**Created:** [February 28, 2024, 3:19am UTC](https://discuss.huggingface.co/t/issue-loading-quantised-model/75087 "2024-02-28T03:19:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![sanjanatule](https://avatars.discourse-cdn.com/v4/letter/s/a4c791/32.png) [@sanjanatule](https://discuss.huggingface.co/u/sanjanatule)\
**Post date:** [February 28, 2024, 3:19am UTC](https://discuss.huggingface.co/t/issue-loading-quantised-model/75087/1 "2024-02-28T03:19:38Z")

</div>

Issue loading quantised model. I get the error KeyError: ‘inv\_freq’. What is causing this? Any help is appreciated.

# Quantization definition

bnb\_config = BitsAndBytesConfig(  
load\_in\_4bit = True,  
bnb\_4bit\_use\_double\_quant=True,  
bnb\_4bit\_quant\_type=“nf4”,  
bnb\_4bit\_compute\_dtype=torch.bfloat16,  
)

# load the qunatized model

esm\_model\_name = “facebook/esm2\_t30\_150M\_UR50D”  
model = AutoModelForTokenClassification.from\_pretrained(esm\_model\_name, quantization\_config = bnb\_config,trust\_remote\_code=True,device\_map=‘cuda’)  
model