# Evaluating with quantized model using Trainer

**URL:** <https://discuss.huggingface.co/t/evaluating-with-quantized-model-using-trainer/82627>\
**Category:** Beginners\
**Created:** [April 19, 2024, 11:55am UTC](https://discuss.huggingface.co/t/evaluating-with-quantized-model-using-trainer/82627 "2024-04-19T11:55:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![daltonh](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@daltonh](https://discuss.huggingface.co/u/daltonh)\
**Post date:** [April 19, 2024, 11:55am UTC](https://discuss.huggingface.co/t/evaluating-with-quantized-model-using-trainer/82627/1 "2024-04-19T11:55:39Z")

</div>

I have quantized a model using BitsAndBytes, and I want to evaluate this model on some benchmark tasks. For evaluation on other models, I use trainer.evaluate() to obtain the metrics.  
This is however different for quantized models. I use the following code:

```auto
...
trainer = Trainer(
    model=model_quantized,
    ....
)
trainer.evaluate(test_data)
...

```

I obtain the following error:

```auto
ValueError: You cannot perform fine-tuning on purely quantized models. Please attach trainable adapters on top of the quantized model to correctly perform fine-tuning. Please see: https://huggingface.co/docs/transformers/peft for more details

```

I understand that the error gets raised, as I am using the Trainer object on a quantized model, but I do not intend to train this model, and only perfom evaluation.  
Is there a way to avoid this error and run evaluation on a quantized model?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![daminho](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/daminho/32/29506_2.png) [@daminho](https://discuss.huggingface.co/u/daminho)\
**Post date:** [August 10, 2024, 11:14am UTC](https://discuss.huggingface.co/t/evaluating-with-quantized-model-using-trainer/82627/2 "2024-08-10T11:14:50Z")

</div>

> **[🤗 Evaluate](https://huggingface.co/docs/evaluate/en/index)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

You can try to use evaluate library from huggingface instead