# Finetuning llama-2 for classification

**URL:** <https://discuss.huggingface.co/t/finetuning-llama-2-for-classification/61564>\
**Category:** 🤗Transformers\
**Created:** [November 8, 2023, 6:52pm UTC](https://discuss.huggingface.co/t/finetuning-llama-2-for-classification/61564 "2023-11-08T18:52:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![zuhashaik](https://avatars.discourse-cdn.com/v4/letter/z/edb3f5/32.png) [@zuhashaik](https://discuss.huggingface.co/u/zuhashaik)\
**Post date:** [November 8, 2023, 6:52pm UTC](https://discuss.huggingface.co/t/finetuning-llama-2-for-classification/61564/1 "2023-11-08T18:52:53Z")

</div>

used AutoModelForSequenceClassification

```auto
model = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    token=token
    )

```

and

```auto
bnb_config = BitsAndBytesConfig(
    load_in_4bit = True,
    bnb_4bit_qunat_type = "nf4",
    bnb_4bit_compute_dtype = torch.float16,
)

```

for loading the model and finetuned this model by using LoRA

```auto
peft_config = LoraConfig(
        r=16,
        lora_alpha=64,
        lora_dropout=0.1,
        bias="none",
        task_type='SEQ_CLS',
)

```

and saved as “tuned\_model”  
while loading the model:

```auto
from transformers import pipeline
pipe = pipeline('text-classification',
                tuned_model,
                device_map="auto")

```

```auto
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
[<ipython-input-19-fd1b8ca97698>](https://localhost:8080/#) in <cell line: 2>()
      1 from transformers import pipeline
----> 2 pipe = pipeline('text-classification',tuned_model, device_map="auto")

5 frames
[/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py](https://localhost:8080/#) in <dictcomp>(.0)
   3809 p: {"safetensors_file": f, "weight_name": p, "dtype": str_dtype}
   3810 for p, f in weight_map.items()
-> 3811 if param_device_map[p] == "disk"
   3812 }
   3813 

KeyError: 'lm_head.weight'

```

Can any one suggest me how to load this tuned\_model ?

---

<div class="post-metadata">

**Author:** ![lyrnat](https://avatars.discourse-cdn.com/v4/letter/l/edb3f5/32.png) [@lyrnat](https://discuss.huggingface.co/u/lyrnat)\
**Post date:** [January 29, 2024, 6:26pm UTC](https://discuss.huggingface.co/t/finetuning-llama-2-for-classification/61564/2 "2024-01-29T18:26:11Z")

</div>

@zuhashaik were you able to solve this? I encountered the same issue but wasn’t able to find a solution

---

<div class="post-metadata">

**Author:** ![zuhashaik](https://avatars.discourse-cdn.com/v4/letter/z/edb3f5/32.png) [@zuhashaik](https://discuss.huggingface.co/u/zuhashaik)\
**Post date:** [January 29, 2024, 6:54pm UTC](https://discuss.huggingface.co/t/finetuning-llama-2-for-classification/61564/3 "2024-01-29T18:54:17Z")

</div>

No, I stopped using pipeline, instead go ahead with the Automodelforsequenceclassification class it will recognise your adapter and load your tuned model and get the logits.  
Hope this helps! Any further problem dm me [zuhashaik12@gmail.com](mailto:zuhashaik12@gmail.com) happy to help! 🙂