# Multi\_class\_classification errors when fine-tuning via TrainerAPI

**URL:** <https://discuss.huggingface.co/t/multi-class-classification-errors-when-fine-tuning-via-trainerapi/32058>\
**Category:** Beginners\
**Created:** [February 20, 2023, 3:07am UTC](https://discuss.huggingface.co/t/multi-class-classification-errors-when-fine-tuning-via-trainerapi/32058 "2023-02-20T03:07:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![SummerSigh](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/summersigh/32/14250_2.png) [@SummerSigh](https://discuss.huggingface.co/u/SummerSigh)\
**Post date:** [February 20, 2023, 3:07am UTC](https://discuss.huggingface.co/t/multi-class-classification-errors-when-fine-tuning-via-trainerapi/32058/1 "2023-02-20T03:07:49Z")

</div>

I’m attempting to do a multiclass finetuning of distilbert using the following code:

import torch  
from transformers import AutoModelForSequenceClassification, AutoTokenizer, TrainingArguments, Trainer, DataCollatorWithPadding  
from datasets import load\_dataset  
import evaluate  
from tqdm.notebook import tqdm  
from datasets import load\_dataset  
import numpy as np

import os

dataset = load\_dataset(“csv”, data\_files=“datafile.csv”,keep\_default\_na=False)  
dataset[“train”] = dataset[“train”].rename\_column(“label”, “labels”)  
dataset = dataset[“train”].train\_test\_split(test\_size=0.3)  
dataset[“train”] = dataset[“train”].shuffle()  
dataset[“test”] = dataset[“test”].shuffle()  
def preprocess\_function(examples):  
return tokenizer(examples[“text”], truncation=True)

model\_id = “distilbert-base-uncased”  
tokenizer = AutoTokenizer.from\_pretrained(model\_id, max\_length = 512, model\_max\_length = 512)  
tokenized\_dataset = dataset.map(preprocess\_function, batched=True)  
data\_collator = DataCollatorWithPadding(tokenizer=tokenizer)  
accuracy = evaluate.load(“accuracy”)

def compute\_metrics(eval\_pred):  
predictions, labels = eval\_pred  
predictions = np.argmax(predictions, axis=1)  
return accuracy.compute(predictions=predictions, references=labels)

id2label = {0: “casual”, 1: “possibly\_needs\_caution”, 2: “probably\_needs\_caution”, 3: “needs\_caution”, 4: “needs\_intervention”}  
label2id = {“casual”: 0, “possibly\_needs\_caution”: 1, “probably\_needs\_caution”: 2, “needs\_caution”: 3, “needs\_intervention”: 4}

model = AutoModelForSequenceClassification.from\_pretrained(

```
model_id, num_labels=5, id2label=id2label, label2id=label2id,problem_type="multi_class_classification"

```

)

tokenized\_dataset[“train”] = tokenized\_dataset[“train”].remove\_columns([‘text’,‘Unnamed: 0’])  
tokenized\_dataset[“test”] = tokenized\_dataset[“test”].remove\_columns([‘text’,‘Unnamed: 0’])

training\_args = TrainingArguments(

```
output_dir="safety_instruct_detect",

learning_rate=4.5e-5,

per_device_train_batch_size=8,

per_device_eval_batch_size=8,

#fp16 = True,

warmup_steps = 1000, 

num_train_epochs=2,

evaluation_strategy="epoch",

save_strategy="epoch",

load_best_model_at_end=True,

push_to_hub=False,

```

)

trainer = Trainer(

```
model=model,

args=training_args,

train_dataset=tokenized_dataset["train"],

eval_dataset=tokenized_dataset["test"],

tokenizer=tokenizer,

data_collator=data_collator,

compute_metrics=compute_metrics,

```

)

trainer.train()

I get the following error:

ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are labels,input\_ids,attention\_mask.

Here are a few things i’ve tried:

- Using BertForSequenceClassification rather than AutoModelForSequenceClassification

- Making the labels in the dataset a float rather than int

- Using both “label” and “labels” as column names a few different times