# Issues with Trainer class on custom dataset

**URL:** <https://discuss.huggingface.co/t/issues-with-trainer-class-on-custom-dataset/946>\
**Category:** 🤗Transformers\
**Created:** [August 31, 2020, 11:28pm UTC](https://discuss.huggingface.co/t/issues-with-trainer-class-on-custom-dataset/946 "2020-08-31T23:28:32Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rbint](https://avatars.discourse-cdn.com/v4/letter/r/85e7bf/32.png) [@rbint](https://discuss.huggingface.co/u/rbint)\
**Post date:** [August 31, 2020, 11:28pm UTC](https://discuss.huggingface.co/t/issues-with-trainer-class-on-custom-dataset/946/1 "2020-08-31T23:28:32Z")

</div>

I am following along with the Sequence Classification on IMDB Dataset example here:  
[https://huggingface.co/transformers/master/custom\_datasets.html?utm\_campaign=Hugging%2BFace&utm\_medium=web&utm\_source=Hugging\_Face\_1](https://huggingface.co/transformers/master/custom_datasets.html?utm_campaign=Hugging%2BFace&utm_medium=web&utm_source=Hugging_Face_1)

However, I am using a custom dataset. Rather than iterate through using read\_imdb\_split I load my data from a csv, get the value of the sequences and labels, then convert them to a list to pass in to the tokenizer() method. From there I convert them to datasets using the subclassed dataset object as shown in the documentation.

Next, I create TrainingArguments and Trainer as shown and my issue then arises when calling trainer.train(). I receive the following error stack trace:

> * * *
> 
> AttributeError Traceback (most recent call last)  
> in   
> ----\> 1 trainer.train()
> 
> ~\anaconda3\lib\site-packages\transformers\trainer.py in train(self, model\_path)  
> 512 self.\_past = None  
> 513  
> → 514 for step, inputs in enumerate(epoch\_iterator):  
> 515  
> 516 # Skip past any already trained steps if resuming training
> 
> ~\anaconda3\lib\site-packages\tqdm\notebook.py in **iter** (self, \*args, \*\*kwargs)  
> 215 def **iter** (self, \*args, \*\*kwargs):  
> 216 try:  
> → 217 for obj in super(tqdm\_notebook, self). **iter** (\*args, \*\*kwargs):  
> 218 # return super(tqdm…) will not catch exception  
> 219 yield obj
> 
> ~\anaconda3\lib\site-packages\tqdm\std.py in **iter** (self)  
> 1127  
> 1128 try:  
> → 1129 for obj in iterable:  
> 1130 yield obj  
> 1131 # Update and possibly print the progressbar.
> 
> ~\anaconda3\lib\site-packages\torch\utils\data\dataloader.py in **next** (self)  
> 343  
> 344 def **next** (self):  
> → 345 data = self.\_next\_data()  
> 346 self.\_num\_yielded += 1  
> 347 if self.\_dataset\_kind == \_DatasetKind.Iterable and \
> 
> ~\anaconda3\lib\site-packages\torch\utils\data\dataloader.py in \_next\_data(self)  
> 383 def \_next\_data(self):  
> 384 index = self.\_next\_index() # may raise StopIteration  
> → 385 data = self.\_dataset\_fetcher.fetch(index) # may raise StopIteration  
> 386 if self.\_pin\_memory:  
> 387 data = \_utils.pin\_memory.pin\_memory(data)
> 
> ~\anaconda3\lib\site-packages\torch\utils\data\_utils\fetch.py in fetch(self, possibly\_batched\_index)  
> 42 def fetch(self, possibly\_batched\_index):  
> 43 if self.auto\_collation:  
> —\> 44 data = [self.dataset[idx] for idx in possibly\_batched\_index]  
> 45 else:  
> 46 data = self.dataset[possibly\_batched\_index]
> 
> ~\anaconda3\lib\site-packages\torch\utils\data\_utils\fetch.py in (.0)  
> 42 def fetch(self, possibly\_batched\_index):  
> 43 if self.auto\_collation:  
> —\> 44 data = [self.dataset[idx] for idx in possibly\_batched\_index]  
> 45 else:  
> 46 data = self.dataset[possibly\_batched\_index]
> 
> in **getitem** (self, idx)  
> 5  
> 6 def **getitem** (self, idx):  
> ----\> 7 item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}  
> 8 item[‘labels’] = torch.tensor(self.labels[idx])  
> 9 return item
> 
> AttributeError: ‘list’ object has no attribute ‘items’

I figure this is a result of me loading my data differently and converting them to lists, but I pass the datasets, not lists, to the trainer so I am unclear what is causing the error.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [August 31, 2020, 11:40pm UTC](https://discuss.huggingface.co/t/issues-with-trainer-class-on-custom-dataset/946/2 "2020-08-31T23:40:43Z")

</div>

The problem is when indexing into your custom dataset, not Trainer. If you want help with that, you’ll have to share your code on how it’s built. To debug your problem, try to see if you can do `train_dataset[0]` (or any other index).

---

<div class="post-metadata">

**Author:** ![rbint](https://avatars.discourse-cdn.com/v4/letter/r/85e7bf/32.png) [@rbint](https://discuss.huggingface.co/u/rbint)\
**Post date:** [September 1, 2020, 12:02am UTC](https://discuss.huggingface.co/t/issues-with-trainer-class-on-custom-dataset/946/3 "2020-09-01T00:02:42Z")

</div>

Thank you for the quick response. I spent the past few minutes typing up a long response with my code, explaining how I built the custom dataset and passed it along to the trainer when I noticed a typo in my code. Turns out I was sending just the list of emails, not the encoded ones, when I called the subclassed dataset function to create my dataset. trainer.train() appears to be functioning correctly now and I will update this if I run into any further issues during this finetuning loop or with my results.

---

<div class="post-metadata">

**Author:** ![adisrini11](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@adisrini11](https://discuss.huggingface.co/u/adisrini11)\
**Post date:** [June 14, 2023, 7:43am UTC](https://discuss.huggingface.co/t/issues-with-trainer-class-on-custom-dataset/946/4 "2023-06-14T07:43:29Z")

</div>

Hi,  
I am also facing the same problem, it will be very helpful if you can help me because I am still a beginner at using LLMs and hugging face. I am pasting my code below.

from transformers import AutoModelForSequenceClassification, AutoTokenizer, TrainingArguments, Trainer  
import datasets  
import torch as t  
from transformers import DataCollatorWithPadding  
import evaluate  
import numpy as np

dataset = datasets.load\_dataset(“tweet\_eval”,“emotion”)  
x\_train = dataset[“train”][“text”]  
y\_train = dataset[“train”][“label”]  
x\_test = dataset[“test”][“text”]  
y\_test = dataset[“test”][“label”]  
def load\_LLM(llm, device):  
num\_labels = 4  
id2label = {0: “Anger”, 1: “Joy”, 2: “Optimism”, 3: “Sadness”}  
label2id = {“Anger”: 0, “Joy”: 1, “Optimism”: 2, “Sadness”:3}  
model = AutoModelForSequenceClassification.from\_pretrained(llm,num\_labels=num\_labels,id2label=id2label, label2id=label2id)  
model.to(device)  
tokenizer = AutoTokenizer.from\_pretrained(llm)  
return model, tokenizer  
llm = “EleutherAI/gpt-neo-2.7B”  
device = t.device(‘cuda’ if t.cuda.is\_available() else ‘cpu’)  
model,tokenizer = load\_LLM(llm,device)  
train\_inputs = tokenizer(x\_train,truncation=True)  
test\_inputs = tokenizer(x\_test,truncation=True)  
data\_collator = DataCollatorWithPadding(tokenizer=tokenizer)  
accuracy = evaluate.load(“accuracy”)  
def compute\_metrics(eval\_pred):  
predictions, labels = eval\_pred  
predictions = np.argmax(predictions, axis=1)  
return accuracy.compute(predictions=predictions, references=labels)  
training\_args = TrainingArguments(  
output\_dir=“my\_awesome\_model”,  
learning\_rate=2e-5,  
per\_device\_train\_batch\_size=16,  
per\_device\_eval\_batch\_size=16,  
num\_train\_epochs=2,  
weight\_decay=0.01,  
evaluation\_strategy=“epoch”,  
save\_strategy=“epoch”,  
load\_best\_model\_at\_end=True,  
push\_to\_hub=True,  
)

trainer = Trainer(  
model=model,  
args=training\_args,  
train\_dataset=train\_inputs,  
eval\_dataset=test\_inputs,  
tokenizer=tokenizer,  
data\_collator=data\_collator,  
compute\_metrics=compute\_metrics,  
)

trainer.train()

I am getting the exact same error and I don’t know how to resolve it.