# Getting error - trainer.train()

**URL:** <https://discuss.huggingface.co/t/getting-error-trainer-train/76813>\
**Category:** 🤗Transformers\
**Created:** [March 11, 2024, 2:59am UTC](https://discuss.huggingface.co/t/getting-error-trainer-train/76813 "2024-03-11T02:59:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![DeepanSrinivasan](https://avatars.discourse-cdn.com/v4/letter/d/71c47a/32.png) [@DeepanSrinivasan](https://discuss.huggingface.co/u/DeepanSrinivasan)\
**Post date:** [March 11, 2024, 2:59am UTC](https://discuss.huggingface.co/t/getting-error-trainer-train/76813/1 "2024-03-11T02:59:11Z")

</div>

Hello, I am trying to fine tune a GPT2 model with my data. While i am use train() method getting below error.

KeyError: ‘Invalid key. Only three types of key are available: (1) string, (2) integers for backend Encoding, and (3) slices for data subsetting.’

below is my code

cleaned\_text =“”

from transformers import GPT2Tokenizer

tokenizer = GPT2Tokenizer.from\_pretrained(‘gpt2’)

# Add a new padding token to the tokenizer

tokenizer.add\_special\_tokens({‘pad\_token’: ‘[PAD]’})

# Tokenize the text using GPT-2 tokenizer

tokenized\_text = tokenizer(cleaned\_text, truncation=True, padding=True)

training\_args = TrainingArguments(  
output\_dir=“./gpt2-finetuned”,  
overwrite\_output\_dir=True,  
num\_train\_epochs=3,  
per\_device\_train\_batch\_size=2,  
save\_steps=10\_000,  
save\_total\_limit=2,  
)

trainer = Trainer(  
model=model,  
args=training\_args,  
train\_dataset=tokenized\_text,

```
tokenizer=tokenizer,

```

)

trainer.train()

---

<div class="post-metadata">

**Author:** ![wonderfullymade](https://avatars.discourse-cdn.com/v4/letter/w/ce7236/32.png) [@wonderfullymade](https://discuss.huggingface.co/u/wonderfullymade)\
**Post date:** [April 23, 2024, 1:02pm UTC](https://discuss.huggingface.co/t/getting-error-trainer-train/76813/2 "2024-04-23T13:02:27Z")

</div>

hello, how were you able to fix this issue?

---

<div class="post-metadata">

**Author:** ![op97](https://avatars.discourse-cdn.com/v4/letter/o/c68b51/32.png) [@op97](https://discuss.huggingface.co/u/op97)\
**Post date:** [April 24, 2024, 9:06pm UTC](https://discuss.huggingface.co/t/getting-error-trainer-train/76813/3 "2024-04-24T21:06:32Z")

</div>

any updates on this please ?  
i’m facing the same problem when fine-tuning mistral 7b

---

<div class="post-metadata">

**Author:** ![Chahnwoo](https://avatars.discourse-cdn.com/v4/letter/c/a88e57/32.png) [@Chahnwoo](https://discuss.huggingface.co/u/Chahnwoo)\
**Post date:** [April 25, 2024, 12:05am UTC](https://discuss.huggingface.co/t/getting-error-trainer-train/76813/4 "2024-04-25T00:05:25Z")

</div>

Could you provide the full error log?

---

<div class="post-metadata">

**Author:** ![ghitak](https://avatars.discourse-cdn.com/v4/letter/g/3d9bf3/32.png) [@ghitak](https://discuss.huggingface.co/u/ghitak)\
**Post date:** [June 3, 2024, 9:53am UTC](https://discuss.huggingface.co/t/getting-error-trainer-train/76813/5 "2024-06-03T09:53:40Z")

</div>

hello, how you solve it ?