# Dataset expected by Trainer

**URL:** <https://discuss.huggingface.co/t/dataset-expected-by-trainer/148>\
**Category:** Beginners\
**Created:** [July 9, 2020, 3:31pm UTC](https://discuss.huggingface.co/t/dataset-expected-by-trainer/148 "2020-07-09T15:31:01Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Isabella](https://avatars.discourse-cdn.com/v4/letter/i/ecb155/32.png) [@Isabella](https://discuss.huggingface.co/u/Isabella)\
**Post date:** [July 9, 2020, 3:31pm UTC](https://discuss.huggingface.co/t/dataset-expected-by-trainer/148/1 "2020-07-09T15:31:01Z")

</div>

Hello everyone,  
I am rewriting some old code to use the new tokenizer syntax and the Trainer class but I believe I am missing something.

This is how I am building the training dataset to be passed to the Trainer constructor:

```auto
    encoded_texts = tokenizer(texts, padding = True, truncation = True, return_tensors = 'pt')
    labels = torch.tensor(labels)
    dataset = TensorDataset(encoded_texts['input_ids'], encoded_texts['attention_mask'], labels)

```

Can you please help me understand what I am doing wrong/missing? When I run `trainer.train()` I get the following error:  
`vars() argument must have __dict__ attribute`

Thanks in advance! 😊

---

<div class="post-metadata">

**Author:** ![prajjwal1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/prajjwal1/32/2246_2.png) [@prajjwal1](https://discuss.huggingface.co/u/prajjwal1)\
**Post date:** [July 9, 2020, 3:54pm UTC](https://discuss.huggingface.co/t/dataset-expected-by-trainer/148/2 "2020-07-09T15:54:03Z")

</div>

I’ll give you the full picture.

### The workflow:

You create an instance of `GlueDataset(data_args, tokenizer)`. Then you pass it to `Trainer(...)` class. In trainer, you also pass in `default_data_collator`. The reason is that `GlueDataset` return `InputExample` which is HF specific and cannot be used by Pytorch dataloader directly. So the `default_data_collator` takes in `List[InputExamples]` and returns a dict. This dict is then used by the dataloader.

So in trainer, if you pass `default_data_collator` with `TensorDataset`, it won’t work directly (That’s why you’re getting the error). This error is raised when dataloader will pass the batch to `default_data_collator`. I’d suggest using the default Pytorch `collate_fn` with your `TensorDataset`, it would work just fine.

#### One more additional thing:

Make sure the dataloader returns the dict with same key values `forward` method expects.  
Inside `_training_step`, you’ll pass `inputs` to the function, and then after the inputs are passed kept on gpu, the function does:  
`output = model(**inputs)`  
In this case, the keyword arguments have to match. In case, they don’t, you can inherit from `Trainer` and redefine your own method.

I hope this answers your question.

---

<div class="post-metadata">

**Author:** ![Isabella](https://avatars.discourse-cdn.com/v4/letter/i/ecb155/32.png) [@Isabella](https://discuss.huggingface.co/u/Isabella)\
**Post date:** [July 10, 2020, 4:28pm UTC](https://discuss.huggingface.co/t/dataset-expected-by-trainer/148/3 "2020-07-10T16:28:11Z")

</div>

Thank you very much for your help!

I tried different approaches; currently I am testing a solution that involves writing my own `data_collator` to pass to the `Trainer` instance:

```auto
def dummy_data_collector(features):
    batch = {}
    batch['input_ids'] = torch.stack([f[0] for f in features])
    batch['attention_mask'] = torch.stack([f[1] for f in features])
    batch['labels'] = torch.stack([f[2] for f in features])
    
    return batch

```

This seems to be working, but I will need to do some more testing 🙂

---

<div class="post-metadata">

**Author:** ![prajjwal1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/prajjwal1/32/2246_2.png) [@prajjwal1](https://discuss.huggingface.co/u/prajjwal1)\
**Post date:** [July 11, 2020, 9:02am UTC](https://discuss.huggingface.co/t/dataset-expected-by-trainer/148/4 "2020-07-11T09:02:42Z")

</div>

I’m glad it worked out.

---

<div class="post-metadata">

**Author:** ![Buckeyes2019](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@Buckeyes2019](https://discuss.huggingface.co/u/Buckeyes2019)\
**Post date:** [September 28, 2020, 8:56am UTC](https://discuss.huggingface.co/t/dataset-expected-by-trainer/148/5 "2020-09-28T08:56:49Z")

</div>

I am not sure I understand how the Trainer class identifies the target from the features. In the given example in the documentation, Trainer is passed “train\_dataset” with no X and Y specified. Is this already assigned somewhere on the dataset object?

from transformers import BartForSequenceClassification, Trainer, TrainingArguments  
model = BartForSequenceClassification.from\_pretrained(“facebook/bart-large-cnn”)

training\_args = TrainingArguments(  
output\_dir=’./results’, # output directory  
num\_train\_epochs=3, # total # of training epochs  
per\_device\_train\_batch\_size=2, # batch size per device during training  
per\_device\_eval\_batch\_size=2, # batch size for evaluation  
warmup\_steps=100, # number of warmup steps for learning rate scheduler  
weight\_decay=0.01, # strength of weight decay  
logging\_dir=’./logs’, # directory for storing logs)

trainer = Trainer(  
model=model, # the instantiated 🤗 Transformers model to be trained  
args=training\_args, # training arguments, defined above  
train\_dataset=train\_df, # training dataset\*\*  
\*\* eval\_dataset=eval\_df)\*\*

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [September 28, 2020, 2:32pm UTC](https://discuss.huggingface.co/t/dataset-expected-by-trainer/148/6 "2020-09-28T14:32:54Z")

</div>

All 🤗 Transformers models will return the loss when fed with the inputs and labels (usually named `labels`). The `Trainer` thus expects each element of the dataset you pass to be a dictionary with all the inputs the model expects to return the loss (including those `labels`).