# Is Eval and Validation same in Trainer API?

**URL:** <https://discuss.huggingface.co/t/is-eval-and-validation-same-in-trainer-api/9948>\
**Category:** Beginners\
**Created:** [September 13, 2021, 7:01pm UTC](https://discuss.huggingface.co/t/is-eval-and-validation-same-in-trainer-api/9948 "2021-09-13T19:01:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![moma1820](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/moma1820/32/4141_2.png) [@moma1820](https://discuss.huggingface.co/u/moma1820)\
**Post date:** [September 13, 2021, 7:01pm UTC](https://discuss.huggingface.co/t/is-eval-and-validation-same-in-trainer-api/9948/1 "2021-09-13T19:01:00Z")

</div>

Hi, I am a bit confused if the eval dataset parameter is used during the training.

```auto
#Trainer itself.
trainer = Trainer(
    model,
    args,
    train_dataset=tokenized_datasets_train,
    eval_dataset=tokenized_datasets_val,
    tokenizer=tokenizer,
    compute_metrics=compute_metrics,
    data_collator = data_collator_
)

```

is the eval\_dataset only used when we do trainer.evaluate() ?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [September 14, 2021, 12:42am UTC](https://discuss.huggingface.co/t/is-eval-and-validation-same-in-trainer-api/9948/2 "2021-09-14T00:42:50Z")

</div>

Yes, it’s the default dataset used for that method (which will be used if you pass an `eval_strategy` to evaluate every epoch or n steps).

---

<div class="post-metadata">

**Author:** ![moma1820](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/moma1820/32/4141_2.png) [@moma1820](https://discuss.huggingface.co/u/moma1820)\
**Post date:** [September 14, 2021, 8:16am UTC](https://discuss.huggingface.co/t/is-eval-and-validation-same-in-trainer-api/9948/3 "2021-09-14T08:16:27Z")

</div>

Okay perfect, then i will not put my test set over there Thanks! @sgugger

Although i want to get the f1 score of my model on the test set, do you know if there is a metric api from hugginface i could use?

If so could you please link me a small script?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [September 14, 2021, 12:12pm UTC](https://discuss.huggingface.co/t/is-eval-and-validation-same-in-trainer-api/9948/4 "2021-09-14T12:12:03Z")

</div>

You can look at any of the [examples](https://github.com/huggingface/transformers/tree/master/examples/pytorch) or the [course section on the Trainer](https://huggingface.co/course/chapter3/3?fw=pt) on how to du this using a `compute_metrics` function.

---

<div class="post-metadata">

**Author:** ![moma1820](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/moma1820/32/4141_2.png) [@moma1820](https://discuss.huggingface.co/u/moma1820)\
**Post date:** [September 14, 2021, 12:36pm UTC](https://discuss.huggingface.co/t/is-eval-and-validation-same-in-trainer-api/9948/5 "2021-09-14T12:36:43Z")

</div>

Huge thanks again!

Have a nice day