# Evaluation without using a Trainer

**URL:** <https://discuss.huggingface.co/t/evaluation-without-using-a-trainer/5542>\
**Category:** Beginners\
**Created:** [April 16, 2021, 4:22am UTC](https://discuss.huggingface.co/t/evaluation-without-using-a-trainer/5542 "2021-04-16T04:22:28Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![david-waterworth](https://avatars.discourse-cdn.com/v4/letter/d/49beb7/32.png) [@david-waterworth](https://discuss.huggingface.co/u/david-waterworth)\
**Post date:** [April 16, 2021, 4:22am UTC](https://discuss.huggingface.co/t/evaluation-without-using-a-trainer/5542/1 "2021-04-16T04:22:28Z")

</div>

Is there a clean way to perform evaluation on a model without using the Trainer?

I have scripts which train and then evaluation my model. I want to perform further evaluation on the trained model - for now I’ve hacked my training script so I create a Trainer but don’t call train() - is there a better way of doing this?

---

<div class="post-metadata">

**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\
**Post date:** [April 16, 2021, 7:51am UTC](https://discuss.huggingface.co/t/evaluation-without-using-a-trainer/5542/2 "2021-04-16T07:51:29Z")

</div>

you’ll need to set up the evaluation dataset / dataloader and write your own evaluation loop to compute the metrics.

you can find examples of this in the _examples/legacy_ directory of the `transformers` repo, e.g. here is the evaluation for question-answering on squad: [transformers/run\_squad.py at 6e1ee47b361f9dc1b6da0104d77b38297042efae · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/6e1ee47b361f9dc1b6da0104d77b38297042efae/examples/legacy/question-answering/run_squad.py#L270)

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [April 16, 2021, 11:31am UTC](https://discuss.huggingface.co/t/evaluation-without-using-a-trainer/5542/3 "2021-04-16T11:31:43Z")

</div>

You can also have a look at our new examples that don’t use a `Trainer` (all the `run_xxx_no_trainer` in the task subfolders of the examples folder of the repo).