# Running generate while evaluating test set?

**URL:** <https://discuss.huggingface.co/t/running-generate-while-evaluating-test-set/49927>\
**Category:** 🤗Transformers\
**Created:** [August 8, 2023, 7:59am UTC](https://discuss.huggingface.co/t/running-generate-while-evaluating-test-set/49927 "2023-08-08T07:59:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![shivance](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shivance/32/18366_2.png) [@shivance](https://discuss.huggingface.co/u/shivance)\
**Post date:** [August 8, 2023, 7:59am UTC](https://discuss.huggingface.co/t/running-generate-while-evaluating-test-set/49927/1 "2023-08-08T07:59:43Z")

</div>

I’m trying to finetune a T5 model using Huggingface `Trainer` class which does simple evaluation and calculates the loss.

I want to run generations upto certain length at end of each evaluation on test set and store it in a file. I think I would need to override trainer to create a CustomTrainer , but I’m not sure how can I do it.

Could anyone help?