# Do you train all layers when fine-tuning T5?

**URL:** <https://discuss.huggingface.co/t/do-you-train-all-layers-when-fine-tuning-t5/1034>\
**Category:** Beginners\
**Created:** [September 8, 2020, 4:04am UTC](https://discuss.huggingface.co/t/do-you-train-all-layers-when-fine-tuning-t5/1034 "2020-09-08T04:04:19Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [September 8, 2020, 6:04am UTC](https://discuss.huggingface.co/t/do-you-train-all-layers-when-fine-tuning-t5/1034/2 "2020-09-08T06:04:39Z")

</div>

I haven’t seen much experiments for this, but IMO it’s better to fine-tune the whole model.

Also when you pass `labels` argument to `T5ForConditionalGeneration`'s `forward` method then it calculates the loss for you and returns it as the first value in the returned `tuple` .

And you can use the `finetune.py` script [here](https://github.com/huggingface/transformers/tree/master/examples/seq2seq) to fine-tuning T5 and other seq2seq models

See this thread [T5 Finetuning Tips](https://discuss.huggingface.co/t/t5-finetuning-tips/684)

---

_[View the full topic](https://discuss.huggingface.co/t/do-you-train-all-layers-when-fine-tuning-t5/1034)._