# Multi-task instruction fine-tuning

**URL:** <https://discuss.huggingface.co/t/multi-task-instruction-fine-tuning/50310>\
**Category:** Amazon SageMaker\
**Created:** [August 10, 2023, 5:13pm UTC](https://discuss.huggingface.co/t/multi-task-instruction-fine-tuning/50310 "2023-08-10T17:13:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jorgeutd](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jorgeutd/32/39292_2.png) [@Jorgeutd](https://discuss.huggingface.co/u/Jorgeutd)\
**Post date:** [August 10, 2023, 5:13pm UTC](https://discuss.huggingface.co/t/multi-task-instruction-fine-tuning/50310/1 "2023-08-10T17:13:42Z")

</div>

Hello everyone,

I have a question regarding supervise instruction fine-tuning LLM models. Most of the examples and documentation I’ve come across focus on single-task instruction fine-tuning. I am wondering if it’s possible to perform multi-task instruction fine-tuning, for instance, using two or three different tasks concurrently with some of these models, such as Llama2, Falcon, etc.?

Any insights or guidance would be greatly appreciated. Thank you!

---

<div class="post-metadata">

**Author:** ![CKeibel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ckeibel/32/55416_2.png) [@CKeibel](https://discuss.huggingface.co/u/CKeibel)\
**Post date:** [February 2, 2024, 1:17pm UTC](https://discuss.huggingface.co/t/multi-task-instruction-fine-tuning/50310/2 "2024-02-02T13:17:01Z")

</div>

Hey!

It’s been a while since you asked your question but I thought I’d answer it in case someone comes across this post later.

It’s actually a good idea to fine tune a model with several different tasks, so the model doesn’t lose the skills it had before.  
For example, if you train a model exclusively on language translation, it may lose the ability to summarize texts. (catastrophic forgetting)  
You can easily fine-tune your model by creating a data set that contains different tasks.

For additional information, I can highly recommend the book “Generative AI on AWS” by Chris Fregly or the Coursera course based on it, also by Chris Fregly.