# Fine tuning a model for a specific task

**URL:** <https://discuss.huggingface.co/t/fine-tuning-a-model-for-a-specific-task/45408>\
**Category:** Beginners\
**Created:** [July 3, 2023, 4:23am UTC](https://discuss.huggingface.co/t/fine-tuning-a-model-for-a-specific-task/45408 "2023-07-03T04:23:21Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![oanwar](https://avatars.discourse-cdn.com/v4/letter/o/439d5e/32.png) [@oanwar](https://discuss.huggingface.co/u/oanwar)\
**Post date:** [July 3, 2023, 4:23am UTC](https://discuss.huggingface.co/t/fine-tuning-a-model-for-a-specific-task/45408/1 "2023-07-03T04:23:21Z")

</div>

Hi,

I was wondering if I can finetune a model such as Falcon-7B to perform two tasks at once i.e. answer and translate. Let’s assume I have a dataset with questions in English and answers in Urdu, will fine-tuning be enough, or I will need to train from scratch? The second option will require me to collect a huge dataset, which I really want to avoid.

Thanks for any help and guidance.

BTW limited to an RTX 4090 for now, but a test run of this is working

> **[finetune\_falcon7b\_oasst1\_with\_bnb\_peft.ipynb ·...](https://huggingface.co/dfurman/falcon-7b-chat-oasst1-peft/blob/main/finetune_falcon7b_oasst1_with_bnb_peft.ipynb)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [July 3, 2023, 10:08am UTC](https://discuss.huggingface.co/t/fine-tuning-a-model-for-a-specific-task/45408/2 "2023-07-03T10:08:53Z")

</div>

Hi,

It’s definitely possible to fine-tune an LLM like Falcon-7B on various tasks at the same time (also called multi-task fine-tuning). However, looking at the model card: [tiiuae/falcon-7b · Hugging Face](https://huggingface.co/tiiuae/falcon-7b), this model was pre-trained on English and French only. Hence it probably doesn’t know a lot of Urdu. It might be better to look for an LLM that has Urdu in its pre-training data. You can filter the HF models by “text generation” and “language=urdu”: [Models - Hugging Face](https://huggingface.co/models?pipeline_tag=text-generation&language=ur&sort=trending).

Alternatively, you could also take a look at existing model primarly targeted towards machine translation, such as Meta’s NLLB model series: [NLLB](https://huggingface.co/docs/transformers/model_doc/nllb). This one can translate between 196 languages, with a single model. Hence you could further fine-tune it if you have additional translation pairs.

---

<div class="post-metadata">

**Author:** ![oanwar](https://avatars.discourse-cdn.com/v4/letter/o/439d5e/32.png) [@oanwar](https://discuss.huggingface.co/u/oanwar)\
**Post date:** [July 4, 2023, 1:09am UTC](https://discuss.huggingface.co/t/fine-tuning-a-model-for-a-specific-task/45408/3 "2023-07-04T01:09:41Z")

</div>

Thank you for the response nielsr. The task from English to Urdu will just be a test case, and eventually, we will be training the model for translating some astronomical data to its brief English summary (fingers crossed for that one). I will try to find a model which has been pretrained in multiple languages, as well as small enough to fit on the GPU I have. I should have a few A100s in a couple of months, which will allow me to train/fine-tune bigger models.