# Llama-2-7b-chat fine-tuning

**URL:** <https://discuss.huggingface.co/t/llama-2-7b-chat-fine-tuning/54877>\
**Category:** Models\
**Created:** [September 13, 2023, 3:53pm UTC](https://discuss.huggingface.co/t/llama-2-7b-chat-fine-tuning/54877 "2023-09-13T15:53:49Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![tozster](https://avatars.discourse-cdn.com/v4/letter/t/bcef8e/32.png) [@tozster](https://discuss.huggingface.co/u/tozster)\
**Post date:** [September 13, 2023, 3:53pm UTC](https://discuss.huggingface.co/t/llama-2-7b-chat-fine-tuning/54877/1 "2023-09-13T15:53:50Z")

</div>

Hi! I’m interested in fine-tuning the Llama-2 chat model to be able to chat about my local .txt documents. I’m familiar with the format required for inference using the [INST]… formatting, and have been somewhat successful in using the context window to provide the model information about domain specific information, but the max context length of ~4k is too limiting.

My primary question is, how should I format the data I have to fine tune the chat model? Do I convert it to a question and answer format and create a string using the same format as I do for inference for training? Any guidance on this topic would be very much appreciated.

---

<div class="post-metadata">

**Author:** ![alfraser](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alfraser/32/23102_2.png) [@alfraser](https://discuss.huggingface.co/u/alfraser)\
**Post date:** [January 26, 2024, 12:05pm UTC](https://discuss.huggingface.co/t/llama-2-7b-chat-fine-tuning/54877/2 "2024-01-26T12:05:21Z")

</div>

Hi @tozster - did you ever solve this one for yourself? I have been following various online guides for this, but I still cannot seem to get consistent fine-tuned results. I get behaviour where sometimes the responses are quite reasonable, but then often the responses include a large number of the delimiter tokens ([INST], \<\<SYS\>\> etc…). I have played with both the training format and the inference prompt but I cannot get decent consistent behaviour.

Did you ever manage to get through this and get a good result for your files?

---

<div class="post-metadata">

**Author:** ![wefewfkjwhfe](https://avatars.discourse-cdn.com/v4/letter/w/8edcca/32.png) [@wefewfkjwhfe](https://discuss.huggingface.co/u/wefewfkjwhfe)\
**Post date:** [February 22, 2024, 1:13pm UTC](https://discuss.huggingface.co/t/llama-2-7b-chat-fine-tuning/54877/3 "2024-02-22T13:13:47Z")

</div>

I have have the same issue where the responses include these tokens.

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [February 22, 2024, 4:15pm UTC](https://discuss.huggingface.co/t/llama-2-7b-chat-fine-tuning/54877/4 "2024-02-22T16:15:43Z")

</div>

Hi,

Similar to inference, fine-tuning also requires chat templates.

See my [demo notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/Mistral/Supervised_fine_tuning_(SFT)_of_an_LLM_using_Hugging_Face_tooling.ipynb) on fine-tuning Mistral-7B (I’m using the chat templates when preparing the data for the model). It would be similar for Llama-2-7B, although LLaMa uses a different chat template compared to Mistral.

---

<div class="post-metadata">

**Author:** ![Sunny111](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sunny111/32/26266_2.png) [@Sunny111](https://discuss.huggingface.co/u/Sunny111)\
**Post date:** [April 26, 2024, 11:24pm UTC](https://discuss.huggingface.co/t/llama-2-7b-chat-fine-tuning/54877/5 "2024-04-26T23:24:25Z")

</div>

Hey I am curious have anybody tried fine-tuning LlaMA-7B on any summarization tasks? or just some regular ranking tasks? I am trying to figure out the standard setup.