# Fine-tuning don't work / bad results

**URL:** <https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788>\
**Category:** Beginners\
**Created:** [December 6, 2023, 1:15pm UTC](https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788 "2023-12-06T13:15:27Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![SuperOvni](https://avatars.discourse-cdn.com/v4/letter/s/45deac/32.png) [@SuperOvni](https://discuss.huggingface.co/u/SuperOvni)\
**Post date:** [December 6, 2023, 1:15pm UTC](https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788/1 "2023-12-06T13:15:27Z")

</div>

Hello,  
I need your advice.

I’m trying to fine-tune Llama2 7B (base Model) for a single task. My goal is the following: for a given text it should replace some words by others if they are present in the text, otherwise it should only return the text.

However, when I make the inference, the task is not understood, the model simply repeats the instruction and returns a bit of nonsense.

I used the huggingface trainer and formulated my prompt as ###INSTRUCTION: {} ###INPUT: {} ###RESPONSE: {}.

I used the basic Lora parameters found in all the tutorials. I have about 2000 prompt data.

Does anyone have any advice or explanation for my problem?

Thanks for your help.

---

<div class="post-metadata">

**Author:** ![gKhera](https://avatars.discourse-cdn.com/v4/letter/g/898d66/32.png) [@gKhera](https://discuss.huggingface.co/u/gKhera)\
**Post date:** [March 14, 2024, 1:45am UTC](https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788/2 "2024-03-14T01:45:52Z")

</div>

Facing the same issue here.

---

<div class="post-metadata">

**Author:** ![fonsecovizk](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/fonsecovizk/32/25400_2.png) [@fonsecovizk](https://discuss.huggingface.co/u/fonsecovizk)\
**Post date:** [April 11, 2024, 11:45am UTC](https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788/3 "2024-04-11T11:45:52Z")

</div>

Same issue here.

---

<div class="post-metadata">

**Author:** ![Anuranjan87](https://avatars.discourse-cdn.com/v4/letter/a/a183cd/32.png) [@Anuranjan87](https://discuss.huggingface.co/u/Anuranjan87)\
**Post date:** [April 22, 2024, 7:27am UTC](https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788/4 "2024-04-22T07:27:47Z")

</div>

You can use fine-tune-ai dot com

---

<div class="post-metadata">

**Author:** ![coeusk](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/coeusk/32/37850_2.png) [@coeusk](https://discuss.huggingface.co/u/coeusk)\
**Post date:** [December 26, 2024, 6:49pm UTC](https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788/5 "2024-12-26T18:49:19Z")

</div>

why does this thread have 1.3k views and no solutions? are we finetuning it badly?

---

<div class="post-metadata">

**Author:** ![nagme12](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nagme12/32/39215_2.png) [@nagme12](https://discuss.huggingface.co/u/nagme12)\
**Post date:** [January 15, 2025, 1:12pm UTC](https://discuss.huggingface.co/t/fine-tuning-dont-work-bad-results/64788/6 "2025-01-15T13:12:52Z")

</div>

I am facing the same issue and find this site, maybe it could be beneficial for you. [Top 7 Large Language Model Fine-tuning Mistakes to Avoid | Entry Point AI](https://www.entrypointai.com/blog/large-language-model-fine-tuning-mistakes/)