# Instruction tuning llm **URL:** \ **Category:** Beginners\ **Created:** [January 1, 2024, 8:45am UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597 "2024-01-01T08:45:26Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![ron5569](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@ron5569](https://discuss.huggingface.co/u/ron5569)\ **Post date:** [January 1, 2024, 8:45am UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/1 "2024-01-01T08:45:26Z")
I want to fine-tune a LLM with an instructions dataset, which consists of pairs of prompts and completions. I have seen a lot of tutorials on how to fine-tune LLMs with supervised datasets. Almost all of them use Trainer or SFTTrainer from Hugging Face. **The strange thing that shocked me is that there is no difference between this fine-tuning and the pretraining process; in both cases, the model tries to predict the next token for both the prompt and the completion.** Intuitively, I would prefer to backpropagate only the tokens of the completion and not the prompt itself. In fact, I believe the next token prediction should only start at the completion stage. Does that make sense? Does anyone know of any library that can perform training as I expect? > [@Fine tune with SFTTrainer](https://discuss.huggingface.co/t/fine-tune-with-sfttrainer/67311/2): > > Hi, So SFT (supervised fine-tuning) is called supervised since we’re collecting the data from humans. However we’re still training the model using the same cross-entropy loss as during pre-training (i.e. predicting the next token). We now just make it more likely that the model will generate a useful completion given an instruction like “what are 10 things to do in London”, then the model should learn to generate “in London, you can visit (…)” for instance. Since the model is still trained to… ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [January 1, 2024, 6:15pm UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/2 "2024-01-01T18:15:05Z")
Hi, That’s supported in the TRL library using the `DataCollatorForCompletionOnlyLM` class: [Supervised Fine-tuning Trainer](https://huggingface.co/docs/trl/sft_trainer#train-on-completions-only) ---
**Author:** ![rangehow](https://avatars.discourse-cdn.com/v4/letter/r/3da27b/32.png) [@rangehow](https://discuss.huggingface.co/u/rangehow)\ **Post date:** [January 3, 2024, 3:33am UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/3 "2024-01-03T03:33:11Z")
The requirement you want may be needs to deal with in data preprocess procedure. To the best of my knowledge,you can manually replace prompts part in lable with -100 which is a special token that would be ignore loss calculation by torch backend(most third party llm finetune repo do things like this, like llama-recipes officially supported by llama or ‘llamafactory’ a very famous llm factory). ---
**Author:** ![cungnlp](https://avatars.discourse-cdn.com/v4/letter/c/e8c25b/32.png) [@cungnlp](https://discuss.huggingface.co/u/cungnlp)\ **Post date:** [January 23, 2024, 4:49pm UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/4 "2024-01-23T16:49:45Z")
Can you help me check how training data is generated when entering the “text” column of the dataset into the trainer() function? I mean test with code. thanks ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [January 23, 2024, 9:31pm UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/5 "2024-01-23T21:31:45Z")
@cungnlp this can be checked by doing [trainer.get\_train\_dataloader](https://github.com/huggingface/transformers/blob/5f81266fb0b444f896fea9322d7c41368abd8526/src/transformers/trainer.py#L778). You can then check some samples of the dataloader: ```auto train_dataloader = trainer.get_train_dataloader() batch = next(iter(train_dataloader)) print(batch) ``` ---
**Author:** ![cungnlp](https://avatars.discourse-cdn.com/v4/letter/c/e8c25b/32.png) [@cungnlp](https://discuss.huggingface.co/u/cungnlp)\ **Post date:** [January 24, 2024, 3:03am UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/6 "2024-01-24T03:03:09Z")
Thank you for your help! Vào Th 4, 24 thg 1, 2024 vào lúc 04:41 Niels Rogge via Hugging Face Forums \<[notifications@hellohellohello.discoursemail.com](mailto:notifications@hellohellohello.discoursemail.com)\> đã viết: ---
**Author:** ![cungnlp](https://avatars.discourse-cdn.com/v4/letter/c/e8c25b/32.png) [@cungnlp](https://discuss.huggingface.co/u/cungnlp)\ **Post date:** [January 29, 2024, 12:34pm UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/7 "2024-01-29T12:34:21Z")
I checked, but it seems the dataset doesn’t look like “shift right one token”. Can you explain to me why? By the way, can you give me the code on how to add your own dataset with 3 columns: input\_ids, attention\_mask, label. We look forward to receiving your feedback! Vào Th 4, 24 thg 1, 2024 vào lúc 10:03 Nguyen Cung \<[cungmachinelearning@gmail.com](mailto:cungmachinelearning@gmail.com)\> đã viết: ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [May 5, 2024, 8:39am UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/8 "2024-05-05T08:39:56Z")
Hi, For LLMs in the Transformers library, the `labels` are typically just a copy of the `input_ids` (with padding tokens replaced by -100, the ignore index of the cross-entropy loss in PyTorch). The model will internally shift the labels one position to the right. ---
**Author:** ![cungnlp](https://avatars.discourse-cdn.com/v4/letter/c/e8c25b/32.png) [@cungnlp](https://discuss.huggingface.co/u/cungnlp)\ **Post date:** [May 8, 2024, 6:51am UTC](https://discuss.huggingface.co/t/instruction-tuning-llm/67597/9 "2024-05-08T06:51:26Z")
Thanks for your help, I understood the problem. Wish you an effective working day. Vào CN, 5 thg 5, 2024 vào lúc 15:50 Niels Rogge via Hugging Face Forums \<[notifications@hellohellohello.discoursemail.com](mailto:notifications@hellohellohello.discoursemail.com)\> đã viết: