# Fine-tuning queries

**URL:** <https://discuss.huggingface.co/t/fine-tuning-queries/142017>\
**Category:** Beginners\
**Created:** [February 20, 2025, 1:25pm UTC](https://discuss.huggingface.co/t/fine-tuning-queries/142017 "2025-02-20T13:25:41Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![PratikJadon](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@PratikJadon](https://discuss.huggingface.co/u/PratikJadon)\
**Post date:** [February 20, 2025, 1:25pm UTC](https://discuss.huggingface.co/t/fine-tuning-queries/142017/1 "2025-02-20T13:25:41Z")

</div>

I am new to fine-tuning, but i have read many post so i have many queries please help me.

When fine-tuning LLM what is mostly done-\>  
text= input + output

now input and labels are given to llm as input\_ids of text.  
Now my queries are:

1. Am i correct that in labels the ids corresponding to pad\_token\_id are set -100 to prevent loss calculation and there are two ways to fine-tune either to calculate loss on both prompt + completion or on just completion.
2. SFTTrainer class by default compute loss on both but using datacollators for completetion we can compute loss on only completion right ?
3. What about Trainer Class what it does by default ? calculate loss on just completion or both.
4. Another Question is that what if i don’t provide data collator and just myself format input with special tokens and tokenize these inputs without setting pad token ids to -100 in label what will happen in both cases of SFTTrainer class and Trainer class ?
5. Do we have to mannualy shift label ? Also where i see how shifting of these labels is done internally cause i am trying to find in github but didn’t find.

What i am doing was:

1. Created a prompt appended input and output in it.
2. added special tokens and tokenzied it.
3. Set labels as input ids like model\_input[‘labels’]= model\_inputs[‘input\_ids’]  
without shifting it and without setting padding tokens to -100.
4. passing these tokenized dataset directly to Trainer Class.