# Get the predictions using DataCollator For Completion OnlyLM after fine-tuning Llama2 using SFT trainer

**URL:** <https://discuss.huggingface.co/t/get-the-predictions-using-datacollator-for-completion-onlylm-after-fine-tuning-llama2-using-sft-trainer/62108>\
**Category:** 🤗Transformers\
**Created:** [November 13, 2023, 6:34pm UTC](https://discuss.huggingface.co/t/get-the-predictions-using-datacollator-for-completion-onlylm-after-fine-tuning-llama2-using-sft-trainer/62108 "2023-11-13T18:34:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hessa](https://avatars.discourse-cdn.com/v4/letter/h/9f8e36/32.png) [@Hessa](https://discuss.huggingface.co/u/Hessa)\
**Post date:** [November 13, 2023, 6:34pm UTC](https://discuss.huggingface.co/t/get-the-predictions-using-datacollator-for-completion-onlylm-after-fine-tuning-llama2-using-sft-trainer/62108/1 "2023-11-13T18:34:37Z")

</div>

I am in the process of fine-tuning Llama2 using SFT trainer and quantization using Lora.  
my dataset is composed of questions structured like:

```
<s>[INST] 
<<SYS>> Please select the correct answer from the given multiple Options based on the given Context: <</SYS>>  
 Context: Abrasion is another type of mechanical weathering. With abrasion, one rock bumps against another rock. Gravity causes abrasion as a rock tumbles down a slope. Moving water causes abrasion it moves rocks so that they bump against one another (Figure 9.3). Strong winds cause abrasion by blasting sand against rock surfaces.  
 Question: Gravity causes erosion by all of the following except \
 Options:(A) glaciers (B) moving air (C) flowing water (D) mass movement  
 Answer: [/INST] D </s>

```

And I am currently using DataCollatorForCompletionOnlyLM to compute the loss based on the predicted answers.

Do I provide the context, question and options as an instruction\_template?

```
instruction_template = "</SYS>>\n\n Context:" response_template = "Answer: [/INST]" collator = DataCollatorForCompletionOnlyLM(instruction_template=instruction_template, response_template=response_template, tokenizer=tokenizer, mlm=False)

```

OR

```
response_template = "Answer: [/INST]" collator = DataCollatorForCompletionOnlyLM(response_template=response_template, tokenizer=tokenizer, mlm=False)

```

and do I provide the answers with the val dataset? or else how would evaluate the model and get the accuracy or other evaluation metrics?