# Dataset format which will be given SFTTrainer

**URL:** <https://discuss.huggingface.co/t/dataset-format-which-will-be-given-sfttrainer/92430>\
**Category:** 🤗Transformers\
**Created:** [June 16, 2024, 2:35pm UTC](https://discuss.huggingface.co/t/dataset-format-which-will-be-given-sfttrainer/92430 "2024-06-16T14:35:22Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![tjrlwjd1](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@tjrlwjd1](https://discuss.huggingface.co/u/tjrlwjd1)\
**Post date:** [June 16, 2024, 2:35pm UTC](https://discuss.huggingface.co/t/dataset-format-which-will-be-given-sfttrainer/92430/1 "2024-06-16T14:35:22Z")

</div>

As I mentioned in title, I’m curious what is exact format of dataset which will be given SFTTrainer?

I want to finetune a LLM in Code Generation task.  
Below is the my ideal situation.

When model is given the below:  
{Natural Language Problem(e.g. Code competition problem),  
I want to make model answer(code) like the below:  
{Solution Code}.

I made the dataset(Dataset class) with columns (“prompt”, “completion”).  
Then, I look at train\_dataset after pass SFTTrainer.

\<start token for tokenizer\> \<INST\> {Natural Language Problem} \</INST\> {Solution Code}\ .

I am wondering that is it wrong??  
If it is, please let me know how to make a dataset, especially column names.

In addition, I tried to use DataCollatorForCompletionOnly. But it didn’t work (always train loss = 0)