# Slower train with collator for completion only

**URL:** <https://discuss.huggingface.co/t/slower-train-with-collator-for-completion-only/56600>\
**Category:** 🤗Transformers\
**Created:** [September 27, 2023, 4:10pm UTC](https://discuss.huggingface.co/t/slower-train-with-collator-for-completion-only/56600 "2023-09-27T16:10:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![miguelwon](https://avatars.discourse-cdn.com/v4/letter/m/858c86/32.png) [@miguelwon](https://discuss.huggingface.co/u/miguelwon)\
**Post date:** [September 27, 2023, 4:10pm UTC](https://discuss.huggingface.co/t/slower-train-with-collator-for-completion-only/56600/1 "2023-09-27T16:10:12Z")

</div>

Hi,

I’m trying to fine tune Llama 2 and I don’t want to fine tune in full (instructions + completion), only the completion. Because of that I’m using the `DataCollatorForCompletionOnlyLM` collator.

The completion of by training dataset is a very small sentence and so I was expecting a faster training, because the training would be resumed to predict the next token of completion. Is this what is happing when we use `DataCollatorForCompletionOnlyLM` collator? The problem is that I’m not measuring any improvement in speed. Actually, I think it got slower.

Any comment on this would very appreciated.  
Thanks

---

<div class="post-metadata">

**Author:** ![silcowitz](https://avatars.discourse-cdn.com/v4/letter/s/ebca7d/32.png) [@silcowitz](https://discuss.huggingface.co/u/silcowitz)\
**Post date:** [April 7, 2024, 6:42pm UTC](https://discuss.huggingface.co/t/slower-train-with-collator-for-completion-only/56600/2 "2024-04-07T18:42:11Z")

</div>

Did you find an answer to this?