# Different padding behaviour of data collator

**URL:** <https://discuss.huggingface.co/t/different-padding-behaviour-of-data-collator/103658>\
**Category:** 🤗Transformers\
**Created:** [August 23, 2024, 7:38am UTC](https://discuss.huggingface.co/t/different-padding-behaviour-of-data-collator/103658 "2024-08-23T07:38:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![pagein](https://avatars.discourse-cdn.com/v4/letter/p/aeb1de/32.png) [@pagein](https://discuss.huggingface.co/u/pagein)\
**Post date:** [August 23, 2024, 7:38am UTC](https://discuss.huggingface.co/t/different-padding-behaviour-of-data-collator/103658/1 "2024-08-23T07:38:00Z")

</div>

Hi all,  
I have encountered, that DataCollatorForLanguageModeling has different behaviour regarding padding, when dicts or just tokens are passed to it.

1. If a sequence of Mapping is passed, then collator [calls](https://github.com/huggingface/transformers/blob/d806fa3e92289876e01ab19c9e19e9264ea1c1a1/src/transformers/data/data_collator.py#L806) pad method without checking (inside pad\_without\_fast\_tokenizer\_warning function).

2. Otherwise, it calls \_torch\_collate\_batch, who [checks](https://github.com/huggingface/transformers/blob/d806fa3e92289876e01ab19c9e19e9264ea1c1a1/src/transformers/data/data_collator.py#L440) where padding is necessary.

So my question is whether this difference is intentional?

The reason why I am asking, is that when I do a LM-training, I already have packed sequences of tokens (so the padding is not required), but some tokenizers (LLaMA-3 for example) does not have the padding token, and the code fails in one case, but not in the other.