# Documentation: Transformers Language Modeling Section

**URL:** <https://discuss.huggingface.co/t/documentation-transformers-language-modeling-section/17954>\
**Category:** Beginners\
**Created:** [May 14, 2022, 3:15pm UTC](https://discuss.huggingface.co/t/documentation-transformers-language-modeling-section/17954 "2022-05-14T15:15:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![wquinn](https://avatars.discourse-cdn.com/v4/letter/w/3be4f8/32.png) [@wquinn](https://discuss.huggingface.co/u/wquinn)\
**Post date:** [May 14, 2022, 3:15pm UTC](https://discuss.huggingface.co/t/documentation-transformers-language-modeling-section/17954/1 "2022-05-14T15:15:12Z")

</div>

Hi:

Might you double check the documentation here: [Transformers-Tasks-Language Modeling](https://huggingface.co/docs/transformers/tasks/language_modeling)?

Specifically, the Tensorflow section that deals with DataCollator reads:

"You can use the end of sequence token as the padding token, and set `mlm=False` . This will use the inputs as labels shifted to the right by one element:

The code reads:

```auto
from transformers import DataCollatorForLanguageModeling 
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False, return_tensors="tf")

```

The code doesn’t contain the padding token.

And then:

"For masked language modeling, use the same [DataCollatorForLanguageModeling](https://huggingface.co/docs/transformers/v4.19.0/en/main_classes/data_collator#transformers.DataCollatorForLanguageModeling) except you should specify `mlm_probability` to randomly mask tokens each time you iterate over the data.

And the code reads:

```auto
from transformers import DataCollatorForLanguageModeling

data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False, return_tensors="tf")

```

The code has `mlm=False`.

Also, the notebook associated uses the default DataCollator in the Causal Language Modeling Section. But elsewhere in the documentation (Course: Training a Causal Language Model from Scratch), it reads: By default [`DataCollatorForLanguageModeling`] prepares data for MLM…

Thanks!