# Data preparation for T5 model

**URL:** <https://discuss.huggingface.co/t/data-preparation-for-t5-model/38615>\
**Category:** Beginners\
**Created:** [May 4, 2023, 4:39am UTC](https://discuss.huggingface.co/t/data-preparation-for-t5-model/38615 "2023-05-04T04:39:32Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![CRini](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@CRini](https://discuss.huggingface.co/u/CRini)\
**Post date:** [May 4, 2023, 4:39am UTC](https://discuss.huggingface.co/t/data-preparation-for-t5-model/38615/1 "2023-05-04T04:39:32Z")

</div>

Hi there!

I want to further a T5 model and have been looking around and came across [transformers/run\_t5\_mlm\_flax.py at main · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/main/examples/flax/language-modeling/run_t5_mlm_flax.py#L232)

I am not sure which part of this huge script do I need for my data preparation. I have an input file that I want to process. I am trying to understand how to find the masking function that adds the extra\_tokens\_ids.  
Is it better to try this whole script with my input rather than trying to pull out the data processing script?

Thank you would appreciate help!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [May 7, 2023, 5:36pm UTC](https://discuss.huggingface.co/t/data-preparation-for-t5-model/38615/2 "2023-05-07T17:36:10Z")

</div>

Hi,

I think these lines are relevant for isolating the data preparation: [transformers/run\_t5\_mlm\_flax.py at ef42c2c487260c2a0111fa9d17f2507d84ddedea · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/ef42c2c487260c2a0111fa9d17f2507d84ddedea/examples/flax/language-modeling/run_t5_mlm_flax.py#L667-L711).