# Train masked language model with custom data on sagemaker

**URL:** <https://discuss.huggingface.co/t/train-masked-language-model-with-custom-data-on-sagemaker/14595>\
**Category:** Amazon SageMaker\
**Created:** [February 10, 2022, 5:47pm UTC](https://discuss.huggingface.co/t/train-masked-language-model-with-custom-data-on-sagemaker/14595 "2022-02-10T17:47:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![enpassant](https://avatars.discourse-cdn.com/v4/letter/e/e95f7d/32.png) [@enpassant](https://discuss.huggingface.co/u/enpassant)\
**Post date:** [February 10, 2022, 5:47pm UTC](https://discuss.huggingface.co/t/train-masked-language-model-with-custom-data-on-sagemaker/14595/1 "2022-02-10T17:47:48Z")

</div>

I have some small text corpus I managed to train on with colab [here](https://gist.github.com/enpassanty/4bac3d1ed5d8995ac3c48050b0c2aca1). It’s basically adapted from the [EsperBerto example](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb).

I looked at the [HF sagemaker training example](https://huggingface.co/docs/sagemaker/train#installation-and-setup) and [this example](https://www.philschmid.de/huggingface-amazon-sagemaker-training-compiler). I cant figure out how to adapt/set the hyper-parameters , estimator params and how to load the correct dataloader and tokenizer files to S3 to do mlm training on SM.

Does anyone have any advice? is there good boilerplate code that can help me?

thanks, eric.

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [February 11, 2022, 3:08pm UTC](https://discuss.huggingface.co/t/train-masked-language-model-with-custom-data-on-sagemaker/14595/2 "2022-02-11T15:08:56Z")

</div>

Hi Eric - this repo has lots of boilerplate code and should be very useful to get started: [notebooks/sagemaker at master · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/tree/master/sagemaker)

Cheers  
Heiko