# Masked language model for BART (Not BERT)

**URL:** <https://discuss.huggingface.co/t/masked-language-model-for-bart-not-bert/19945>\
**Category:** 🤗Transformers\
**Created:** [July 3, 2022, 11:33am UTC](https://discuss.huggingface.co/t/masked-language-model-for-bart-not-bert/19945 "2022-07-03T11:33:07Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![cog](https://avatars.discourse-cdn.com/v4/letter/c/df788c/32.png) [@cog](https://discuss.huggingface.co/u/cog)\
**Post date:** [July 5, 2022, 10:21am UTC](https://discuss.huggingface.co/t/masked-language-model-for-bart-not-bert/19945/4 "2022-07-05T10:21:08Z")

</div>

hi @ahadda5 ,

is there config setting different?

setting config’s max length size or hidden layer dimension.

[HF bart config docs](https://huggingface.co/docs/transformers/model_doc/bart#transformers.BartConfig)

Also, if you want to build BART for Masked LM, add some last layers to predict hidden layer’s output to your output dimension.

for example, in [BertForMaskedLM](https://huggingface.co/docs/transformers/model_doc/bert#transformers.BertForMaskedLM), class BertLMPredictionHead(nn.Module) fit dimension of hidden to output(vocab)

`self.decoder = nn.Linear(config.hidden_size, config.vocab_size, bias=False)`

you can get hint from [BertForMaskedLM](https://huggingface.co/docs/transformers/model_doc/bert#transformers.BertForMaskedLM) 's layer structure to build BART Masked LM .

hope to help.

---

_[View the full topic](https://discuss.huggingface.co/t/masked-language-model-for-bart-not-bert/19945)._