# How could different model be trained same way?

**URL:** <https://discuss.huggingface.co/t/how-could-different-model-be-trained-same-way/8698>\
**Category:** Beginners\
**Created:** [July 26, 2021, 2:40pm UTC](https://discuss.huggingface.co/t/how-could-different-model-be-trained-same-way/8698 "2021-07-26T14:40:32Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![sangil](https://avatars.discourse-cdn.com/v4/letter/s/3be4f8/32.png) [@sangil](https://discuss.huggingface.co/u/sangil)\
**Post date:** [July 26, 2021, 2:40pm UTC](https://discuss.huggingface.co/t/how-could-different-model-be-trained-same-way/8698/1 "2021-07-26T14:40:33Z")

</div>

hello.  
I am doing fine-tuning and pre-training bert and electra. But I really don’t understand how could those two models be trained same way.  
Let’s say bert.  
bert is trained using Masked Language Model(MLM) and next sentence prediction so logically if I fine-tuning bert using MLM, bert embedding would be more appropriate than before.  
Problem is electra.  
transformers documentation suggest same fine-tuning step like bert.  
electra does not use MLM instead using discriminators. electra predict which token is fake and which token is true. similar with gan.  
So logically If i try to fine-tuning electra, I guess i have to train electra which token is fake and which is true.  
But transformers documentation suggest MLM. Even normal fine tuning also same.  
model( input\_ids, attention\_mask, token\_type\_ids ,…etc)  
this are the parameter which electra model get.  
but what i think is this kinds of parameter should work for bert not electra.  
anyone pls help me.