# Loading pretrained weights into model for sequence classifcation

**URL:** <https://discuss.huggingface.co/t/loading-pretrained-weights-into-model-for-sequence-classifcation/421>\
**Category:** Beginners\
**Created:** [July 22, 2020, 12:09am UTC](https://discuss.huggingface.co/t/loading-pretrained-weights-into-model-for-sequence-classifcation/421 "2020-07-22T00:09:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rsvarma](https://avatars.discourse-cdn.com/v4/letter/r/a183cd/32.png) [@rsvarma](https://discuss.huggingface.co/u/rsvarma)\
**Post date:** [July 22, 2020, 12:09am UTC](https://discuss.huggingface.co/t/loading-pretrained-weights-into-model-for-sequence-classifcation/421/1 "2020-07-22T00:09:35Z")

</div>

Hi there,

I am a bit confused as to how saving models works. I have been pretraining with an ElectraForPreTraining model and have saved it using save\_pretrained(). However, I am now interested in fine-tuning the model on a sequence classification task, so I figure I would have to load my pretrained weights into an ElectraForSequenceClassification model, except the ElectraForPreTraining model has a different head on the discriminator than the ElectraForSequenceClassification and I am not sure how to handle that correctly. Does the from\_pretrained() method handle this out of the box and if not what would I have to do?

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [July 22, 2020, 4:44am UTC](https://discuss.huggingface.co/t/loading-pretrained-weights-into-model-for-sequence-classifcation/421/2 "2020-07-22T04:44:00Z")

</div>

Hi @rsvarma the `.from_pretrained` method handles this. You can do

```auto
ElectraForSequenceClassification.from_pretrained("path to your pre-trained electra model")

```

---

<div class="post-metadata">

**Author:** ![rsvarma](https://avatars.discourse-cdn.com/v4/letter/r/a183cd/32.png) [@rsvarma](https://discuss.huggingface.co/u/rsvarma)\
**Post date:** [July 22, 2020, 7:56am UTC](https://discuss.huggingface.co/t/loading-pretrained-weights-into-model-for-sequence-classifcation/421/3 "2020-07-22T07:56:31Z")

</div>

Hi @valhalla,

Thanks, thats what I needed to know!