# Loading model from pytorch\_pretrained\_bert into transformers library

**URL:** <https://discuss.huggingface.co/t/loading-model-from-pytorch-pretrained-bert-into-transformers-library/6124>\
**Category:** 🤗Transformers\
**Created:** [May 11, 2021, 1:23pm UTC](https://discuss.huggingface.co/t/loading-model-from-pytorch-pretrained-bert-into-transformers-library/6124 "2021-05-11T13:23:06Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![twistedstats](https://avatars.discourse-cdn.com/v4/letter/t/dec6dc/32.png) [@twistedstats](https://discuss.huggingface.co/u/twistedstats)\
**Post date:** [May 11, 2021, 1:23pm UTC](https://discuss.huggingface.co/t/loading-model-from-pytorch-pretrained-bert-into-transformers-library/6124/1 "2021-05-11T13:23:06Z")

</div>

I have a pretrained BERT model for a classification task trained on the pytorch\_pretrained\_bert library. I would like to use the initial weights from this model for further training with transformers library.

When I try to load this model, I get the following runtime error. Does anyone know how to resolve this?

model = BertClassification(weight\_path= pretrained\_weights\_path, num\_labels=num\_labels)  
state\_dict = torch.load(fine\_tuned\_weight\_path,map\_location=‘cuda:0’)  
model.load\_state\_dict(state\_dict)

RuntimeError: Error(s) in loading state\_dict for BertClassification:  
Missing key(s) in state\_dict: “bert.embeddings.position\_ids”.

Thanks very much.

---

<div class="post-metadata">

**Author:** ![mithunthakkar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mithunthakkar/32/2250_2.png) [@mithunthakkar](https://discuss.huggingface.co/u/mithunthakkar)\
**Post date:** [July 11, 2021, 3:34pm UTC](https://discuss.huggingface.co/t/loading-model-from-pytorch-pretrained-bert-into-transformers-library/6124/2 "2021-07-11T15:34:08Z")

</div>

Hi. Please help with this. I am facing the same issue.

---

<div class="post-metadata">

**Author:** ![evilying](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@evilying](https://discuss.huggingface.co/u/evilying)\
**Post date:** [January 5, 2022, 8:23pm UTC](https://discuss.huggingface.co/t/loading-model-from-pytorch-pretrained-bert-into-transformers-library/6124/3 "2022-01-05T20:23:30Z")

</div>

Hi. This is probably caused by the transformer verison. You might downgrade your transformer version from 4.4 to 2.8 with **pip install transformers==2.8.0**