# Load Bert model weights to transformers v3 from model trained with transformers v2

**URL:** <https://discuss.huggingface.co/t/load-bert-model-weights-to-transformers-v3-from-model-trained-with-transformers-v2/1807>\
**Category:** 🤗Transformers\
**Created:** [October 29, 2020, 6:11pm UTC](https://discuss.huggingface.co/t/load-bert-model-weights-to-transformers-v3-from-model-trained-with-transformers-v2/1807 "2020-10-29T18:11:06Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![salman](https://avatars.discourse-cdn.com/v4/letter/s/f6c823/32.png) [@salman](https://discuss.huggingface.co/u/salman)\
**Post date:** [October 29, 2020, 6:11pm UTC](https://discuss.huggingface.co/t/load-bert-model-weights-to-transformers-v3-from-model-trained-with-transformers-v2/1807/1 "2020-10-29T18:11:07Z")

</div>

I have a BertModel (bert-base-uncased) trained with transformers==2.1.1 (using PyTorch v1.3)

However, we want to move to transformers v3. I was wondering if it would be possible to load the model weights of that model into transformers v3. Otherwise, we would have to re-train the language model and we want to avoid doing that.

What would be the best way to do this?

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [October 31, 2020, 2:48pm UTC](https://discuss.huggingface.co/t/load-bert-model-weights-to-transformers-v3-from-model-trained-with-transformers-v2/1807/2 "2020-10-31T14:48:39Z")

</div>

It’s possible, you won’t need to anything special, you should be able to load it using `from_pretrained` method.

---

<div class="post-metadata">

**Author:** ![salman](https://avatars.discourse-cdn.com/v4/letter/s/f6c823/32.png) [@salman](https://discuss.huggingface.co/u/salman)\
**Post date:** [November 2, 2020, 7:33pm UTC](https://discuss.huggingface.co/t/load-bert-model-weights-to-transformers-v3-from-model-trained-with-transformers-v2/1807/3 "2020-11-02T19:33:54Z")

</div>

Thanks, I figured what the issue was. We were pickling our models.

I had to load the model and write it out to a directory using `bert_model.save_pretrained(save_dir)` and then load it in v3 using `BertModel.from_pretrained(save_dir)`