# Save custom transformer as PreTrainedModel

**URL:** <https://discuss.huggingface.co/t/save-custom-transformer-as-pretrainedmodel/9768>\
**Category:** Intermediate\
**Created:** [September 7, 2021, 8:36am UTC](https://discuss.huggingface.co/t/save-custom-transformer-as-pretrainedmodel/9768 "2021-09-07T08:36:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![cgutknecht](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cgutknecht/32/2663_2.png) [@cgutknecht](https://discuss.huggingface.co/u/cgutknecht)\
**Post date:** [September 7, 2021, 8:36am UTC](https://discuss.huggingface.co/t/save-custom-transformer-as-pretrainedmodel/9768/1 "2021-09-07T08:36:29Z")

</div>

I have a custom BERT-like model (with modified attention) that I pretrained with PyTorch. Data preparation was done with a Huggingface tokenizer. Now I want to integrate this PyTorch-model in the Huggingface environment so it can be used in pipelines and for finetuning as a PreTrainedModel. How do I generate the necessary config files?

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](https://discuss.huggingface.co/u/BramVanroy)\
**Post date:** [September 7, 2021, 9:34am UTC](https://discuss.huggingface.co/t/save-custom-transformer-as-pretrainedmodel/9768/2 "2021-09-07T09:34:27Z")

</div>

There might be a better way, but I would:

- subclass PretrainedModel with your own class
- load your trained model weights into this new class
- run yourmodel.save\_pretrained() to save the model weights for this class
- now you can do YourCustomModel.from\_pretrained() as it now can use those HF methods