# Custom modification on transformers **URL:** \ **Category:** 🤗Transformers\ **Created:** [June 13, 2024, 8:42am UTC](https://discuss.huggingface.co/t/custom-modification-on-transformers/91930 "2024-06-13T08:42:34Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![ronit33](https://avatars.discourse-cdn.com/v4/letter/r/e79b87/32.png) [@ronit33](https://discuss.huggingface.co/u/ronit33)\ **Post date:** [June 13, 2024, 8:42am UTC](https://discuss.huggingface.co/t/custom-modification-on-transformers/91930/1 "2024-06-13T08:42:34Z")
Are there any resources in huggingface on adding or removing layers in an already build model like bert or roberta. I want to replace some layers in a transformer model for my project. for example changing the self attention layer to some other kind of attention i also want to add my own layer inside the transformer as well. Thank you. ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [June 13, 2024, 12:24pm UTC](https://discuss.huggingface.co/t/custom-modification-on-transformers/91930/2 "2024-06-13T12:24:14Z")
Hi, The Transformers library is not really aimed for this use case. It’s not meant to be a modular toolbox but rather aimed at people who want to use and fine-tune pre-trained models. Of course, you could fork the Transformers library and tweak [modeling\_bert.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/bert/modeling_bert.py) yourself, but if you want to hack around I’d recommend checking out other projects such as the ones from [Phil Wang](https://github.com/lucidrains) or [GitHub - facebookresearch/xformers: Hackable and optimized Transformers building blocks, supporting a composable construction.](https://github.com/facebookresearch/xformers).