# Inserting custom layer after embeddings layer in BERT

**URL:** <https://discuss.huggingface.co/t/inserting-custom-layer-after-embeddings-layer-in-bert/44577>\
**Category:** 🤗Transformers\
**Created:** [June 26, 2023, 9:27am UTC](https://discuss.huggingface.co/t/inserting-custom-layer-after-embeddings-layer-in-bert/44577 "2023-06-26T09:27:44Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![riyag2805](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@riyag2805](https://discuss.huggingface.co/u/riyag2805)\
**Post date:** [June 26, 2023, 9:27am UTC](https://discuss.huggingface.co/t/inserting-custom-layer-after-embeddings-layer-in-bert/44577/1 "2023-06-26T09:27:45Z")

</div>

Hi,

I want to add a fully connected dense layer of dimension 256, right after the initial BERT embedding layer, i.e. between the embedding layer and the first encoder layer. Is there an easier way to go about this, other than making changes in the code of the [original BertModel class](https://huggingface.co/transformers/v3.5.1/_modules/transformers/modeling_bert.html)?