# New layer in bert embeddings

**URL:** <https://discuss.huggingface.co/t/new-layer-in-bert-embeddings/13852>\
**Category:** 🤗Transformers\
**Created:** [January 18, 2022, 2:45pm UTC](https://discuss.huggingface.co/t/new-layer-in-bert-embeddings/13852 "2022-01-18T14:45:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ShivaniSri](https://avatars.discourse-cdn.com/v4/letter/s/3ab097/32.png) [@ShivaniSri](https://discuss.huggingface.co/u/ShivaniSri)\
**Post date:** [January 18, 2022, 2:45pm UTC](https://discuss.huggingface.co/t/new-layer-in-bert-embeddings/13852/1 "2022-01-18T14:45:50Z")

</div>

How to I add a new layer in bert embedding  
These are bert embeddings  
(embeddings): BertEmbeddings(  
(word\_embeddings): Embedding(30522, 768, padding\_idx=0)  
(position\_embeddings): Embedding(512, 768)  
(token\_type\_embeddings): Embedding(2, 768)  
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise\_affine=True)  
(dropout): Dropout(p=0.1, inplace=False)  
)  
in which I would like to have  
(embeddings): BertEmbeddings(  
(word\_embeddings): Embedding(30522, 768, padding\_idx=0)  
(position\_embeddings): Embedding(512, 768)  
(token\_type\_embeddings): Embedding(2, 768)  
(custom\_embeddings): Embedding(1,768)  
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise\_affine=True)  
(dropout): Dropout(p=0.1, inplace=False)  
)

My bert\_embeddings= (word\_embeddings)+(position\_embeddings)+(custom\_embeddings)

---

<div class="post-metadata">

**Author:** ![mconati](https://avatars.discourse-cdn.com/v4/letter/m/d2c977/32.png) [@mconati](https://discuss.huggingface.co/u/mconati)\
**Post date:** [April 1, 2022, 11:22pm UTC](https://discuss.huggingface.co/t/new-layer-in-bert-embeddings/13852/2 "2022-04-01T23:22:51Z")

</div>

Did you ever find an answer? I am hoping to do a similar thing