# \`BertEmbeddings\` contains positional embedding?

**URL:** <https://discuss.huggingface.co/t/bertembeddings-contains-positional-embedding/28442>\
**Category:** 🤗Transformers\
**Created:** [December 26, 2022, 10:36am UTC](https://discuss.huggingface.co/t/bertembeddings-contains-positional-embedding/28442 "2022-12-26T10:36:15Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![NimaBoscarino](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nimaboscarino/32/4358_2.png) [@NimaBoscarino](https://discuss.huggingface.co/u/NimaBoscarino)\
**Post date:** [December 26, 2022, 7:19pm UTC](https://discuss.huggingface.co/t/bertembeddings-contains-positional-embedding/28442/2 "2022-12-26T19:19:09Z")

</div>

Hello there! `BertEmbeddings` has the positional embeddings for you: [transformers/modeling\_bert.py at 31d452c68b34c2567b62924ee0df40a83cbc52d5 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/31d452c68b34c2567b62924ee0df40a83cbc52d5/src/transformers/models/bert/modeling_bert.py#L187)

We use a vanilla nn.Embedding layer instead of the sin/cos positional encoding (more on that here: [Why positional embeddings are implemented as just simple embeddings? - #6 by yjernite](https://discuss.huggingface.co/t/why-positional-embeddings-are-implemented-as-just-simple-embeddings/585/6)). If you want to override that and use sinusoidal embeddings though, you can follow the tip here: [How to use custom positional embedding while fine tuning Bert - #2 by JinKing](https://discuss.huggingface.co/t/how-to-use-custom-positional-embedding-while-fine-tuning-bert/7992/2)

Hope that helps! Cheers!

---

_[View the full topic](https://discuss.huggingface.co/t/bertembeddings-contains-positional-embedding/28442)._