# How to use decoder generation upon features and not token ids?

**URL:** <https://discuss.huggingface.co/t/how-to-use-decoder-generation-upon-features-and-not-token-ids/21370>\
**Category:** 🤗Transformers\
**Created:** [August 8, 2022, 8:40am UTC](https://discuss.huggingface.co/t/how-to-use-decoder-generation-upon-features-and-not-token-ids/21370 "2022-08-08T08:40:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![proy](https://avatars.discourse-cdn.com/v4/letter/p/a87d85/32.png) [@proy](https://discuss.huggingface.co/u/proy)\
**Post date:** [August 8, 2022, 8:40am UTC](https://discuss.huggingface.co/t/how-to-use-decoder-generation-upon-features-and-not-token-ids/21370/1 "2022-08-08T08:40:53Z")

</div>

Hi,

I am trying to use the generate function on a BertLM model using beam search using embeddings and not upon the inputs ids and I can’t make it work. The usecase is as follows: I want to insert features from external model for the generation process (for example, as done in frozen’s paper). How can I do it? is it possible?