# Why model.generate does encoding multiple times

**URL:** <https://discuss.huggingface.co/t/why-model-generate-does-encoding-multiple-times/23413>\
**Category:** 🤗Transformers\
**Created:** [September 20, 2022, 7:49pm UTC](https://discuss.huggingface.co/t/why-model-generate-does-encoding-multiple-times/23413 "2022-09-20T19:49:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![yuxiang630](https://avatars.discourse-cdn.com/v4/letter/y/9de0a6/32.png) [@yuxiang630](https://discuss.huggingface.co/u/yuxiang630)\
**Post date:** [September 20, 2022, 7:49pm UTC](https://discuss.huggingface.co/t/why-model-generate-does-encoding-multiple-times/23413/1 "2022-09-20T19:49:16Z")

</div>

Hi community, I have a question on the implementation of `model.generate`. From my understanding, an encoder-decoder-based model first encodes the input context, and then autoregressively decodes the outputs, which means it should only encode once and decodes multiple times. [This blog post](https://huggingface.co/blog/encoder-decoder) also supports my point of view. However, the detailed implementation of the high-level API `model.generate` is strange. At [this line](https://github.com/huggingface/transformers/blob/578e18e0028c26d8988413ef270da1b292e87dc2/src/transformers/generation_utils.py#L1786), the `input_ids` is appended with new decoded output, which is then sent to the entire encoder-decoder model. Is it on purpose? Thanks!

---

<div class="post-metadata">

**Author:** ![yuxiang630](https://avatars.discourse-cdn.com/v4/letter/y/9de0a6/32.png) [@yuxiang630](https://discuss.huggingface.co/u/yuxiang630)\
**Post date:** [September 20, 2022, 8:03pm UTC](https://discuss.huggingface.co/t/why-model-generate-does-encoding-multiple-times/23413/2 "2022-09-20T20:03:33Z")

</div>

I think I got it. If it is an encoder-decoder based model, the algorithm would perform encoding first and save the result to `model_kwargs`, and at the meantime `input_ids` would always be the decoder’s output.