# Reusing cached context to generate multiple sequences?

**URL:** <https://discuss.huggingface.co/t/reusing-cached-context-to-generate-multiple-sequences/34684>\
**Category:** 🤗Transformers\
**Created:** [March 26, 2023, 5:49am UTC](https://discuss.huggingface.co/t/reusing-cached-context-to-generate-multiple-sequences/34684 "2023-03-26T05:49:45Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![kz919](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kz919/32/31282_2.png) [@kz919](https://discuss.huggingface.co/u/kz919)\
**Post date:** [March 26, 2023, 5:49am UTC](https://discuss.huggingface.co/t/reusing-cached-context-to-generate-multiple-sequences/34684/1 "2023-03-26T05:49:45Z")

</div>

I wonder if it’s possible to make generate() use cached context to generate multiple sequences without iteratively computing the context, especially for large model that can only run batch size of 1, hence I can not use num\_output\_sequence, which is effectively increasing the batch size.

The current hack is to recomputing the context again everytime, which seems to be very unnecessary.

---

<div class="post-metadata">

**Author:** ![kz919](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kz919/32/31282_2.png) [@kz919](https://discuss.huggingface.co/u/kz919)\
**Post date:** [March 26, 2023, 9:55pm UTC](https://discuss.huggingface.co/t/reusing-cached-context-to-generate-multiple-sequences/34684/2 "2023-03-26T21:55:47Z")

</div>

Never mind. I implemented it myself.