# KV cache sizing

**URL:** <https://discuss.huggingface.co/t/kv-cache-sizing/52010>\
**Category:** 🤗Transformers\
**Created:** [August 24, 2023, 5:21pm UTC](https://discuss.huggingface.co/t/kv-cache-sizing/52010 "2023-08-24T17:21:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![sruthikesh](https://avatars.discourse-cdn.com/v4/letter/s/ee59a6/32.png) [@sruthikesh](https://discuss.huggingface.co/u/sruthikesh)\
**Post date:** [August 24, 2023, 5:21pm UTC](https://discuss.huggingface.co/t/kv-cache-sizing/52010/1 "2023-08-24T17:21:17Z")

</div>

# KV cache sizing

Need help with understanding of KV cache

1. Is the KV cache fixed to a specific size (max\_seq\_length)? From generation utils it appears there is no limit to the size. IIUC `past_key_values` are always are forwarding without limiting the size
2. If yes, how are the older entries of KV cache replaced?

Thanks