# VRAM keeps increasing during sequential llama2-13b inferencing

**URL:** <https://discuss.huggingface.co/t/vram-keeps-increasing-during-sequential-llama2-13b-inferencing/65330>\
**Category:** Models\
**Created:** [December 11, 2023, 9:58am UTC](https://discuss.huggingface.co/t/vram-keeps-increasing-during-sequential-llama2-13b-inferencing/65330 "2023-12-11T09:58:01Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![xerxes01](https://avatars.discourse-cdn.com/v4/letter/x/73ab20/32.png) [@xerxes01](https://discuss.huggingface.co/u/xerxes01)\
**Post date:** [December 11, 2023, 9:58am UTC](https://discuss.huggingface.co/t/vram-keeps-increasing-during-sequential-llama2-13b-inferencing/65330/1 "2023-12-11T09:58:01Z")

</div>

VRAM consumption starts with 26 GB, then balloons upto 40 GB during sequential inferencing. I wish to operate multiple instances of the service on a single 80GB gpu (non quantized), want to know if there is any way to disable/limit the caching? Or if there’s a command to free up cached GPU memory without impacting the model latency?

---

<div class="post-metadata">

**Author:** ![jatingarg546](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@jatingarg546](https://discuss.huggingface.co/u/jatingarg546)\
**Post date:** [July 15, 2024, 5:36am UTC](https://discuss.huggingface.co/t/vram-keeps-increasing-during-sequential-llama2-13b-inferencing/65330/2 "2024-07-15T05:36:13Z")

</div>

We are facing the similar problem, have you found any fix yet? It would be really helpful.