# Compute VRAM size for Text2Text text generation

**URL:** <https://discuss.huggingface.co/t/compute-vram-size-for-text2text-text-generation/128673>\
**Category:** 🤗Transformers\
**Created:** [December 1, 2024, 12:15pm UTC](https://discuss.huggingface.co/t/compute-vram-size-for-text2text-text-generation/128673 "2024-12-01T12:15:39Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![langueneers](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/langueneers/32/36714_2.png) [@langueneers](https://discuss.huggingface.co/u/langueneers)\
**Post date:** [December 1, 2024, 12:15pm UTC](https://discuss.huggingface.co/t/compute-vram-size-for-text2text-text-generation/128673/1 "2024-12-01T12:15:40Z")

</div>

Hello,

How would I go about calculating / estimating the VRAM that I need to fine-tune models such as flan-t5-large or mt5-large for seq2seq text generation while taking into consideration the input and output sequence size? let us assume I have an input size of 1000 and a target text size of 2000. The model I want to use is MT5-large. How do I compute the amount of GPU memory needed with a minimal batch\_size of 1? How would this increase with every increment to the batch-size?  
I tried using mt5-small on a dual-GPU setup with 2 x 20 GB but had a memory overflow despite mixed precision and batch-size of 1 due to the number of prediction steps. When I reduced the target text size to a fee dozen tokens only, everything worked perfectly. A few dozen tokens are simply not enough…