# GPU memory calculator

**URL:** <https://discuss.huggingface.co/t/gpu-memory-calculator/95302>\
**Category:** 🤗Accelerate\
**Created:** [July 3, 2024, 9:43am UTC](https://discuss.huggingface.co/t/gpu-memory-calculator/95302 "2024-07-03T09:43:55Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ron5569](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@ron5569](https://discuss.huggingface.co/u/ron5569)\
**Post date:** [July 3, 2024, 9:43am UTC](https://discuss.huggingface.co/t/gpu-memory-calculator/95302/1 "2024-07-03T09:43:55Z")

</div>

I am looking for a calculator tool that can estimate the amount of memory a GPU machine instance will use in advance. Specifically, I’m interested in the following configuration:

- FSDP Zero 2

- Model: e.g., LLama3 8B

-Lora

- Max tokens: 8K

- 16-bit precision

- Gradient checkpointing

- No quantization

Any guidance or recommendations would be greatly appreciated! Thank you.

Edit: I know about hf tool ([Understanding how big of a model can fit on your machine](https://huggingface.co/docs/accelerate/v0.31.0/en/usage_guides/model_size_estimator)) but this is rudimentary and lacks many of parameters such as max sequence token, lora, …

---

<div class="post-metadata">

**Author:** ![lance42](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lance42/32/28647_2.png) [@lance42](https://discuss.huggingface.co/u/lance42)\
**Post date:** [July 4, 2024, 11:34am UTC](https://discuss.huggingface.co/t/gpu-memory-calculator/95302/2 "2024-07-04T11:34:41Z")

</div>

This one has a bit more than the HF one: [Tech 42](https://www.tech42consulting.com/resources/inferentia-calculator). There’s an option to add a custom model.

---

<div class="post-metadata">

**Author:** ![nikita200](https://avatars.discourse-cdn.com/v4/letter/n/dfb087/32.png) [@nikita200](https://discuss.huggingface.co/u/nikita200)\
**Post date:** [July 5, 2024, 12:33pm UTC](https://discuss.huggingface.co/t/gpu-memory-calculator/95302/3 "2024-07-05T12:33:48Z")

</div>

You can try the Python code for the calculations. Use the formulas with all parameters:

total\_memory=(model\_memory+activation\_memory+lora\_memory)Xcheckpointing\_factorXfsdp\_factor