# Finetuning cost estimation

**URL:** <https://discuss.huggingface.co/t/finetuning-cost-estimation/57016>\
**Category:** Languages at Hugging Face\
**Created:** [October 1, 2023, 5:39pm UTC](https://discuss.huggingface.co/t/finetuning-cost-estimation/57016 "2023-10-01T17:39:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![alielfilali01](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alielfilali01/32/59086_2.png) [@alielfilali01](https://discuss.huggingface.co/u/alielfilali01)\
**Post date:** [October 1, 2023, 5:39pm UTC](https://discuss.huggingface.co/t/finetuning-cost-estimation/57016/1 "2023-10-01T17:39:32Z")

</div>

Hi everyone, i’am starting a research project for arabic and the idea is to finetune multiple SOTA models then compare them , the thing is we are short in terms of budget so i would like to know if there is a way to estimate the cost giving the model size and the dataset size (assuming it will be done on an A100 80GB)

i.e : is there a magical formula to estimate the cost ?

---

<div class="post-metadata">

**Author:** ![MattiLinnanvuori](https://avatars.discourse-cdn.com/v4/letter/m/ec9cab/32.png) [@MattiLinnanvuori](https://discuss.huggingface.co/u/MattiLinnanvuori)\
**Post date:** [October 2, 2023, 6:08pm UTC](https://discuss.huggingface.co/t/finetuning-cost-estimation/57016/2 "2023-10-02T18:08:01Z")

</div>

Hugging Face has a model memory usage calculator that I guess can be used in calculating the number of A100 80Gs needed. [https://huggingface.co/spaces/hf-accelerate/model-memory-usage](https://huggingface.co/spaces/hf-accelerate/model-memory-usage)

---

<div class="post-metadata">

**Author:** ![alielfilali01](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alielfilali01/32/59086_2.png) [@alielfilali01](https://discuss.huggingface.co/u/alielfilali01)\
**Post date:** [October 2, 2023, 8:42pm UTC](https://discuss.huggingface.co/t/finetuning-cost-estimation/57016/3 "2023-10-02T20:42:17Z")

</div>

Not very useful in my case ! I’m more interested to know that given a model size let’s 560M params (8% of 7B since we are using LoRA) and a dataset size of let’s say 2B tokens, how much would it take to fine-tune that model for 1 epoch on a single A100 80gb ? I don’t even know if it’s possible to do such calculation but the HF AutoTrain ([https://ui.autotrain.huggingface.co](https://ui.autotrain.huggingface.co)) gives a cost estimation before launching the training so i assumed it is possible after all 🙃

@merve can you guide to who from the team can help with that ?