# torch.cuda.OutOfMemoryError

**URL:** <https://discuss.huggingface.co/t/torch-cuda-outofmemoryerror/45664>\
**Category:** 🤗Transformers\
**Created:** [July 5, 2023, 3:05am UTC](https://discuss.huggingface.co/t/torch-cuda-outofmemoryerror/45664 "2023-07-05T03:05:58Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![oshani](https://avatars.discourse-cdn.com/v4/letter/o/b77776/32.png) [@oshani](https://discuss.huggingface.co/u/oshani)\
**Post date:** [July 5, 2023, 3:05am UTC](https://discuss.huggingface.co/t/torch-cuda-outofmemoryerror/45664/1 "2023-07-05T03:05:58Z")

</div>

Hi All, I keep keep getting this error while running transformers train.train():

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 GiB (GPU 0; 39.50 GiB total capacity; 38.72 GiB already allocated; 225.12 MiB free; 38.72 GiB reserved in total by PyTorch) If reserved memory is \>\> allocated memory try setting max\_split\_size\_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH\_CUDA\_ALLOC\_CONF

Following some advise online, I tried setting PYTORCH\_CUDA\_ALLOC\_CONF to “garbage\_collection\_threshold:0.6,max\_split\_size\_mb:128” and also adding

torch.cuda.empty\_cache()

to my code but that doesn’t help.

So any ideas? My GPU is A100 with 40GB of memory and I use cuda-11.4 and torch-2.0.1

Thanks,

Oren