# Setting PyTorch CUDA memory configuration while using HF transformers

**URL:** <https://discuss.huggingface.co/t/setting-pytorch-cuda-memory-configuration-while-using-hf-transformers/11529>\
**Category:** 🤗Transformers\
**Created:** [November 9, 2021, 8:33pm UTC](https://discuss.huggingface.co/t/setting-pytorch-cuda-memory-configuration-while-using-hf-transformers/11529 "2021-11-09T20:33:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abhishek-P](https://avatars.discourse-cdn.com/v4/letter/a/51bf81/32.png) [@Abhishek-P](https://discuss.huggingface.co/u/Abhishek-P)\
**Post date:** [November 9, 2021, 8:33pm UTC](https://discuss.huggingface.co/t/setting-pytorch-cuda-memory-configuration-while-using-hf-transformers/11529/1 "2021-11-09T20:33:42Z")

</div>

Seeing below error when I am trying to additional train XLM using transformers library.

```auto
RuntimeError: CUDA out of memory. Tried to allocate 978.00 MiB (GPU 0; 11.92 GiB total c
remote: Compressing objects: 100% (1/1), done. │capacity; 10.83 GiB already allocated; 442.62 MiB free; 10.99 GiB reserved in total by Py
remote: Total 3 (delta 1), reused 3 (delta 1), pack-reused 0 │Torch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid
Unpacking objects: 100% (3/3), done. │fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

```

I have set the environment variable as suggested in [CUDA semantics — PyTorch 2.1 documentation](https://pytorch.org/docs/stable/notes/cuda.html) Memory Management section

> 11/09/2021 01:21:12 PM Pytorch CUDA conf max\_split\_size\_mb:40

```auto
(pair-nlp) [abpu9500@login12 continued-pretraining]$ pip freeze| grep "transformers"
~ │transformers==4.12.3

(pair-nlp) [abpu9500@login12 continued-pretraining]$ pip freeze| grep "torch"
~ │torch==1.10.0

```

---

<div class="post-metadata">

**Author:** ![ThrinathMphasis](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@ThrinathMphasis](https://discuss.huggingface.co/u/ThrinathMphasis)\
**Post date:** [November 23, 2022, 10:46am UTC](https://discuss.huggingface.co/t/setting-pytorch-cuda-memory-configuration-while-using-hf-transformers/11529/3 "2022-11-23T10:46:18Z")

</div>

It still shows the same error:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.50 GiB (GPU 0; 14.56 GiB total capacity; 11.18 GiB already allocated; 1.34 GiB free; 11.66 GiB reserved in total by PyTorch) If reserved memory is \>\> allocated memory try setting max\_split\_size\_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH\_CUDA\_ALLOC\_CONF

env:

environ({‘LS\_COLORS’: ‘rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:_.tar=01;31:_.tgz=01;31:_.arc=01;31:_.arj=01;31:_.taz=01;31:_.lha=01;31:_.lz4=01;31:_.lzh=01;31:_.lzma=01;31:_.tlz=01;31:_.txz=01;31:_.tzo=01;31:_.t7z=01;31:_.zip=01;31:_.z=01;31:_.Z=01;31:_.dz=01;31:_.gz=01;31:_.lrz=01;31:_.lz=01;31:_.lzo=01;31:_.xz=01;31:_.zst=01;31:_.tzst=01;31:_.bz2=01;31:_.bz=01;31:_.tbz=01;31:_.tbz2=01;31:_.tz=01;31:_.deb=01;31:_.rpm=01;31:_.jar=01;31:_.war=01;31:_.ear=01;31:_.sar=01;31:_.rar=01;31:_.alz=01;31:_.ace=01;31:_.zoo=01;31:_.cpio=01;31:_.7z=01;31:_.rz=01;31:_.cab=01;31:_.wim=01;31:_.swm=01;31:_.dwm=01;31:_.esd=01;31:_.jpg=01;35:_.jpeg=01;35:_.mjpg=01;35:_.mjpeg=01;35:_.gif=01;35:_.bmp=01;35:_.pbm=01;35:_.pgm=01;35:_.ppm=01;35:_.tga=01;35:_.xbm=01;35:_.xpm=01;35:_.tif=01;35:_.tiff=01;35:_.png=01;35:_.svg=01;35:_.svgz=01;35:_.mng=01;35:_.pcx=01;35:_.mov=01;35:_.mpg=01;35:_.mpeg=01;35:_.m2v=01;35:_.mkv=01;35:_.webm=01;35:_.ogm=01;35:_.mp4=01;35:_.m4v=01;35:_.mp4v=01;35:_.vob=01;35:_.qt=01;35:_.nuv=01;35:_.wmv=01;35:_.asf=01;35:_.rm=01;35:_.rmvb=01;35:_.flc=01;35:_.avi=01;35:_.fli=01;35:_.flv=01;35:_.gl=01;35:_.dl=01;35:_.xcf=01;35:_.xwd=01;35:_.yuv=01;35:_.cgm=01;35:_.emf=01;35:_.ogv=01;35:_.ogx=01;35:_.aac=00;36:_.au=00;36:_.flac=00;36:_.m4a=00;36:_.mid=00;36:_.midi=00;36:_.mka=00;36:_.mp3=00;36:_.mpc=00;36:_.ogg=00;36:_.ra=00;36:_.wav=00;36:_.oga=00;36:_.opus=00;36:_.spx=00;36:_.xspf=00;36:’, ‘LESSCLOSE’: ‘/usr/bin/lesspipe %s %s’, ‘LANG’: ‘C.UTF-8’, ‘SUDO\_GID’: ‘1002’, ‘OLDPWD’: ‘/home/user’, ‘USERNAME’: ‘root’, ‘SUDO\_COMMAND’: ‘/bin/su’, ‘VIRTUAL\_ENV’: ‘/home/user/project’, ‘PYTORCH\_CUDA\_ALLOC\_CONF’: ‘max\_split\_size\_mb:40’, ‘USER’: ‘root’, ‘PWD’: ‘/home/user/project’, ‘HOME’: ‘/root’, ‘SUDO\_USER’: ‘user’, ‘SUDO\_UID’: ‘1001’, ‘MAIL’: ‘/var/mail/root’, ‘SHELL’: ‘/bin/bash’, ‘TERM’: ‘xterm-256color’, ‘SHLVL’: ‘1’, ‘LOGNAME’: ‘root’, ‘PATH’: ‘/home/user/proj/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin’, ‘PS1’: '(project) \[\e]0;\u@\h: \w\a\]${debian\_chroot:+($debian\_chroot)}\u@\h:\w\$ ', ‘LESSOPEN’: ‘| /usr/bin/lesspipe %s’, ‘\_’: ‘/home/user/project/bin/accelerate’, ‘USE\_CPU’: ‘False’, ‘USE\_MPS\_DEVICE’: ‘False’, ‘MIXED\_PRECISION’: ‘no’, ‘OMP\_NUM\_THREADS’: ‘1’})