# Multi-GPU Training using Accelerate: RAM Issue Leading to Failure

**URL:** <https://discuss.huggingface.co/t/multi-gpu-training-using-accelerate-ram-issue-leading-to-failure/97542>\
**Category:** 🤗Accelerate\
**Created:** [July 16, 2024, 2:09am UTC](https://discuss.huggingface.co/t/multi-gpu-training-using-accelerate-ram-issue-leading-to-failure/97542 "2024-07-16T02:09:25Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![nwirandx](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nwirandx](https://discuss.huggingface.co/u/nwirandx)\
**Post date:** [July 16, 2024, 2:09am UTC](https://discuss.huggingface.co/t/multi-gpu-training-using-accelerate-ram-issue-leading-to-failure/97542/1 "2024-07-16T02:09:25Z")

</div>

I am currently using `accelerate` for multi-GPU training. Running `python train.py` on a single GPU works fine. However, when I execute the following command, the RAM usage keeps increasing until it eventually fails:

`CUDA_VISIBLE_DEVICES=2,3 accelerate launch --num_processes 2 train.py`

Execution Environment:  
accelerate : 0.28.0  
python : 3.8.10  
cuda : nvcc -V : 12.1 nvidia-smi 12.0  
pytorch : 2.1.0+cu121

and I use in docker.