# Having trouble accelerate on my 2 GPU machine

**URL:** <https://discuss.huggingface.co/t/having-trouble-accelerate-on-my-2-gpu-machine/40835>\
**Category:** Beginners\
**Created:** [May 24, 2023, 11:25am UTC](https://discuss.huggingface.co/t/having-trouble-accelerate-on-my-2-gpu-machine/40835 "2023-05-24T11:25:13Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![PopsFun](https://avatars.discourse-cdn.com/v4/letter/p/57b2e6/32.png) [@PopsFun](https://discuss.huggingface.co/u/PopsFun)\
**Post date:** [May 24, 2023, 11:25am UTC](https://discuss.huggingface.co/t/having-trouble-accelerate-on-my-2-gpu-machine/40835/1 "2023-05-24T11:25:14Z")

</div>

Hello, I am trying to fine-tune a model on my personal 2GPU machine accelerate framework.  
My huggingface accelerate configuration is given below. I am using peft huggingface to finetune AutoModelForCausalLM. However, the model is being loaded on only one GPU and the other GPU isn’t used. The training gives the following error: torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 100.00 MiB (GPU 0; 15.73 GiB total capacity; 11.44 GiB already allocated; 54.19 MiB free; 11.44 GiB reserved in total by PyTorch) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 100.00 MiB (GPU 0; 15.73 GiB total capacity; 11.44 GiB already allocated; 54.19 MiB free; 11.44 GiB reserved in total by PyTorch)

- Accelerate version: 0.19.0
- Platform: Linux-5.4.0-146-generic-x86\_64-with-glibc2.29
- Python version: 3.8.10
- Numpy version: 1.24.3
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- System RAM: 251.41 GB
- GPU type: NVIDIA RTX A4000
- Accelerate default config:  
- compute\_environment: LOCAL\_MACHINE  
- distributed\_type: DEEPSPEED  
- mixed\_precision: fp8  
- use\_cpu: False  
- num\_processes: 2  
- machine\_rank: 0  
- num\_machines: 1  
- rdzv\_backend: static  
- same\_network: True  
- main\_training\_function: main  
- deepspeed\_config: {‘gradient\_accumulation\_steps’: 1, ‘offload\_optimizer\_device’: ‘cpu’, ‘offload\_param\_device’: ‘nvme’, ‘zero3\_init\_flag’: False, ‘zero\_stage’: 2}  
- downcast\_bf16: no  
- tpu\_use\_cluster: False  
- tpu\_use\_sudo: False  
- tpu\_env: