# Using AMD'S RocM with accelerate library

**URL:** <https://discuss.huggingface.co/t/using-amds-rocm-with-accelerate-library/69551>\
**Category:** 🤗Accelerate\
**Created:** [January 17, 2024, 11:20pm UTC](https://discuss.huggingface.co/t/using-amds-rocm-with-accelerate-library/69551 "2024-01-17T23:20:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![imamcsiro](https://avatars.discourse-cdn.com/v4/letter/i/e99b99/32.png) [@imamcsiro](https://discuss.huggingface.co/u/imamcsiro)\
**Post date:** [January 17, 2024, 11:20pm UTC](https://discuss.huggingface.co/t/using-amds-rocm-with-accelerate-library/69551/1 "2024-01-17T23:20:49Z")

</div>

I have two AMD GPUs with ROCm. I want to use the SFTTrainer class with the accelerate library to fine-tune an LLM on the two GPUs with distributed data parallelism (DDP). However, I keep running into out of memory (OOM) errors, despite fine tuning running fine on one GPU. Does the accelerate library support ROCm? Is this an issue with something else?

---

<div class="post-metadata">

**Author:** ![marcsun13](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcsun13/32/19059_2.png) [@marcsun13](https://discuss.huggingface.co/u/marcsun13)\
**Post date:** [January 24, 2024, 6:32pm UTC](https://discuss.huggingface.co/t/using-amds-rocm-with-accelerate-library/69551/2 "2024-01-24T18:32:27Z")

</div>

Hi @imamcsiro , could you try with a smaller model and try to see how much is the increase of memory ? It should work since Pytorch supports AMD ROCm GPU