# Seq2SeqTrainer multiple GPUs **URL:** \ **Category:** 🤗Transformers\ **Created:** [January 21, 2025, 7:56pm UTC](https://discuss.huggingface.co/t/seq2seqtrainer-multiple-gpus/136741 "2025-01-21T19:56:26Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![gandg](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/gandg/32/38814_2.png) [@gandg](https://discuss.huggingface.co/u/gandg)\ **Post date:** [January 21, 2025, 7:56pm UTC](https://discuss.huggingface.co/t/seq2seqtrainer-multiple-gpus/136741/1 "2025-01-21T19:56:26Z")
Does Seq2SeqTrainer automatically utilize every available GPU on the system without expressly setting any parameter? The **per\_device\_train\_batch\_size** seems to default to 8. Is there anything in particular I need to do to instruct the trainer to use every GPU available on the system? ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [January 22, 2025, 8:24am UTC](https://discuss.huggingface.co/t/seq2seqtrainer-multiple-gpus/136741/2 "2025-01-22T08:24:59Z")
Cases where it is not used appropriately and evenly are often reported, but it seems that it will automatically use multiple GPUs in most cases. If you want to specify in detail, check out PyTorch’s DDP. Also, if I may say so, you should definitely install the accelerate library. ```py pip install -U accelerate ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\ **Post date:** [January 22, 2025, 8:25pm UTC](https://discuss.huggingface.co/t/seq2seqtrainer-multiple-gpus/136741/3 "2025-01-22T20:25:59Z")
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.