# What algorithm Trainer uses for multi GPU training (without torchrun)

**URL:** <https://discuss.huggingface.co/t/what-algorithm-trainer-uses-for-multi-gpu-training-without-torchrun/30016>\
**Category:** Beginners\
**Created:** [January 19, 2023, 11:20am UTC](https://discuss.huggingface.co/t/what-algorithm-trainer-uses-for-multi-gpu-training-without-torchrun/30016 "2023-01-19T11:20:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![HoNTMi](https://avatars.discourse-cdn.com/v4/letter/h/ed655f/32.png) [@HoNTMi](https://discuss.huggingface.co/u/HoNTMi)\
**Post date:** [January 19, 2023, 11:20am UTC](https://discuss.huggingface.co/t/what-algorithm-trainer-uses-for-multi-gpu-training-without-torchrun/30016/1 "2023-01-19T11:20:52Z")

</div>

Hello!  
As I can see, now Trainer can runs multi GPU training even without using torchrun / python -m torch.distributed.launch / accelerate  
(Just by running the training script like a regular python script: python my\_script.py)  
Can you tell me what algorithm it uses? DP or DDP?  
And will the fsdp argument (from TrainingArguments) work correctly in this case?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [January 19, 2023, 4:25pm UTC](https://discuss.huggingface.co/t/what-algorithm-trainer-uses-for-multi-gpu-training-without-torchrun/30016/2 "2023-01-19T16:25:57Z")

</div>

It uses DP if you launch the script with python and DDP if you launch it with `torchrun`. FSDP will be ignored if you don’t launch it with `torchrun`.