# Difference between accelerate/torch\_distributed/deepspeed

**URL:** <https://discuss.huggingface.co/t/difference-between-accelerate-torch-distributed-deepspeed/17164>\
**Category:** DeepSpeed\
**Created:** [April 25, 2022, 6:28pm UTC](https://discuss.huggingface.co/t/difference-between-accelerate-torch-distributed-deepspeed/17164 "2022-04-25T18:28:50Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![arunwzd](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@arunwzd](https://discuss.huggingface.co/u/arunwzd)\
**Post date:** [April 25, 2022, 6:28pm UTC](https://discuss.huggingface.co/t/difference-between-accelerate-torch-distributed-deepspeed/17164/1 "2022-04-25T18:28:50Z")

</div>

Hi, I am new to distributed training and am using huggingface to train large models. I see many options to run distributed training. Can I know what is the difference between the following options:

1. `python train.py .....<ARGS>` -
2. `python -m torch.distributed.launch <ARGS>`
3. `deepspeed train.py <ARGS>`
4. `hf accelerate`

I did not expect option 1 to use distributed training. But it even seem to use some sort of torch distributed training? In that case, whats the difference between option 1 and option 2?

Does deepspeed use torch.distributed in the background?

Also, huggingface by default seem to use distributed training using torch? Whats the difference between accelerate?