# Multi-node training

**URL:** <https://discuss.huggingface.co/t/multi-node-training/22582>\
**Category:** 🤗Accelerate\
**Created:** [September 5, 2022, 4:15pm UTC](https://discuss.huggingface.co/t/multi-node-training/22582 "2022-09-05T16:15:08Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![SaulLu](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saullu/32/2637_2.png) [@SaulLu](https://discuss.huggingface.co/u/SaulLu)\
**Post date:** [September 5, 2022, 4:15pm UTC](https://discuss.huggingface.co/t/multi-node-training/22582/1 "2022-09-05T16:15:08Z")

</div>

Hello,

Thank you very much for the `accelerate` lib. 🤗

We are currently experiencing a difficulty and were wondering if this could be a known case.

We want to run a training with `accelerate` and `deepspeed` on 4 nodes with 4 GPUs each. However, we see in our logs that 4 processes consider to be both a `main_process` and a `local_main_process`. We would have expected to see 1 `main_process` and 4 `local_main_process`.

Is what we expected wrong? Do you see a mistake we could have made?

Thanks a lot in advance!

---

<div class="post-metadata">

**Author:** ![SaulLu](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saullu/32/2637_2.png) [@SaulLu](https://discuss.huggingface.co/u/SaulLu)\
**Post date:** [September 6, 2022, 7:08am UTC](https://discuss.huggingface.co/t/multi-node-training/22582/2 "2022-09-06T07:08:29Z")

</div>

For someone who may encounter this problem in the future. This was a mistake on our part! We had not specified the `deepspeed_multinode_launcher` argument in the accelerate configuration, so it was set to `None`. Adding `deepspeed_multinode_launcher: standard` inside the `deepspeed_config` section to the configuration file solved our problem.

---

<div class="post-metadata">

**Author:** ![mderakhshani](https://avatars.discourse-cdn.com/v4/letter/m/7ea924/32.png) [@mderakhshani](https://discuss.huggingface.co/u/mderakhshani)\
**Post date:** [January 16, 2023, 7:15pm UTC](https://discuss.huggingface.co/t/multi-node-training/22582/3 "2023-01-16T19:15:50Z")

</div>

Hi @SaulLu, I am trying to do the same thing but with two nodes, each with 3 GPUs. Could you please share your config file content here?