# How to calculate effective batch size while using DDP?

**URL:** <https://discuss.huggingface.co/t/how-to-calculate-effective-batch-size-while-using-ddp/163996>\
**Category:** Beginners\
**Created:** [July 21, 2025, 8:27pm UTC](https://discuss.huggingface.co/t/how-to-calculate-effective-batch-size-while-using-ddp/163996 "2025-07-21T20:27:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jaydeepb](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jaydeepb/32/14906_2.png) [@jaydeepb](https://discuss.huggingface.co/u/jaydeepb)\
**Post date:** [July 21, 2025, 8:27pm UTC](https://discuss.huggingface.co/t/how-to-calculate-effective-batch-size-while-using-ddp/163996/1 "2025-07-21T20:27:46Z")

</div>

Hi there! I’m doing DDP style training with the help of trainer. I set up my accelerate config to use 4 GPUs. I have set these args for training:

- `per_device_train_batch_size=2, gradient_accumulation_steps=16`

I want to know what will be the effective batch size in this case?

When I start the training it shows total `24024` steps. There are total `1025024` training examples. My intuition was that `per_device_train_batch_size * gradient_accumulation_steps * 4` would be the effective batch size which comes out to be `128`. But on dividing `1025024` by `128` I don’t get `24024`. Is there anything that I’m missing?

---

<div class="post-metadata">

**Author:** ![Pimpcat-AU](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pimpcat-au/32/48989_2.png) [@Pimpcat-AU](https://discuss.huggingface.co/u/Pimpcat-AU)\
**Post date:** [July 21, 2025, 8:34pm UTC](https://discuss.huggingface.co/t/how-to-calculate-effective-batch-size-while-using-ddp/163996/3 "2025-07-21T20:34:08Z")

</div>

Seq length

tokens\_processed / (seq\_len \* effective\_batch\_size)

---

<div class="post-metadata">

**Author:** ![jaydeepb](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jaydeepb/32/14906_2.png) [@jaydeepb](https://discuss.huggingface.co/u/jaydeepb)\
**Post date:** [July 21, 2025, 11:02pm UTC](https://discuss.huggingface.co/t/how-to-calculate-effective-batch-size-while-using-ddp/163996/4 "2025-07-21T23:02:48Z")

</div>

resolved! was miscalculating!