# Batch size, gradient accumulation steps for Linear schedule

**URL:** <https://discuss.huggingface.co/t/batch-size-gradient-accumulation-steps-for-linear-schedule/5910>\
**Category:** Models\
**Created:** [May 1, 2021, 3:35pm UTC](https://discuss.huggingface.co/t/batch-size-gradient-accumulation-steps-for-linear-schedule/5910 "2021-05-01T15:35:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jordiclive](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jordiclive/32/28000_2.png) [@jordiclive](https://discuss.huggingface.co/u/jordiclive)\
**Post date:** [May 1, 2021, 3:35pm UTC](https://discuss.huggingface.co/t/batch-size-gradient-accumulation-steps-for-linear-schedule/5910/1 "2021-05-01T15:35:08Z")

</div>

If I am trying to recreate paper’s results, and they used a Batch size of 16 with 30 epochs, gradient accumulation steps of 3.

But I can only fit a batch size of 8 on my GPU.

I am using a Linear warm up scheduler with get\_linear\_schedule\_with\_warmup(  
self.opt, num\_warmup\_steps=0, num\_training\_steps=(dataset\_size / effective\_batch\_size) \* self.hparams.max\_epochs  
)

How should I adjust my max epochs, and gradient accumulation steps if I am using half the batch size?

Should I train for the same number of epochs. But increase the gradient accumulation steps?