# Regarding the eval batch size for large models

**URL:** <https://discuss.huggingface.co/t/regarding-the-eval-batch-size-for-large-models/17729>\
**Category:** Models\
**Created:** [May 9, 2022, 8:13pm UTC](https://discuss.huggingface.co/t/regarding-the-eval-batch-size-for-large-models/17729 "2022-05-09T20:13:04Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![arunwzd](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@arunwzd](https://discuss.huggingface.co/u/arunwzd)\
**Post date:** [May 9, 2022, 8:13pm UTC](https://discuss.huggingface.co/t/regarding-the-eval-batch-size-for-large-models/17729/1 "2022-05-09T20:13:04Z")

</div>

I am training T5-base with 2040 effective batch size (train\_batch\_size\_per\_gpu=8, grad.accumartion\_steps=51, n\_gpus=8) on 8 v100 GPUs. The training works fine w/o any memory error. But during the evaluation step (with eval\_batch\_size\_per\_gpu=8), I get `MemoryError` though its the same size as train batch size. Its weird that I am getting the memory error just during evaluation.

Also, I tried reducing the eval\_batch\_size\_per\_gpu=4 (halved), even then, I get memory error. Can someone help me with whats going on?