# Evaluation loss depends on batch size **URL:** \ **Category:** Beginners\ **Created:** [October 14, 2024, 8:30pm UTC](https://discuss.huggingface.co/t/evaluation-loss-depends-on-batch-size/112046 "2024-10-14T20:30:28Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![ItaiP24](https://avatars.discourse-cdn.com/v4/letter/i/3e96dc/32.png) [@ItaiP24](https://discuss.huggingface.co/u/ItaiP24)\ **Post date:** [October 14, 2024, 8:30pm UTC](https://discuss.huggingface.co/t/evaluation-loss-depends-on-batch-size/112046/1 "2024-10-14T20:30:28Z")
I train a token classification model on a private dataset. I noticed that the evaluation loss has different values if I change the value of per\_device\_eval\_batch\_size. Is it a known thing? The total test set size is not divisible with the batch size (per\_device\_eval\_batch\_size\*gpu\_num). When I put a batch size that divides the total test set size, I still get a different number from what I get when I calculate the loss directly with torch.nn.CrossEntropyLoss. Also, accuracy, recall, etc do not change with the batch size. Thanks for you help! ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 14, 2024, 11:46pm UTC](https://discuss.huggingface.co/t/evaluation-loss-depends-on-batch-size/112046/2 "2024-10-14T23:46:30Z")
Apparently a known but neglected issue for several years… > [@Batch size during training vs batch size during evaluation](https://discuss.huggingface.co/t/batch-size-during-training-vs-batch-size-during-evaluation/20827): > > I am confused about the difference between batch size during training versus batch size during evaluation. I am trying to measure how batch size influences the inference time(speed of prediction) of different NLP models after they have been trained using the Huggingface Trainer API. The code I used is below def print\_summary(result): print(f"Time: {result.metrics['test\_runtime']:.2f}") print(f"Samples/second: {result.metrics['test\_samples\_per\_second']:.2f}") print(f"Test Loss: {res…