# Shockingly Incorrect Evaluate Function in Huggingface API

**URL:** <https://discuss.huggingface.co/t/shockingly-incorrect-evaluate-function-in-huggingface-api/60848>\
**Category:** 🤗Transformers\
**Created:** [November 2, 2023, 4:08pm UTC](https://discuss.huggingface.co/t/shockingly-incorrect-evaluate-function-in-huggingface-api/60848 "2023-11-02T16:08:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sayan01](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@Sayan01](https://discuss.huggingface.co/u/Sayan01)\
**Post date:** [November 2, 2023, 4:08pm UTC](https://discuss.huggingface.co/t/shockingly-incorrect-evaluate-function-in-huggingface-api/60848/1 "2023-11-02T16:08:02Z")

</div>

Two different accuracy, one from manual evaluation and one using huggingface’s trainer function. They are completely different. I am shocked

 ![Screen Shot 2023-11-03 at 3.05.20 AM](https://us1.discourse-cdn.com/hellohellohello/original/3X/a/1/a1a74637ab89bb42f1d1b185dd725896786cceb9.png)

---

<div class="post-metadata">

**Author:** ![muellerzr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/muellerzr/32/21583_2.png) [@muellerzr](https://discuss.huggingface.co/u/muellerzr)\
**Post date:** [November 2, 2023, 4:25pm UTC](https://discuss.huggingface.co/t/shockingly-incorrect-evaluate-function-in-huggingface-api/60848/2 "2023-11-02T16:25:08Z")

</div>

> [@Sayan01](#):
>
> am shocked

You are calling `drop_last` in your eval dataloader, which in general you should never do. Does not doing so fix this? 🙂