# Evaluate fine-tuned LLM for question answering

**URL:** <https://discuss.huggingface.co/t/evaluate-fine-tuned-llm-for-question-answering/153168>\
**Category:** Beginners\
**Created:** [May 1, 2025, 8:03pm UTC](https://discuss.huggingface.co/t/evaluate-fine-tuned-llm-for-question-answering/153168 "2025-05-01T20:03:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![itskavya](https://avatars.discourse-cdn.com/v4/letter/i/6de8d8/32.png) [@itskavya](https://discuss.huggingface.co/u/itskavya)\
**Post date:** [May 1, 2025, 8:03pm UTC](https://discuss.huggingface.co/t/evaluate-fine-tuned-llm-for-question-answering/153168/1 "2025-05-01T20:03:12Z")

</div>

How can I evaluate the output of a fine-tuned LLM for question answering on a test set? The test set has varying lengths of output, hence, what value should I set for max length? If I set it too big, the output is much longer than the reference and if too small, the output can be less or it abruptly stops. I would greatly appreciate input on this. Also, would BLEU be a suitable metric?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [May 2, 2025, 6:24am UTC](https://discuss.huggingface.co/t/evaluate-fine-tuned-llm-for-question-answering/153168/2 "2025-05-02T06:24:48Z")

</div>

It might be a good idea to use the benchmarks used in the leaderboard for your own LLM.

This will allow you to understand the relative performance of your own LLM.

> **[GitHub - huggingface/evaluation-guidebook: Sharing both practical insights and theoretical...](https://github.com/huggingface/evaluation-guidebook)**
>
> Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

> **[GitHub - huggingface/yourbench: 🤗 Benchmark Large Language Models Reliably On Your...](https://github.com/huggingface/yourbench)**
>
> 🤗 Benchmark Large Language Models Reliably On Your Data

> **[Spaces - Hugging Face](https://huggingface.co/spaces?sort=trending&search=leaderboard)**
>
> Discover amazing ML apps made by the community