# Question answeirng Fine tuning

**URL:** <https://discuss.huggingface.co/t/question-answeirng-fine-tuning/89555>\
**Category:** Models\
**Created:** [May 31, 2024, 7:57am UTC](https://discuss.huggingface.co/t/question-answeirng-fine-tuning/89555 "2024-05-31T07:57:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Berny93](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@Berny93](https://discuss.huggingface.co/u/Berny93)\
**Post date:** [May 31, 2024, 7:57am UTC](https://discuss.huggingface.co/t/question-answeirng-fine-tuning/89555/1 "2024-05-31T07:57:20Z")

</div>

Hello. I need to do fine tuning to extract some fields from invoices (text extracted from invoices); some pretrained models such as distilled-bert require a dataset with the content, questions, the answers and the start\_id of the answer. I have a dataset containing only text extracted (the content) and the answers but not the start\_id. How can I overcome this limitation? Do you know other ways or models requiring only quesions and answers as dataset?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [May 31, 2024, 1:03pm UTC](https://discuss.huggingface.co/t/question-answeirng-fine-tuning/89555/2 "2024-05-31T13:03:35Z")

</div>

There are multiple ways to solve this, as you’re working with invoices I’d assume a vision-language model to perform better than a text-only one.

See our blog post on document AI for an overview: [Accelerating Document AI](https://huggingface.co/blog/document-ai). Models like LayoutLM are better than text-only models like DistilBERT.

Nowadays there are also a lot of generative document AI models including PaliGemma, Idefics2, LLaVa,… besides Donut, Pix2Struct, UDOP.

You can find demo notebooks for all of those here: [GitHub - NielsRogge/Transformers-Tutorials: This repository contains demos I made with the Transformers library by HuggingFace.](https://github.com/NielsRogge/Transformers-Tutorials/tree/master).

Another option is to fine-tune a text-only LLM on OCR-ed text as I explained here: [Fine tune LLMs on PDF Documents - #9 by nielsr](https://discuss.huggingface.co/t/fine-tune-llms-on-pdf-documents/71374/9)

---

<div class="post-metadata">

**Author:** ![Berny93](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@Berny93](https://discuss.huggingface.co/u/Berny93)\
**Post date:** [June 3, 2024, 10:20am UTC](https://discuss.huggingface.co/t/question-answeirng-fine-tuning/89555/3 "2024-06-03T10:20:59Z")

</div>

Thank you very much for the answer! I will surely have a look