# Question-Answering/Text-generation/Summarizing: Fine-tune on multiple answers **URL:** \ **Category:** Beginners\ **Created:** [December 10, 2020, 7:31pm UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778 "2020-12-10T19:31:33Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![enrico](https://avatars.discourse-cdn.com/v4/letter/e/e495f1/32.png) [@enrico](https://discuss.huggingface.co/u/enrico)\ **Post date:** [December 10, 2020, 7:31pm UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/1 "2020-12-10T19:31:33Z")
Hi all! Looking to fine-tune a model for QA/Text-Generation (not sure how to frame this) and I’m wondering how to best prepare the dataset in a way that I can feed multiple answers to the same question? My goal is to facilitate the creation of a **unique** answer to a given question that is based on the input answers. The answers are longer-form (2 to 3 sentences) and I want the model to output this kind of length too. For now my idea is to fine-tune GPT-2 and look at this as a text generation problem (but I don’t know how GPT-2 would treat multiple answers to the same question - would it adjust the weight simply in favor of the used tokens in the answers?). Maybe creating a summary of the given answers would have the same effect. What would be the best approach to this? ---
**Author:** ![clem](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/clem/32/2838_2.png) [@clem](https://discuss.huggingface.co/u/clem)\ **Post date:** [December 16, 2020, 1:18am UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/2 "2020-12-16T01:18:57Z")
Maybe @valhalla has some pointers? ---
**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\ **Post date:** [December 16, 2020, 5:56am UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/3 "2020-12-16T05:56:24Z")
hi @enrico! I can think of two approaches 1. text generation/text2text Hear as you said you could feed the question and multiple answers to the model and ask it to generate the correct answer. One way to prepare the dataset for this is to use some delimiter tokens to separate the answers. For ex. Input text could be `context: context text question: question_text Answer1: answer1_text Answer2: answer2_text` and output text would be the answer text. You could also delimit the answers using `sep` tokens if the models have them. 1. Frame this as a multiple-choice problem Here instead of generating correct answer text, you ask the model to identify the correct answer within given answers discriminatively. You can use the `*ForMultipleChoice` models classes for this. You can find multiple choice example script in `examples/multiple-choice` dir, [here](https://github.com/huggingface/transformers/tree/master/examples/multiple-choice) ---
**Author:** ![enrico](https://avatars.discourse-cdn.com/v4/letter/e/e495f1/32.png) [@enrico](https://discuss.huggingface.co/u/enrico)\ **Post date:** [December 17, 2020, 2:11pm UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/4 "2020-12-17T14:11:50Z")
Thank you @valhalla leaning towards the first idea. Will give this a try ---
**Author:** ![cahya](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cahya/32/48725_2.png) [@cahya](https://discuss.huggingface.co/u/cahya)\ **Post date:** [December 18, 2020, 9:04am UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/5 "2020-12-18T09:04:42Z")
Maybe you could try also this solution [https://huggingface.co/qa/](https://huggingface.co/qa/) ---
**Author:** ![enrico](https://avatars.discourse-cdn.com/v4/letter/e/e495f1/32.png) [@enrico](https://discuss.huggingface.co/u/enrico)\ **Post date:** [December 18, 2020, 9:17am UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/6 "2020-12-18T09:17:55Z")
@cahya thanks a lot! Looks really interesting didn’t know about it! ---
**Author:** ![cahya](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cahya/32/48725_2.png) [@cahya](https://discuss.huggingface.co/u/cahya)\ **Post date:** [December 18, 2020, 10:02am UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/7 "2020-12-18T10:02:17Z")
@enrico please have a look also on the discussion we had about it [Demo of Open Domain Long Form Question Answering](https://discuss.huggingface.co/t/demo-of-open-domain-long-form-question-answering/138) ---
**Author:** ![Ontor79](https://avatars.discourse-cdn.com/v4/letter/o/b9bd4f/32.png) [@Ontor79](https://discuss.huggingface.co/u/Ontor79)\ **Post date:** [September 28, 2021, 9:57am UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/8 "2021-09-28T09:57:42Z")
Hello can i use multiple context in a question and answer huggingface api?? I am actually trying to make a context that involves several paragraphs about all the past 46 US presidents and I cannot seem to put all those under a single context. Can you please help me ??? ---
**Author:** ![lucasresck](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lucasresck/32/3260_2.png) [@lucasresck](https://discuss.huggingface.co/u/lucasresck)\ **Post date:** [November 20, 2021, 7:03pm UTC](https://discuss.huggingface.co/t/question-answering-text-generation-summarizing-fine-tune-on-multiple-answers/2778/9 "2021-11-20T19:03:26Z")
Hi there, If I understood your situation, you have a context, questions and answers, but your context is too big to feed the model in one passage. I suggest you to take a look on [Hugging Face’s question answering example notebook](https://github.com/huggingface/notebooks/blob/master/examples/question_answering.ipynb). They manage to solve this problem splitting up the context in several parts, when necessary. During training, if the context split does not contain the answer, they point the answer to the CLS token. Finally, during evaluation, they search for the answer in all splits and consider the most likely answer.