# Fine-tuning conversational models with the technical documentation

**URL:** <https://discuss.huggingface.co/t/fine-tuning-conversational-models-with-the-technical-documentation/66311>\
**Category:** Beginners\
**Created:** [December 19, 2023, 8:40am UTC](https://discuss.huggingface.co/t/fine-tuning-conversational-models-with-the-technical-documentation/66311 "2023-12-19T08:40:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![JulienThielleux](https://avatars.discourse-cdn.com/v4/letter/j/59ef9b/32.png) [@JulienThielleux](https://discuss.huggingface.co/u/JulienThielleux)\
**Post date:** [December 19, 2023, 8:40am UTC](https://discuss.huggingface.co/t/fine-tuning-conversational-models-with-the-technical-documentation/66311/1 "2023-12-19T08:40:54Z")

</div>

Hello.

For context, I am currently working in IT maintenance on a large accounting application for a bank.  
Using different LLMs and discovering fine-tuning, I wondered if it was possible to fine-tune a conversational model with data from the technical documentation of the application?

If I am not mistaken, this would speed up the work of the maintenance team a lot if we can just ask questions about the application to a model.

Will fine-tuning allow the model to retain the “knowledge” of the application?  
The application contains many obscure variable names. Will the Llama tokenizer be suitable?  
Would we get better results by simply passing the information in the context of the LLM or via files as GPT4 now allows ?

---

<div class="post-metadata">

**Author:** ![justinchips](https://avatars.discourse-cdn.com/v4/letter/j/bcef8e/32.png) [@justinchips](https://discuss.huggingface.co/u/justinchips)\
**Post date:** [July 11, 2024, 1:33am UTC](https://discuss.huggingface.co/t/fine-tuning-conversational-models-with-the-technical-documentation/66311/2 "2024-07-11T01:33:56Z")

</div>

This is a common use-case for LLMs 🙂. You will want to research “Retrieval-Augmented Generation” (RAG). It’s the process of enabling an LLM to reference a knowledge base outside of its training data sources before generating a response.

---

<div class="post-metadata">

**Author:** ![mukundbagul](https://avatars.discourse-cdn.com/v4/letter/m/48db29/32.png) [@mukundbagul](https://discuss.huggingface.co/u/mukundbagul)\
**Post date:** [July 18, 2024, 12:25pm UTC](https://discuss.huggingface.co/t/fine-tuning-conversational-models-with-the-technical-documentation/66311/3 "2024-07-18T12:25:08Z")

</div>

but how one can used it’s dataset to fine tune it using rag ? i’m new in this tech if there is any doc or video you know just provide the link of it it will be very beneficial for me