# Getting Additional response from my RAG using HuggingFaceEndpoint inference

**URL:** <https://discuss.huggingface.co/t/getting-additional-response-from-my-rag-using-huggingfaceendpoint-inference/145964>\
**Category:** Beginners\
**Created:** [March 16, 2025, 9:00am UTC](https://discuss.huggingface.co/t/getting-additional-response-from-my-rag-using-huggingfaceendpoint-inference/145964 "2025-03-16T09:00:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![solo-leveling](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/solo-leveling/32/43389_2.png) [@solo-leveling](https://discuss.huggingface.co/u/solo-leveling)\
**Post date:** [March 16, 2025, 9:00am UTC](https://discuss.huggingface.co/t/getting-additional-response-from-my-rag-using-huggingfaceendpoint-inference/145964/1 "2025-03-16T09:00:09Z")

</div>

Hi folks

I am utilising remote inference using HuggingFaceEndpoint:

```auto
llm = HuggingFaceEndpoint(
    repo_id="huggingfaceh4/zephyr-7b-alpha",
    task="text-generation",
    temperature=0.5,
    max_new_tokens=1024
)

```

I have used `langchain-ai/retrieval-qa-chat` prompt, vectorstore retriever and created rag chain using below approach:

```auto
combine_docs_chain = create_stuff_documents_chain(llm, retrieval_qa_chat_prompt)
rag_chain = create_retrieval_chain(retriever, combine_docs_chain)

```

**Input:** Which runtime does Transformers.js uses  
**Sample answer I am getting**  
‘answer’: ’ to run models in the browser?\nAssistant: Transformers.js uses ONNX Runtime to run models in the browser.’

Any idea, why I am getting extra result before **Assistant: Transformers.js uses ONNX Runtime to run models in the browser.**

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [March 16, 2025, 1:13pm UTC](https://discuss.huggingface.co/t/getting-additional-response-from-my-rag-using-huggingfaceendpoint-inference/145964/2 "2025-03-16T13:13:00Z")

</div>

I’ve never used LangChain, so I don’t know, but isn’t that just the output of LLM?  
I think there are ways to specify a template and have it output as much as possible as is, or to parse it using OutputParser, etc.

> **[Output parsers | 🦜️🔗 LangChain](https://python.langchain.com/docs/concepts/output_parsers/)**
>
> The information here refers to parsers that take a text output from a model try to parse it into a more structured representation.

> **[HuggingFaceEndpoint — 🦜🔗 LangChain documentation](https://python.langchain.com/v0.2/api_reference/huggingface/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint)**

> **[How to Build RAG Pipelines for LLM Projects? - GeeksforGeeks](https://www.geeksforgeeks.org/how-to-build-rag-pipelines-for-llm-projects/)**
>
> Integrating Retrieval-Augmented Generation (RAG) pipelines with Large Language Models (LLMs) enhances their ability to provide accurate, context-specific responses by incorporating external knowledge sources.

---

<div class="post-metadata">

**Author:** ![solo-leveling](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/solo-leveling/32/43389_2.png) [@solo-leveling](https://discuss.huggingface.co/u/solo-leveling)\
**Post date:** [March 16, 2025, 4:48pm UTC](https://discuss.huggingface.co/t/getting-additional-response-from-my-rag-using-huggingfaceendpoint-inference/145964/3 "2025-03-16T16:48:44Z")

</div>

Thanks.

The GFG link helped.  
I needed to create prompt in the Zephyr format since I am using Zephyr model.

This is the prompt that helped give output without additional response in the start:

```auto
chat_prompt_2 = ChatPromptTemplate.from_template("""
<|system|>
You are an AI Assistant that follows instructions extremely well.
Please be truthful and give direct answers. Please tell 'I don't know' if user query is not in context.
</s>
<|user|>
Context: {context}

Question: {input}
</s>
<|assistant|>
""")

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [March 17, 2025, 4:48am UTC](https://discuss.huggingface.co/t/getting-additional-response-from-my-rag-using-huggingfaceendpoint-inference/145964/4 "2025-03-17T04:48:49Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.