# Using Langchain ChatHuggingface with Text Generation Inference: missing field \`inputs\`

**URL:** <https://discuss.huggingface.co/t/using-langchain-chathuggingface-with-text-generation-inference-missing-field-inputs/101487>\
**Category:** Beginners\
**Created:** [August 8, 2024, 10:50am UTC](https://discuss.huggingface.co/t/using-langchain-chathuggingface-with-text-generation-inference-missing-field-inputs/101487 "2024-08-08T10:50:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![phucdev](https://avatars.discourse-cdn.com/v4/letter/p/f475e1/32.png) [@phucdev](https://discuss.huggingface.co/u/phucdev)\
**Post date:** [August 8, 2024, 10:50am UTC](https://discuss.huggingface.co/t/using-langchain-chathuggingface-with-text-generation-inference-missing-field-inputs/101487/1 "2024-08-08T10:50:18Z")

</div>

I deployed Mistral 7B Instruct v0.3 with TGI using the official docker container:

```auto
model=mistralai/Mistral-7B-Instruct-v0.3
# share a volume with the Docker container to avoid downloading weights every run
volume=$PWD/data
hf_token=<your_hf_api_token>

docker run --gpus all --shm-size 1g -e HF_TOKEN=$hf_token -p 8080:80 -v $volume:/data \
    ghcr.io/huggingface/text-generation-inference:2.2.0 --model-id $model --quantize bitsandbytes-nf4

```

I then created a HuggingFaceEndpoint and was able to interact with the Mistral model:

```auto
llm = HuggingFaceEndpoint(
    endpoint_url="http://localhost:8080/",
    max_new_tokens=512
)
llm.invoke("What is the capital of France?")

```

Working with the `InferenceClient` from `huggingface_hub` or directly via `curl` or `requests` works fine.

Then I created a ChatHuggingFace, so that I don’t have to manually add the special tokens, etc.:

```auto
llm_engine_hf = ChatHuggingFace(llm=llm, model_id="mistralai/Mistral-7B-Instruct-v0.3") # model_id can't be inferred from the endpoint_url
llm_engine_hf.invoke("Hugging Face is")

```

This fails because the json dict in the request does not have an `inputs` field, but a `messages` field.  
The error description in the response is:

```auto
'Failed to deserialize the JSON body into the target type: missing field `inputs` at line 1 column 400'

```

Btw. the actual error message that is output in the console is not that helpful 😅

```auto
huggingface_hub.utils._errors.HfHubHTTPError: 422 Client Error: Unprocessable Entity for url: http://localhost:8080/

```

What am I missing? Is there some parameter that I have to change when instantiating the `HuggingFaceEndpoint`/`ChatHuggingFace` or when deploying the model via TGI? I couldn’t find anything in the official guides.

---

<div class="post-metadata">

**Author:** ![phucdev](https://avatars.discourse-cdn.com/v4/letter/p/f475e1/32.png) [@phucdev](https://discuss.huggingface.co/u/phucdev)\
**Post date:** [August 8, 2024, 2:36pm UTC](https://discuss.huggingface.co/t/using-langchain-chathuggingface-with-text-generation-inference-missing-field-inputs/101487/2 "2024-08-08T14:36:02Z")

</div>

Okay, the solution was pretty trivial.  
I changed the endpoint\_url to “[http://localhost:8080/v1/chat/completions](http://localhost:8080/v1/chat/completions)”.  
It now works with `messages`.

I guess using the official Inference API from Huggingface chooses the correct url for you, but when you self-host you have to manually specify the url like that in order to use the Messages API. Otherwise it uses the “/generate” endpoint, which requires an `inputs` field.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [August 9, 2024, 2:36am UTC](https://discuss.huggingface.co/t/using-langchain-chathuggingface-with-text-generation-inference-missing-field-inputs/101487/3 "2024-08-09T02:36:02Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.