# Tool calling gets stuck in an infinite loop **URL:** \ **Category:** Inference Endpoints on the Hub\ **Created:** [April 10, 2025, 12:55pm UTC](https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792 "2025-04-10T12:55:37Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![hadleywickham](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hadleywickham/32/45174_2.png) [@hadleywickham](https://discuss.huggingface.co/u/hadleywickham)\ **Post date:** [April 10, 2025, 12:55pm UTC](https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792/1 "2025-04-10T12:55:37Z")
I’m having problems using tool calling with the inference endpoint. I’m following the documentation (such as it is), and the examples in [Tool Use, Unified](https://huggingface.co/blog/unified-tool-use). But the API doesn’t seem to recognise that I’ve returned a result, and just returns another tool call request: ````json { "messages": [ { "role": "system", "content": "Be very terse, not even punctuation." }, { "role": "user", "content": [ { "type": "text", "text": "What's the current date in Y-M-D format?" } ] }, { "role": "assistant", "tool_calls": [ { "id": "0", "function": { "name": "tool_001", "arguments": "{}" }, "type": "function" } ] }, { "role": "tool", "content": "2024-01-01", "name": "tool_001", "tool_call_id": "0" }, { "role": "assistant", "tool_calls": [ { "id": "0", "function": { "name": "tool_001", "arguments": "{}" }, "type": "function" } ] }, { "role": "tool", "content": "2024-01-01", "name": "tool_001", "tool_call_id": "0" } ], "model": "meta-llama/Llama-3.1-8B-Instruct", "stream": false, "tools": [ { "type": "function", "function": { "name": "tool_001", "description": "Return the current date", "strict": true, "parameters": { "type": "object", "description": "", "properties": {}, "required": [], "additionalProperties": false } } } ] }``` Any ideas what I'm doing wrong? ```` ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [April 11, 2025, 4:12am UTC](https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792/2 "2025-04-11T04:12:29Z")
There is probably something wrong with the function calling in all models. It may be a bug in TGI. > [@Function calling not working with inference clients on (seemingly) any model](https://discuss.huggingface.co/t/function-calling-not-working-with-inference-clients-on-seemingly-any-model/138581): > > I’m attempting to use function-calling capabilities with text inference models, but none of the ways I’ve attempted to do with the inference clients work. I use OpenAI’s function calling regularly, but Hugging Face capabilities seem fragile and unpredictable, but I suspect I’m just “doing it wrong.” I’d appreciate any insights into the following cases I’m seeing: JavaScript inference client fails If I just include the first message (e.g. “What’s the weather in X place?”), the models will prope… > > > \### Description > > When using the inference client with function calling, models s…eem to never resolve their calls. > > As we know, typically, with the OpenAI pattern, the simplest function/tool call is a series of messages of various roles (system, user, assistant, tool) organized like this: > > system → user ("what's the weather?") → assistant (tool\_calls) → tool (result: "4ºC") → assistant (content: "it's 4ºC") > > The HF docs seem to indicate this is the same pattern, although the messages have some minor differences (e.g. description: null, which never happens with OpenAI). When using the Python inference client, these tool\_calls never resolve even after functions are called and their return values are included and seemingly properly referenced. Instead, they look like this: > > system → user ("what's the weather?") → assistant (tool\_calls) → tool (result: "4ºC") → assistant (tool\_calls) … > > Instead of returning a text completion, the HF inference client returns the same "assistant" message specifying a required tool\_calls. In OpenAI, they resolve to a typical "assistant" message with token content if the function calls have been satisfied and no further calls are required. > > Models used that exhibit this behavior: > \- NousResearch/Hermes-3-Llama-3.1-8B > \- Qwen/Qwen2.5-72B-Instruct > \- meta-llama/Meta-Llama-3-8B-Instruct > > \--- > > It's worth noting that Mistral models also error out, specifying that a 9-character alphanumeric string is required for the \`tool\_call\_id\`. Now, the models themselves don't provide such IDs, so we need to supply them ourselves. But even when doing so, the same error occurs, that 9-char identifiers are missing. (e.g. mistralai/Mistral-7B-Instruct-v0.3) > > The JavaScript client also fails with the above errors, and also a third: "An error occurred while fetching the blob". > > \### System Info > > \- macOS 15.2 > \- Python 3.13.1 > \- huggingface\_hub 0.28.1 > > \### Information > > \- \[\] Docker > \- \[x\] The CLI directly > > \### Tasks > > \- \[x\] An officially supported command > \- \[\] My own modifications > > \### Reproduction > > Gist of sample error code is here: https://gist.github.com/awmartin/c64c84fbbdc3a9f0c2ce6e5ae0dab3dc > > 1. Provide API token > 2. python inference-tool-calls.py > > A message results that's unexpected. I expected this to be a typical message with a string content, something like, "It's 4 degrees today." Instead, it just repeats the assistant message with the original tool\_call message: > > \[ChatCompletionOutputComplete(finish\_reason='stop', index=0, message=ChatCompletionOutputMessage(role='assistant', content=None, tool\_calls=\[ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'unit': 'Celsius', 'location': 'Philadelphia, PA, US'}, name='get\_current\_temperature', description=None), id='0', type='function')\]), logprobs=None)\] > > \### Expected behavior > > I expected a message that resolved to something similar to "It's 4 degrees Celsius today" rather than the tool\_call message repeated. ---
**Author:** ![hadleywickham](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hadleywickham/32/45174_2.png) [@hadleywickham](https://discuss.huggingface.co/u/hadleywickham)\ **Post date:** [April 12, 2025, 12:33pm UTC](https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792/3 "2025-04-12T12:33:39Z")
Thank you!