# Issue with Qwen model in HuggingFace

**URL:** <https://discuss.huggingface.co/t/issue-with-qwen-model-in-huggingface/175637>\
**Category:** Models\
**Created:** [April 29, 2026, 5:23am UTC](https://discuss.huggingface.co/t/issue-with-qwen-model-in-huggingface/175637 "2026-04-29T05:23:44Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Surajgjadhav](https://avatars.discourse-cdn.com/v4/letter/s/f475e1/32.png) [@Surajgjadhav](https://discuss.huggingface.co/u/Surajgjadhav)\
**Post date:** [April 29, 2026, 5:23am UTC](https://discuss.huggingface.co/t/issue-with-qwen-model-in-huggingface/175637/1 "2026-04-29T05:23:44Z")

</div>

When I am using Qwen model in smolagent or Llamaindex, it’s giving error  
In smolagent, it’s giving following error:

```bash
Error in generating model output:
Client.post() got an unexpected keyword argument 'stream'

```

In LlamaIndex, getting this error:

```bash
RuntimeError: Cannot send a request, as the client has been closed.

```

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 29, 2026, 11:20am UTC](https://discuss.huggingface.co/t/issue-with-qwen-model-in-huggingface/175637/2 "2026-04-29T11:20:19Z")

</div>

Because API specifications change rapidly, drift issues are likely to occur. If you’re lucky, updating `huggingface_hub` alone may resolve the issue, but “the features supported via the API may have changed”:

* * *

# Qwen model errors in smolagents and LlamaIndex: likely causes and practical fixes

You are probably dealing with **two different integration problems** , not one “Qwen is broken” problem.

The two errors point to different layers:

```bash
# smolagents
Error in generating model output:
Client.post() got an unexpected keyword argument 'stream'

```

This points to a **streaming / HTTP-client / wrapper compatibility problem**.

```bash
# LlamaIndex
RuntimeError: Cannot send a request, as the client has been closed.

```

This points to an **async client lifecycle problem** , especially if you used async calls, streaming calls, reused the same LLM/query engine object, or ran this in a notebook/server where objects persist between calls.

The common pattern is:

```plaintext
Qwen model
+ Hugging Face provider routing
+ wrapper library
+ stream/tool/chat/async behavior
+ package versions
= failure surface

```

So the better diagnosis is not:

```plaintext
Qwen does not work.

```

The better diagnosis is:

```plaintext
The selected Qwen route works differently through different wrappers.
smolagents and LlamaIndex are hitting different edge cases in the inference stack.

```

* * *

## 1. Background: why this happens with Qwen + Hugging Face wrappers

Modern Hugging Face inference is no longer just:

```plaintext
model_id -> API call -> output

```

It is more like:

```plaintext
model_id
-> task type
-> provider selection
-> Hugging Face router / provider backend
-> Python client
-> framework wrapper
-> sync / async / streaming / tool-calling mode

```

Hugging Face’s [`InferenceClient`](https://huggingface.co/docs/huggingface_hub/en/package_reference/inference_client) is a unified client that can work with the free Inference API, self-hosted Inference Endpoints, and third-party Inference Providers.

Hugging Face [Inference Providers](https://huggingface.co/docs/inference-providers/index) also support provider-backed serverless inference, and the [chat-completion task](https://huggingface.co/docs/inference-providers/tasks/chat-completion) supports OpenAI-compatible calls, tools/constraints, and streaming.

That matters because Qwen models are often used for:

- chat
- coding
- agents
- tool calling
- streaming
- RAG response synthesis
- OpenAI-compatible routes
- Hugging Face provider routes

Those features do **not** always have the same support across every provider/wrapper combination.

A Qwen model might work here:

```plaintext
Direct Hugging Face InferenceClient
-> explicit provider
-> non-streaming chat

```

but fail here:

```plaintext
smolagents
-> auto provider
-> tool-calling agent
-> streaming

```

or here:

```plaintext
LlamaIndex
-> HuggingFaceInferenceAPI
-> async streaming
-> reused client

```

Same model name. Different route. Different behavior.

* * *

## 2. The smolagents error

### Error

```bash
Error in generating model output:
Client.post() got an unexpected keyword argument 'stream'

```

### What this error usually means

This is a Python client-level error. It usually means some code called something shaped like:

```python
client.post(..., stream=True)

```

but that `Client.post()` method does **not** accept a `stream` keyword argument.

This is important because [`huggingface_hub` v1 migrated from `requests`/`aiohttp` to `httpx`](https://huggingface.co/docs/huggingface_hub/concepts/migration). That migration is a good thing overall, but it can expose old wrapper assumptions.

Older `requests`-style code commonly uses:

```python
requests.post(url, stream=True)

```

But `httpx` streaming is shaped differently. If a wrapper forwards `stream=True` into the wrong layer, you can get:

```bash
Client.post() got an unexpected keyword argument 'stream'

```

### Critical distinction: two meanings of `stream`

There are two different layers where “streaming” can appear.

#### Valid: model/API-level streaming

This is normal:

```python
client.chat.completions.create(
    model="...",
    messages=[...],
    stream=True,
)

```

Hugging Face’s [TGI streaming docs](https://huggingface.co/docs/text-generation-inference/conceptual/streaming) show this style: pass `stream=True` to `InferenceClient.chat.completions.create(...)` and iterate over chunks.

#### Risky: HTTP-client-level streaming

This is the suspicious shape:

```python
client.post(..., stream=True)

```

Your error suggests that `stream=True` is reaching a lower-level `.post()` method that does not accept it.

So the likely problem is **not** “streaming is impossible.” The likely problem is:

```plaintext
stream=True is being passed to the wrong abstraction layer.

```

* * *

## 3. Likely causes of the smolagents error

### Cause 1 — version mismatch between `smolagents`, `huggingface_hub`, `httpx`, and possibly `openai`

This is the most likely cause.

A broken environment may look like this:

```plaintext
smolagents: older
huggingface_hub: newer, HTTPX-based
httpx: newer
openai: newer

```

or the reverse:

```plaintext
smolagents: newer
huggingface_hub: older
httpx: incompatible

```

The wrapper expects one API shape; the installed lower-level client exposes another.

Relevant docs:

- [`huggingface_hub` v1 migration guide](https://huggingface.co/docs/huggingface_hub/concepts/migration)
- [`InferenceClient` reference](https://huggingface.co/docs/huggingface_hub/en/package_reference/inference_client)
- [Hugging Face chat-completion task](https://huggingface.co/docs/inference-providers/tasks/chat-completion)

### Cause 2 — streaming is enabled implicitly

You may not be explicitly writing `stream=True`, but a framework can enable streaming internally.

For example, an agent framework may stream model output to:

- show tokens incrementally
- collect intermediate tool calls
- support function/tool-call deltas
- stream logs into a UI
- support notebook display

So even if your own code does not contain `stream=True`, the wrapper path may still use streaming.

### Cause 3 — provider auto-selection changed or selected a provider that does not support the needed mode

This is especially relevant for smolagents.

There is a relevant smolagents issue: [Default ToolCallingAgent InferenceClient examples failing](https://github.com/huggingface/smolagents/issues/1808). In that case, the report says Hugging Face provider selection picked a provider that did not support tool calling, while another provider had worked before.

That issue is not the exact same `stream` error, but it is the same architectural class:

```plaintext
agent wrapper sends advanced parameters
-> selected provider does not support them
-> failure appears inside the wrapper

```

In that issue, the advanced parameters were `tools` / `tool_choice`.

In your case, the advanced parameter is probably `stream`.

### Cause 4 — using an agent/tool-calling path before plain chat is verified

Agent frameworks send more complex payloads than normal chat.

Plain chat might send:

```json
{
  "messages": [...],
  "max_tokens": 256
}

```

Agent/tool-calling may send:

```json
{
  "messages": [...],
  "tools": [...],
  "tool_choice": "auto",
  "stream": true,
  "max_tokens": 256
}

```

If any part of that payload is unsupported by the provider or wrapper, you can get an error that looks unrelated to tools.

* * *

## 4. smolagents solutions

### Solution A — upgrade related packages together

Do not upgrade only one package.

Use a clean environment if possible:

```bash
python -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade pip
python -m pip install -U \
  "smolagents[toolkit]" \
  huggingface_hub \
  httpx \
  openai

```

Then restart the Python process or notebook kernel.

This matters because if you install new packages in an already-running notebook, Python may still hold old imported modules in memory.

### Solution B — print versions before debugging

Run this in the failing environment:

```python
import sys
import importlib.metadata as md

packages = [
    "smolagents",
    "huggingface_hub",
    "httpx",
    "openai",
]

print("Python:", sys.version)

for package in packages:
    try:
        print(f"{package}: {md.version(package)}")
    except md.PackageNotFoundError:
        print(f"{package}: not installed")

```

Look for a mixed environment such as:

```plaintext
new huggingface_hub + old smolagents
new httpx + old wrapper
old notebook imports after pip upgrade

```

Also check your Python version. `huggingface_hub` v1 requires Python 3.9+ according to the [v1 migration guide](https://huggingface.co/docs/huggingface_hub/concepts/migration).

### Solution C — test Hugging Face directly before smolagents

Before using smolagents, check whether the model/provider route works directly.

```python
import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="<provider>",
    api_key=os.environ["HF_TOKEN"],
)

response = client.chat.completions.create(
    model="<qwen-model-id>",
    messages=[
        {"role": "user", "content": "Return exactly: ok"}
    ],
    max_tokens=10,
    temperature=0,
    stream=False,
)

print(response.choices[0].message.content)

```

Example values:

```plaintext
<provider> = together
<qwen-model-id> = Qwen/Qwen3-Coder-30B-A3B-Instruct

```

Interpretation:

| Result | Meaning |
| --- | --- |
| Direct call works | Qwen/provider/token/basic route are OK; smolagents wrapper is the likely issue |
| Direct call fails with auth/quota/provider error | Fix token/provider/model availability first |
| Direct call fails with stream/client error | Your lower-level client environment is suspect |
| Non-streaming works but streaming fails | Streaming/provider/client path is the issue |

### Solution D — set provider explicitly

Avoid provider auto-selection while debugging.

```python
from smolagents import CodeAgent, InferenceClientModel
import os

model = InferenceClientModel(
    model_id="<qwen-model-id>",
    provider="<provider>",
    api_key=os.environ["HF_TOKEN"],
    max_tokens=512,
    temperature=0.2,
)

agent = CodeAgent(
    tools=[],
    model=model,
)

result = agent.run("Return exactly: ok")
print(result)

```

Example:

```python
model = InferenceClientModel(
    model_id="Qwen/Qwen3-Coder-30B-A3B-Instruct",
    provider="together",
    api_key=os.environ["HF_TOKEN"],
    max_tokens=512,
    temperature=0.2,
)

```

The key parts are:

```python
provider="<provider>"
tools=[]

```

This removes two common sources of failure:

1. implicit provider selection
2. tool-calling payload complexity

You can later change `provider` to another provider that is currently supported for your exact Qwen model.

### Solution E — add tools only after the basic model works

Use this order:

```plaintext
1. Direct Hugging Face non-streaming call
2. Direct Hugging Face streaming call
3. smolagents with tools=[]
4. smolagents with one simple local tool
5. smolagents with multiple tools
6. smolagents with web/search/external tools
7. only then tool-calling + streaming together

```

Do **not** start with:

```plaintext
Qwen + smolagents + auto provider + tools + streaming

```

That is too many moving parts.

* * *

## 5. The LlamaIndex error

### Error

```bash
RuntimeError: Cannot send a request, as the client has been closed.

```

### What this error usually means

This usually means a client object was closed and then reused.

In LlamaIndex, this is especially plausible with `HuggingFaceInferenceAPI`.

The [LlamaIndex `HuggingFaceInferenceAPI` reference](https://developers.llamaindex.ai/python/framework-api-reference/llms/huggingface_api/) describes a wrapper around Hugging Face’s Inference API. It uses Hugging Face’s `InferenceClient` for sync calls and `AsyncInferenceClient` for async calls.

The same reference/source shows async streaming paths shaped like:

```python
async for delta in await self._async_client.text_generation(
    prompt, stream=True, **model_kwargs
):
    ...
await self._async_client.close()

```

That creates a likely failure sequence:

```plaintext
1. You create one HuggingFaceInferenceAPI object.
2. You use async streaming.
3. The stream finishes.
4. LlamaIndex closes its internal AsyncInferenceClient.
5. You reuse the same LLM object, query engine, chat engine, or app object.
6. The next request tries to use the closed client.
7. RuntimeError: Cannot send a request, as the client has been closed.

```

This fits your LlamaIndex error closely.

* * *

## 6. Additional LlamaIndex-specific issue: old conversational task path

There is another important problem in the LlamaIndex Hugging Face wrapper.

The LlamaIndex Hugging Face wrapper has historically separated chat/completion behavior around Hugging Face task types. A relevant issue, [Hugging Face conversational API returns 404](https://github.com/run-llama/llama_index/issues/18547), reports that Hugging Face’s old `conversational` task route returned 404 and that setting `task="text-generation"` was the workaround.

There are also Hugging Face course/notebook discussions showing similar Qwen + `HuggingFaceInferenceAPI` breakage, such as:

- [404 Not Found when using Qwen models with HuggingFaceInferenceAPI](https://huggingface.co/agents-course/notebooks/discussions/130)
- [Agents Course Unit 2.2 error 404](https://discuss.huggingface.co/t/agents-course-unit-2-2-error-404/168035)

So with Qwen + LlamaIndex, you may have **two overlapping problems** :

```plaintext
Problem A: async client closed and reused
Problem B: chat/conversational task route is fragile or stale

```

That is why the safer first test is:

```plaintext
sync + non-streaming + text-generation

```

* * *

## 7. LlamaIndex solutions

### Solution A — first test sync, non-streaming, text generation

Start here:

```python
import os
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI

llm = HuggingFaceInferenceAPI(
    model_name="<qwen-model-id>",
    token=os.environ["HF_TOKEN"],
    provider="<provider>",
    task="text-generation",
    is_chat_model=False,
    num_output=128,
    temperature=0.2,
)

response = llm.complete("Return exactly: ok")
print(response.text)

```

Example:

```python
llm = HuggingFaceInferenceAPI(
    model_name="Qwen/Qwen3-Coder-30B-A3B-Instruct",
    token=os.environ["HF_TOKEN"],
    provider="together",
    task="text-generation",
    is_chat_model=False,
    num_output=128,
    temperature=0.2,
)

```

Why this is the right first test:

- avoids async
- avoids streaming
- avoids the fragile chat/conversational route
- avoids query-engine complexity
- uses text generation directly
- keeps provider explicit

If this works, then Qwen + Hugging Face + LlamaIndex basic completion is fine.

If your full app still fails, the problem is likely in:

- async
- streaming
- query engine reuse
- chat engine reuse
- chat/conversational route
- package mismatch

### Solution B — avoid reusing the same LlamaIndex object after async streaming

Risky pattern:

```python
llm = HuggingFaceInferenceAPI(...)

stream = await llm.astream_complete("First question")
async for chunk in stream:
    print(chunk.delta, end="")

# Later reuse the same llm
stream = await llm.astream_complete("Second question")

```

Safer diagnostic pattern:

```python
import os
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI

def make_llm():
    return HuggingFaceInferenceAPI(
        model_name="<qwen-model-id>",
        token=os.environ["HF_TOKEN"],
        provider="<provider>",
        task="text-generation",
        is_chat_model=False,
        num_output=256,
        temperature=0.2,
    )

llm1 = make_llm()
# use llm1 once

llm2 = make_llm()
# use llm2 for the next async/streaming call

```

This is not the prettiest architecture, but it is a strong workaround if the issue is “closed client reused.”

### Solution C — avoid async streaming until the basic path works

First prove:

```python
llm.complete(...)

```

Then test:

```python
await llm.acomplete(...)

```

Then test:

```python
llm.stream_complete(...)

```

Then test:

```python
await llm.astream_complete(...)

```

Do not jump directly into a RAG query engine with async streaming.

### Solution D — use LlamaIndex’s OpenAI-compatible route through the Hugging Face router

Because Hugging Face Inference Providers expose an OpenAI-compatible chat route, this can be cleaner than using the LlamaIndex Hugging Face-specific wrapper for chat-style Qwen calls.

Relevant docs:

- [Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers/index)
- [Hugging Face chat completion](https://huggingface.co/docs/inference-providers/tasks/chat-completion)
- [LlamaIndex Hugging Face LLM integration](https://developers.llamaindex.ai/python/framework/integrations/llm/huggingface/)
- [LlamaIndex `HuggingFaceInferenceAPI` reference](https://developers.llamaindex.ai/python/framework-api-reference/llms/huggingface_api/)

Install:

```bash
python -m pip install -U \
  llama-index \
  llama-index-core \
  llama-index-llms-openai-like \
  openai

```

Then:

```python
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="<qwen-model-id>:<provider>",
    api_base="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
    is_chat_model=True,
    is_function_calling_model=False,
    max_tokens=512,
    temperature=0.2,
)

response = llm.chat([
    {"role": "user", "content": "Return exactly: ok"}
])

print(response.message.content)

```

Example:

```python
llm = OpenAILike(
    model="Qwen/Qwen3-Coder-30B-A3B-Instruct:together",
    api_base="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
    is_chat_model=True,
    is_function_calling_model=False,
    max_tokens=512,
    temperature=0.2,
)

```

This path is often conceptually simpler:

```plaintext
LlamaIndex OpenAILike
-> Hugging Face router /v1
-> explicit provider
-> Qwen

```

instead of:

```plaintext
LlamaIndex HuggingFaceInferenceAPI
-> conversational/text-generation task logic
-> sync/async Hugging Face clients
-> provider/router
-> Qwen

```

For your exact error, I would strongly consider the OpenAI-compatible route.

* * *

## 8. Package alignment

Both errors can be worsened by partially upgraded packages.

Use a coherent install:

```bash
python -m pip install --upgrade pip

python -m pip install -U \
  huggingface_hub \
  httpx \
  openai \
  "smolagents[toolkit]" \
  llama-index \
  llama-index-core \
  llama-index-llms-huggingface-api \
  llama-index-llms-openai-like

```

Then restart the Python process.

Print versions:

```python
import sys
import importlib.metadata as md

packages = [
    "smolagents",
    "huggingface_hub",
    "httpx",
    "openai",
    "llama-index",
    "llama-index-core",
    "llama-index-llms-huggingface-api",
    "llama-index-llms-openai-like",
]

print("Python:", sys.version)

for package in packages:
    try:
        print(f"{package}: {md.version(package)}")
    except md.PackageNotFoundError:
        print(f"{package}: not installed")

```

Why this matters:

- [`huggingface_hub` v1 changed the HTTP layer to `httpx`](https://huggingface.co/docs/huggingface_hub/concepts/migration).
- [LlamaIndex has many separate `llama-index-*` packages](https://developers.llamaindex.ai/python/framework/changelog/), and the changelog has explicitly warned in past releases that updating one package can require updating the rest.
- Notebook kernels can keep old imports alive after `pip install -U`.

* * *

## 9. Recommended debugging order

Use this exact order.

### Step 1 — verify environment

```python
import sys
import importlib.metadata as md

for package in [
    "smolagents",
    "huggingface_hub",
    "httpx",
    "openai",
    "llama-index",
    "llama-index-core",
    "llama-index-llms-huggingface-api",
]:
    try:
        print(package, md.version(package))
    except md.PackageNotFoundError:
        print(package, "not installed")

print(sys.version)

```

### Step 2 — direct Hugging Face non-streaming

```python
import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="<provider>",
    api_key=os.environ["HF_TOKEN"],
)

response = client.chat.completions.create(
    model="<qwen-model-id>",
    messages=[{"role": "user", "content": "Return exactly: ok"}],
    max_tokens=10,
    stream=False,
)

print(response.choices[0].message.content)

```

### Step 3 — direct Hugging Face streaming

```python
stream = client.chat.completions.create(
    model="<qwen-model-id>",
    messages=[{"role": "user", "content": "Count to 3."}],
    max_tokens=64,
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

```

If Step 2 works but Step 3 fails, the issue is specifically streaming/provider/client related.

### Step 4 — smolagents with no tools

```python
import os
from smolagents import CodeAgent, InferenceClientModel

model = InferenceClientModel(
    model_id="<qwen-model-id>",
    provider="<provider>",
    api_key=os.environ["HF_TOKEN"],
    max_tokens=512,
    temperature=0.2,
)

agent = CodeAgent(
    tools=[],
    model=model,
)

print(agent.run("Return exactly: ok"))

```

If this fails with the `stream` keyword error, focus on:

- package versions
- `smolagents` / `huggingface_hub` / `httpx` compatibility
- whether smolagents is internally streaming
- provider route

### Step 5 — LlamaIndex sync text generation

```python
import os
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI

llm = HuggingFaceInferenceAPI(
    model_name="<qwen-model-id>",
    token=os.environ["HF_TOKEN"],
    provider="<provider>",
    task="text-generation",
    is_chat_model=False,
    num_output=128,
    temperature=0.2,
)

print(llm.complete("Return exactly: ok").text)

```

If this works, but your app fails, the issue is not basic Qwen generation. It is likely one of:

- async
- streaming
- query engine reuse
- chat route
- object lifecycle

### Step 6 — LlamaIndex OpenAI-compatible path

```python
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="<qwen-model-id>:<provider>",
    api_base="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
    is_chat_model=True,
    is_function_calling_model=False,
    max_tokens=512,
    temperature=0.2,
)

response = llm.chat([
    {"role": "user", "content": "Return exactly: ok"}
])

print(response.message.content)

```

If this works, you can use it as the stable LlamaIndex route.

* * *

## 10. Decision table

| Symptom | Most likely cause | What to try |
| --- | --- | --- |
| `Client.post() got an unexpected keyword argument 'stream'` in smolagents | `stream` flag forwarded to wrong HTTP/client layer | Upgrade `smolagents`, `huggingface_hub`, `httpx`, `openai` together |
| Same smolagents code worked before, now fails | Provider auto-selection changed | Set provider explicitly |
| smolagents works without tools but fails with tools | Provider/model does not support tool calling | Use a tool-capable provider/model pair |
| Direct HF non-streaming works, streaming fails | Streaming support mismatch | Disable streaming or change provider |
| LlamaIndex fails with `client has been closed` | Async client closed then reused | Avoid async streaming or recreate LLM object per stream |
| LlamaIndex chat path fails but completion works | Fragile conversational/chat task route | Use `task="text-generation"` |
| LlamaIndex HF wrapper remains unstable | Wrapper/task/lifecycle mismatch | Use `OpenAILike` with HF router |
| Errors vary between notebook runs | Kernel has mixed old/new imports | Restart runtime after package changes |

* * *

## 11. What I would do in your case

### For smolagents

Use this first:

```python
from smolagents import CodeAgent, InferenceClientModel
import os

model = InferenceClientModel(
    model_id="Qwen/Qwen3-Coder-30B-A3B-Instruct",
    provider="together",
    api_key=os.environ["HF_TOKEN"],
    max_tokens=512,
    temperature=0.2,
)

agent = CodeAgent(
    tools=[],
    model=model,
)

print(agent.run("Return exactly: ok"))

```

Then add tools one by one.

If that still gives:

```bash
Client.post() got an unexpected keyword argument 'stream'

```

then assume package/API mismatch and reinstall cleanly:

```bash
python -m pip install -U \
  "smolagents[toolkit]" \
  huggingface_hub \
  httpx \
  openai

```

Then restart the runtime.

### For LlamaIndex

Use this first:

```python
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI
import os

llm = HuggingFaceInferenceAPI(
    model_name="Qwen/Qwen3-Coder-30B-A3B-Instruct",
    token=os.environ["HF_TOKEN"],
    provider="together",
    task="text-generation",
    is_chat_model=False,
    num_output=512,
    temperature=0.2,
)

print(llm.complete("Return exactly: ok").text)

```

If your actual app needs chat-style LlamaIndex usage, move to:

```python
from llama_index.llms.openai_like import OpenAILike
import os

llm = OpenAILike(
    model="Qwen/Qwen3-Coder-30B-A3B-Instruct:together",
    api_base="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
    is_chat_model=True,
    is_function_calling_model=False,
    max_tokens=512,
    temperature=0.2,
)

```

If your actual app needs async streaming, either:

- recreate the LLM object per streamed call
- avoid `astream_*` temporarily
- use sync calls
- switch to the OpenAI-compatible route

* * *

## 12. Why Qwen appears to be involved

Qwen is likely not the root cause, but Qwen makes the problem visible because Qwen models are often used with:

- coding agents
- tool calling
- chat completion
- OpenAI-compatible endpoints
- provider-backed inference
- streaming
- LlamaIndex RAG

All of those features stress integration layers.

A plain Qwen call may work:

```plaintext
Qwen + direct HF client + non-streaming

```

but an agent/RAG call may fail:

```plaintext
Qwen + wrapper + provider auto-selection + streaming + tools + async reuse

```

That is why the same model can look broken in smolagents and LlamaIndex for different reasons.

For Qwen-native agent/tool work, also inspect:

- [Qwen function calling docs](https://qwen.readthedocs.io/en/latest/framework/function_call.html)
- [Qwen-Agent GitHub repo](https://github.com/QwenLM/Qwen-Agent)
- [Qwen-Agent quickstart](https://qwenlm.github.io/Qwen-Agent/en/guide/get_started/quickstart/)
- [Qwen3 model card agentic-use note](https://huggingface.co/Qwen/Qwen3-8B)

The deeper point:

```plaintext
Tool calling is not just a model capability.
It is a contract among:
model output format + serving-layer parser + client wrapper.

```

* * *

## 13. Most likely final explanation

### smolagents

The smolagents wrapper is probably enabling or forwarding streaming, and the `stream` parameter is reaching a lower-level `Client.post()` method that does not accept it.

Most likely causes:

- `smolagents` / `huggingface_hub` / `httpx` version mismatch
- wrapper forwarding `stream=True` to the wrong layer
- provider route not supporting the requested streaming/tool mode
- implicit provider selection choosing a different backend than expected

Relevant links:

- [`huggingface_hub` v1 migration guide](https://huggingface.co/docs/huggingface_hub/concepts/migration)
- [`InferenceClient` reference](https://huggingface.co/docs/huggingface_hub/en/package_reference/inference_client)
- [Hugging Face streaming docs](https://huggingface.co/docs/text-generation-inference/conceptual/streaming)
- [smolagents provider/tool-calling issue](https://github.com/huggingface/smolagents/issues/1808)

### LlamaIndex

The LlamaIndex Hugging Face wrapper likely closes its internal async client after an async streaming call, and then the same object is reused.

Additionally, LlamaIndex’s Hugging Face wrapper can hit fragile task-route assumptions around chat/conversational behavior. A Qwen-related issue reported that setting `task="text-generation"` avoids a failing conversational route.

Relevant links:

- [LlamaIndex `HuggingFaceInferenceAPI` reference](https://developers.llamaindex.ai/python/framework-api-reference/llms/huggingface_api/)
- [LlamaIndex Hugging Face LLM integration](https://developers.llamaindex.ai/python/framework/integrations/llm/huggingface/)
- [LlamaIndex issue: Hugging Face conversational API returns 404](https://github.com/run-llama/llama_index/issues/18547)
- [LlamaIndex changelog](https://developers.llamaindex.ai/python/framework/changelog/)

* * *

## 14. Short summary

- Your smolagents error is probably a **streaming/client compatibility** problem.
- Your LlamaIndex error is probably an **async client reuse-after-close** problem.
- Qwen is likely not broken; the wrappers/routes around Qwen are the issue.
- Use **explicit provider** , not provider auto-selection.
- Start with **non-streaming** , **no tools** , **direct Hugging Face client**.
- For LlamaIndex, start with `task="text-generation"` and sync `.complete()`.
- For LlamaIndex chat, consider `OpenAILike` with the Hugging Face router.
- Upgrade related packages together and restart the runtime after upgrading.

Recommended debugging order:

```plaintext
1. Print package versions.
2. Test direct Hugging Face non-streaming.
3. Test direct Hugging Face streaming.
4. Test smolagents with tools=[].
5. Test LlamaIndex sync text-generation.
6. Test LlamaIndex OpenAI-compatible route.
7. Add tools.
8. Add streaming.
9. Add async.

```