# Gemma 4 bug fixes and Research Request

**URL:** <https://discuss.huggingface.co/t/gemma-4-bug-fixes-and-research-request/176979>\
**Category:** Research\
**Created:** [June 19, 2026, 10:44am UTC](https://discuss.huggingface.co/t/gemma-4-bug-fixes-and-research-request/176979 "2026-06-19T10:44:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zek-Takai](https://avatars.discourse-cdn.com/v4/letter/z/c5a1d2/32.png) [@Zek-Takai](https://discuss.huggingface.co/u/Zek-Takai)\
**Post date:** [June 19, 2026, 10:44am UTC](https://discuss.huggingface.co/t/gemma-4-bug-fixes-and-research-request/176979/1 "2026-06-19T10:44:10Z")

</div>

Gemma-4 has an ecosystem-wide agentic bug: under real load (long context + reasoning + generating  
content) it malforms its own tool calls and then loops on the broken output — reported across vLLM,  
llama.cpp, Ollama, and oobabooga, and unfixed by Google.

I open-sourced a full diagnosis + a reproduction harness + a working parser-level repair + a stock  
NVFP4 quant & recipe + an experimental format LoRA — with all raw data. Honest findings: the LoRA  
_reduces_ the slip (I ruled out quantization as the cause via a BF16 control) but doesn’t fully  
eliminate it yet — it overfit my synthetic data, so a bigger/more-diverse dataset is likely the real fix.

Everything’s here, Built with Gemma 👇 — looking for help pushing the LoRA further with richer data 🙏

> **[Zek-Takai/gemma4-toolcall-fix · Hugging Face](https://huggingface.co/Zek-Takai/gemma4-toolcall-fix)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [June 20, 2026, 3:28am UTC](https://discuss.huggingface.co/t/gemma-4-bug-fixes-and-research-request/176979/2 "2026-06-20T03:28:28Z")

</div>

Hmm… after reading your report and the recent model-repo/template updates, this looks pretty badly broken across a surprisingly wide part of the ecosystem…😱

* * *

## Short diagnosis

After reading the original report, the Hugging Face model-repo discussions/updates, and related runtime/client issues, I would frame this slightly differently from “Gemma 4 weights are broken”.

My current read:

> Gemma 4 probably has a real ecosystem-wide agentic failure mode, but the first-order failure does not look like a single model-weights bug. It looks more like a multi-layer protocol-boundary problem around Gemma 4’s native, non-JSON tool-call format and the OpenAI-compatible agent stacks trying to wrap it.

So I mostly agree with the “ecosystem-wide” part of the report. I would just be cautious about attributing the whole thing to the weights themselves.

Likely layers:

1. Gemma 4 native tool-call protocol
2. HF chat templates / tokenizer configs / model-repo packaging
3. GGUF or other converted artifacts with stale embedded templates
4. backend runtime parsers: vLLM, llama.cpp, Ollama, MLX, SGLang
5. streaming delta parsers
6. OpenAI-compatible proxy layers: LiteLLM / ADK-style adapters
7. coding-agent clients: OpenCode / AI SDK integrations
8. agent-loop recovery: retry, duplicate suppression, malformed-turn handling

The core mismatch is that Gemma 4 tool calls are not ordinary OpenAI-style JSON tool calls. Gemma 4 uses a native format like:

```plaintext
<|tool_call>call:func_name{key:<|"|>value<|"|>,num:42}<tool_call|>

```

That has different string delimiters, unquoted keys, and different multi-call behavior from JSON. Any layer that assumes “OpenAI-compatible endpoint = ordinary JSON tool-call transcript” can corrupt the conversation while converting, streaming, storing, or re-rendering tool calls.

Useful starting links:

- [Original Hugging Face forum report](https://discuss.huggingface.co/t/gemma-4-bug-fixes-and-research-request/176979)
- [Google — Function calling with Gemma 4](https://ai.google.dev/gemma/docs/capabilities/text/function-calling-gemma4)
- [Google — Gemma 4 model card](https://ai.google.dev/gemma/docs/core/model_card_4)
- [vLLM — Gemma4ToolParser docs](https://docs.vllm.ai/en/latest/api/vllm/tool_parsers/gemma4_tool_parser/)
- [vLLM — Gemma 4 usage guide](https://docs.vllm.ai/projects/recipes/en/latest/Google/Gemma4.html)

* * *

## What kind of failure is this?

I would call this a **multi-layer protocol-boundary failure** , or **ecosystem drift around Gemma 4 native tool calling**.

| Layer | Failure mode |
| --- | --- |
| Official model repo / chat template | OpenAI-shaped messages are rendered incorrectly into Gemma-native dialogue. |
| GGUF / artifact distribution | Old quantized artifacts embed stale chat templates. |
| Runtime parser | Gemma native tool syntax is parsed as JSON-ish text, or not parsed at all. |
| Streaming parser | Partial deltas corrupt arguments, numbers, booleans, or boundaries. |
| Proxy / OpenAI adapter | `role:"tool"` / `tool_calls.arguments` are translated incorrectly. |
| Client / coding agent | Backend returns tool calls, but client-side stream/event parser misses them. |
| Agent loop | Malformed calls are fed back into history, causing self-reinforcing loops. |
| LoRA / fine-tune | May reduce bad generations, but does not fix broken protocol conversion. |

This explains why reports differ. vLLM streaming users may see corrupted arguments. Ollama-through-LiteLLM users may see infinite tool loops. Old-GGUF users may be running stale templates. OpenCode users may have a backend that returns tool calls, while the client fails to consume them. These all look like “Gemma 4 tools are broken”, but they are not necessarily the same bug.

* * *

## The concrete bug pattern I would check first

The strongest pattern I would look for is **OpenAI-style JSON arguments being re-rendered into Gemma-native syntax incorrectly**.

OpenAI-compatible APIs often represent arguments as a JSON string:

```json
{
  "tool_calls": [
    {
      "type": "function",
      "function": {
        "name": "write_file",
        "arguments": "{\"path\":\"foo.txt\",\"content\":\"hello\"}"
      }
    }
  ]
}

```

Gemma native rendering instead wants a structured object that can serialize to Gemma’s DSL:

```plaintext
<|tool_call>call:write_file{path:<|"|>foo.txt<|"|>,content:<|"|>hello<|"|>}<tool_call|>

```

If the renderer inserts the OpenAI JSON string into Gemma braces, it may produce hybrid syntax:

```plaintext
call:write_file{{"path":"foo.txt","content":"hello"}}

```

That is neither valid OpenAI JSON tool calling nor proper Gemma native tool calling.

Then the loop becomes:

```plaintext
correct Gemma native tool call
  -> converted to OpenAI-compatible tool_calls
  -> arguments stored as JSON string
  -> later re-rendered into Gemma prompt
  -> JSON string inserted into Gemma-native braces
  -> malformed hybrid syntax appears in history
  -> model imitates malformed history
  -> parser fails
  -> retry re-injects poisoned turn
  -> loop

```

That is why I would treat **transcript re-rendering** and **tool-result mapping** as first-class suspects, not just sampling or LoRA.

The official HF model-repo template fixes point in the same direction. Recent Gemma 4 template discussions/fixes mention string-typed arguments, tool-response rendering, turn-tag balance, ordering, and thinking preservation:

- [google/gemma-4-31B-it discussion #118](https://huggingface.co/google/gemma-4-31B-it/discussions/118)
- [google/gemma-4-E4B-it discussion #36](https://huggingface.co/google/gemma-4-E4B-it/discussions/36)
- [google/gemma-4-12B-it discussion #12](https://huggingface.co/google/gemma-4-12B-it/discussions/12)
- [google/gemma-4-12B-it discussion #35](https://huggingface.co/google/gemma-4-12B-it/discussions/35)

So the official model repos are part of the fix surface, not only third-party runtimes.

* * *

## Why LoRA is probably not the durable first-line fix

Your LoRA attempt makes sense as an initial experiment: from the outside, the symptom can look like model-side format drift.

But after looking at the model-repo template changes and related runtime/client issues, I would treat LoRA as a behavioral mitigation rather than the most likely durable root fix.

| Approach | Can help with | Cannot fix |
| --- | --- | --- |
| LoRA / fine-tune | Model behavior, format preference, tool-use tendency | Broken parser, streaming delta, transcript re-render, wrong role mapping |
| Template fix | Correct Gemma-native rendering | Client-side stream parser bugs |
| Runtime parser fix | Native extraction and conversion | Proxy rewriting tool results incorrectly |
| Proxy/adapter fix | OpenAI messages ↔ Gemma-native semantics | Runtime parser bugs |
| Agent-loop healing | Retry safety, duplicate suppression, malformed-call containment | Incorrect canonical protocol implementation |

So I would not say “LoRA is useless”. I would say: **LoRA is not the durable root fix if the failure is protocol-boundary corruption**.

* * *

## Fix vs workaround

### Durable fixes

| Layer | Who should fix it | Durable fix |
| --- | --- | --- |
| Official model repo / HF | Google / HF maintainers | Canonical chat template, tokenizer config, response schema, tool-response rendering, examples. |
| Artifact / GGUF | Quant providers, Unsloth, LM Studio community, Bartowski-style distributors | Re-export/re-quant with fixed template metadata; provide known-good template overrides. |
| Backend runtime | vLLM, llama.cpp, Ollama, MLX, SGLang | Gemma-native parser/serializer, schema handling, streaming delta handling, reasoning/channel handling. |
| Proxy / adapter | LiteLLM, ADK, OpenAI-compatible bridges | Correct `role:"tool"` ↔ Gemma `tool_responses`; deserialize OpenAI `arguments` before Gemma rendering. |
| Client / coding agent | OpenCode, AI SDK integrations, LM Studio, Claude Code/OpenClaw adapters | Recognize streamed tool-call events, preserve IDs, support model-specific parser hooks. |
| Agent loop | App/framework authors | Retry cap, duplicate suppression, malformed-turn suppression, final-answer nudges, tool-call healing. |
| Third-party bridge repo | Community / researchers | Compatibility matrix, patched adapter, prompt-dump checker, regression suite, known-good combinations. |

### Practical workaround decision tree

`stream:false` helps streaming parser bugs, but it will not fix stale GGUF templates or proxy role-mapping bugs.

```plaintext
0. Update first:
   backend runtime
   model repo files
   tokenizer_config / chat_template
   GGUF / quantized artifact
   proxy / agent framework

1. If using GGUF:
   re-download a post-template-fix artifact, or use a runtime/UI that overrides stale embedded templates.
   Do not assume updating only the binary updates the embedded model template.

2. Test backend directly:
   bypass LiteLLM / ADK / OpenCode / Studio / OpenAI-compatible proxy.

3. If backend-direct works:
   suspect proxy / adapter / client layer.
   check role:"tool" vs tool_responses.
   check whether OpenAI function.arguments JSON strings are deserialized before Gemma rendering.
   check whether streamed tool-call events are recognized by the client.

4. If backend-direct fails:
   try stream:false for tool-call requests.

5. If stream:false fixes it:
   likely streaming delta / parser bug.
   keep tool calls non-streaming until that path is fixed.

6. If stream:false does not fix it:
   disable MTP / speculative decoding if enabled.

7. If single-turn works but second-turn fails:
   suspect tool_response mapping or transcript re-rendering.
   inspect the final prompt.

8. If first-turn fails:
   suspect native parser, chat template, schema complexity, or unsupported runtime.
   simplify schema and test a minimal tool.

9. If loops repeat:
   cap retries.
   suppress duplicate tool calls.
   never feed malformed assistant tool-call turns back into history.
   add a clean nudge or abort instead of replaying poisoned output.

```

* * *

## Prompt-dump checks

Dump the exact prompt reaching the model if possible.

Suspicious:

```plaintext
call:NAME{{"key":"value"}}
{{"key":
role: "tool"
<channel|>
<|tool_call> ... raw JSON ... <tool_call|>

```

Suspicious template input:

```json
{
  "function": {
    "arguments": "{\"x\":1}"
  }
}

```

Preferred template input:

```json
{
  "function": {
    "arguments": {
      "x": 1
    }
  }
}

```

Then the Gemma serializer can emit native syntax:

```plaintext
call:some_tool{x:1}

```

or:

```plaintext
call:some_tool{x:<|"|>value<|"|>}

```

depending on type.

* * *

## Minimal regression tests

Before calling something “Gemma 4 tool support”, I would run:

```plaintext
1. Single tool call.
2. Tool call -> tool result -> final answer.
3. Tool call -> tool result -> second tool call.
4. Multiple tool calls in one assistant turn.
5. stream:false vs stream:true.
6. MTP/speculative decoding on/off.
7. Long string argument containing comma, colon, braces, quotes.
8. Boolean / null / integer / decimal arguments.
9. Code/html argument.
10. Malformed tool call retry.
11. Proxy vs backend-direct.
12. Old GGUF embedded template vs patched template override.
13. Assistant content + tool_calls in the same turn.
14. Consecutive tool results.
15. Repeated identical tool call suppression.

```

Good torture string:

```plaintext
Fix: deploy, retry: twice, reason: "missing { brace } in HTML"

```

A regex-ish parser will often break this. A real string-aware parser should not.

* * *

> **Known related issues and fixes by layer**
>
> ## Layer 0 — Official model repo / chat template / tokenizer config
> 
> The official repos are part of the fix surface because the chat template defines how OpenAI-shaped messages become Gemma-native dialogue.
> 
> | Project / link | Status | What it suggests |
> | --- | --- | --- |
> | [google/gemma-4-31B-it discussion #118](https://huggingface.co/google/gemma-4-31B-it/discussions/118) | Template-fix discussion / PR | Covers string arguments, null rendering, turn-tag balance, thinking preservation, OpenAI-compatible content aliases. |
> | [google/gemma-4-E4B-it discussion #36](https://huggingface.co/google/gemma-4-E4B-it/discussions/36) | Similar family-level fix | Suggests this was not only a 31B issue. |
> | [google/gemma-4-12B-it discussion #12](https://huggingface.co/google/gemma-4-12B-it/discussions/12) | Ordering issue | Deals with `content + tool_calls + tool_responses` ordering. |
> | [google/gemma-4-12B-it discussion #35](https://huggingface.co/google/gemma-4-12B-it/discussions/35) | Thinking/tool-call context | Suggests thinking needs different handling in tool-call chains than normal chat. |
> | [Google Gemma 4 function calling docs](https://ai.google.dev/gemma/docs/capabilities/text/function-calling-gemma4) | Official docs | Function calling is an app/developer loop, not magic execution inside the model. |
> 
> Interpretation: official template fixes make this look more like a **distribution/runtime contract** issue than a pure weight issue.
> 
> * * *
> 
> ## Layer 1 — Artifact / GGUF / template distribution
> 
> For GGUF users, updating the runtime may not be enough. Old artifacts can carry stale embedded templates or stale conversion metadata.
> 
> | Project / link | Status | What it suggests |
> | --- | --- | --- |
> | [Unsloth Gemma 4 31B GGUF discussion #16](https://huggingface.co/unsloth/gemma-4-31B-it-GGUF/discussions/16) | Updated quants | Re-uploaded Gemma 4 GGUFs with Google chat-template fixes and llama.cpp fixes. |
> | [Unsloth Gemma 4 31B GGUF discussion #20](https://huggingface.co/unsloth/gemma-4-31B-it-GGUF/discussions/20) | Provider update note | Says Google updated the official template and providers should update quants/implementations. |
> | [Unsloth Gemma 4 docs](https://unsloth.ai/docs/models/gemma-4) | User docs | Advises re-downloads around updated templates/fixes. |
> | [LM Studio issue #2012](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2012) | Template crash | Treats some Gemma 4 tool/API failures as prompt-template issues and points to fixed templates. |
> 
> Interpretation: practical advice is **update backend and artifact** , not just “update backend”.
> 
> * * *
> 
> ## Layer 2 — Backend runtime parser
> 
> Runtime parser bugs are real and numerous.
> 
> | Runtime / link | Status | What it suggests |
> | --- | --- | --- |
> | [vLLM Gemma4ToolParser docs](https://docs.vllm.ai/en/latest/api/vllm/tool_parsers/gemma4_tool_parser/) | Official docs | Gemma 4 tool calls use custom serialization, not JSON. |
> | [vLLM Gemma4 utils docs](https://docs.vllm.ai/en/latest/api/vllm/tool_parsers/gemma4_utils/) | Parser utilities | Standalone Gemma 4 parsing utilities for raw decoded strings. |
> | [vLLM issue #38946](https://github.com/vllm-project/vllm/issues/38946) | Closed | Streaming parser produced invalid JSON for Gemma 4 tool usage. |
> | [vLLM issue #39069](https://github.com/vllm-project/vllm/issues/39069) | Closed | Offline parser quote/truncation issue. |
> | [MLX issue #1096](https://github.com/ml-explore/mlx-lm/issues/1096) | Closed / parser area | Gemma 4 native tool calls returned as raw content instead of parsed tool calls. |
> | [MLX issue #1125](https://github.com/ml-explore/mlx-lm/issues/1125) | Edge-case report | Tool-call extraction still had edge cases after parser support. |
> | [llama.cpp discussion #21839](https://github.com/ggml-org/llama.cpp/discussions/21839) | Design discussion | Gemma 4 notation is not ordinary JSON schema grammar. |
> 
> Interpretation: parser fixes must be **Gemma-native, string-aware, not just JSON repair**.
> 
> * * *
> 
> ## Layer 3 — Streaming / MTP / concurrency
> 
> Streaming is its own failure class.
> 
> | Project / link | Status | What it suggests |
> | --- | --- | --- |
> | [vLLM issue #38946](https://github.com/vllm-project/vllm/issues/38946) | Closed | Streaming parser could emit invalid JSON. |
> | [vLLM issue #39380](https://github.com/vllm-project/vllm/issues/39380) | Reported | Streaming tool calls could corrupt decimals; non-streaming was reported correct. |
> | [vLLM issue #39392](https://github.com/vllm-project/vllm/issues/39392) | Reported | Concurrent Gemma4ToolParser / `<pad>`-storm-like behavior. |
> | [vLLM issue #41967](https://github.com/vllm-project/vllm/issues/41967) | Reported | MTP/speculative + streaming + multi-tool edge case. |
> | [vLLM PR #44741](https://github.com/vllm-project/vllm/pull/44741) | Related PR | Narrow fix around multi-boundary streamed deltas. |
> | [OpenCode issue #20995](https://github.com/anomalyco/opencode/issues/20995) | Open | Backend returns tool calls, but streaming/client layer does not recognize them. |
> 
> Interpretation: if `stream:false` fixes it, use non-streaming tool calls. If not, look elsewhere.
> 
> * * *
> 
> ## Layer 4 — Proxy / OpenAI-compatible adapter
> 
> Backend-direct working does not prove the full stack works.
> 
> | Project / link | Status | What it suggests |
> | --- | --- | --- |
> | [ADK issue #5650](https://github.com/google/adk-python/issues/5650) | Closed | Gemma via Ollama + LiteLLM entered infinite loop because of wrong tool message role. |
> | [LiteLLM issue #28530](https://github.com/BerriAI/litellm/issues/28530) | Open/partial | Gemma 4 expects `tool_responses`, while LiteLLM standardized to OpenAI-compatible `tool`. |
> | [LiteLLM function-calling docs](https://docs.litellm.ai/docs/completion/function_call) | General docs | OpenAI-style function calling is the assumed abstraction; Gemma may need adaptation. |
> | [ADK Gemma docs](https://adk.dev/agents/models/google-gemma/) | ADK docs | ADK supports Gemma tool calling, but adapter correctness still matters. |
> | [Unsloth issue #4999](https://github.com/unslothai/unsloth/issues/4999) | Related proxy report | Same backend/model/flags worked directly but failed through Studio proxy. |
> 
> Interpretation: many “model failed to use tools” reports may be **tool result not recognized** because the proxy mapped the role/transcript incorrectly.
> 
> * * *
> 
> ## Layer 5 — Client / coding agent / UI integration
> 
> This layer is distinct from the backend.
> 
> | Project / link | Status | What it suggests |
> | --- | --- | --- |
> | [OpenCode issue #20995](https://github.com/anomalyco/opencode/issues/20995) | Open | Ollama returns tool calls, but OpenCode / AI SDK streaming path does not recognize them. |
> | [OpenCode issue #21034](https://github.com/anomalyco/opencode/issues/21034) | Open | Gemma 4 26B/31B still loop/fail in OpenCode even with updated tokenizers, patched engines, LM Studio improvements, updated quants. |
> | [OpenCode issue #21850](https://github.com/anomalyco/opencode/issues/21850) | Reported | Gemma 4 entered repetitive tool/edit failure loop. |
> | [LM Studio issue #1732](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1732) | Fixed-ish / related | Gemma 4 repeated the same tool call in Claude Code integration. |
> | [LM Studio issue #1749](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1749) | Open | Gemma 4 Jinja render crash with OpenClaw tool payload. |
> | [LM Studio issue #2048](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2048) | Reported | Tool subset / prompt-template issue with Gemma 4 compressed model. |
> 
> Interpretation: “backend returned a tool call” is not enough if the client cannot consume it.
> 
> * * *
> 
> ## Layer 6 — Agent-loop recovery / healing
> 
> Unsloth is a useful reference here. It treats tool-call robustness as an agent-loop problem, not only a parser problem.
> 
> | Project / link | Status | What it suggests |
> | --- | --- | --- |
> | [Unsloth discussion #4800](https://github.com/unslothai/unsloth/discussions/4800) | Release discussion | Mentions Gemma 4, tool-call improvements, healing, de-duplication, loop reduction. |
> | [Unsloth discussion #4732](https://github.com/unslothai/unsloth/discussions/4732) | Tool accuracy discussion | Advertises improved tool-call accuracy and healing. |
> | [Unsloth PR #5583](https://github.com/unslothai/unsloth/pull/5583) | Tool-healing extraction | Exposes standalone tool-healing utilities for external inference-server wrappers. |
> | [Unsloth issue #5502](https://github.com/unslothai/unsloth/issues/5502) | Feature request | Asked to decouple Tool Healing for external inference servers. |
> | [Unsloth PR #4783](https://github.com/unslothai/unsloth/pull/4783) | Re-prompt fix | Improves re-prompting when small models answer instead of calling tools. |
> | [Unsloth PR #4733](https://github.com/unslothai/unsloth/pull/4733) | Tool lifecycle UI | Clears tool status after execution; minor, but shows tool-execution lifecycle handling. |
> | [Unsloth Studio chat docs](https://unsloth.ai/docs/new/studio/chat) | User docs | Describes self-healing tool calling in Studio. |
> | [Unsloth changelog](https://unsloth.ai/docs/new/changelog) | Changelog | Mentions healing, de-duplication, termination, reduced malformed calls. |
> 
> Interpretation: Unsloth is a useful reference implementation of the mitigation stack: patched templates, stale-template override, reusable parser/stripper utilities, re-prompting, duplicate/loop suppression, and UI/runtime lifecycle fixes.
> 
> * * *
> 
> ## Layer 7 — LoRA / fine-tune / behavioral mitigation
> 
> I would place LoRA here, not at the root.
> 
> | Project / link | Status | What it suggests |
> | --- | --- | --- |
> | Original forum post | Experimental | LoRA was a reasonable initial attempt if the symptom looked like model format drift. |
> | [roshangrewal/gemma4-e4b-toolcall-v01-lora](https://huggingface.co/roshangrewal/gemma4-e4b-toolcall-v01-lora) | Third-party LoRA | Fine-tuning/tool-call behavior patching is being explored. |
> | [Google Gemma 4 model card](https://ai.google.dev/gemma/docs/core/model_card_4) | Official capability claim | Gemma 4 has native function-calling support; failures are not automatically evidence of missing capability. |
> 
> Interpretation: LoRA may help if the model often chooses bad syntax or refuses tools, but protocol conversion bugs should be fixed structurally.
> 
> * * *
> 
> ## Areas still unclear / need confirmation
> 
> | Area | What is unclear |
> | --- | --- |
> | Ollama version matrix | Exact fixed/regressed boundary across `0.20.0-rc1`, `0.20.1`, `0.20.5`, `0.20.6`, `0.20.7`, `0.21.x`, `0.30.x`. |
> | SGLang | It has explicit Gemma 4 parser support, but I found fewer Gemma4-tool-call-specific failure reports than vLLM/Ollama/OpenCode. |
> | OpenCode PR #16531 | The issue references a custom tool compatibility layer, but I have not fully traced final merge/replacement status. |
> | lmstudio-community fixed templates | LM Studio points users toward fixed prompt templates, but I have not fully traced the exact Gemma 4 template artifact lineage. |

* * *

> **Suggested known-good debugging protocol**
>
> Minimal backend-direct test:
> 
> ```plaintext
> 1. One tiny tool:
> get_time(location: string) -> string
> 
> 2. One prompt:
> "What time is it in Tokyo? Use the tool."
> 
> 3. Run backend direct:
> no proxy
> no coding agent
> no streaming
> no MTP/speculative decoding
> 
> 4. Confirm:
> model emits a structured tool call
> backend parses it as tool_calls
> arguments are valid and not raw content
> 
> ```
> 
> Then add complexity one dimension at a time:
> 
> ```plaintext
> A. Add tool result:
> feed result back, check final answer, inspect rendered prompt.
> 
> B. Multi-turn tool chain:
> tool call -> result -> second tool call.
> confirm prior tool calls are rendered in valid Gemma syntax.
> 
> C. Argument torture test:
> commas, colons, quotes, braces, HTML, code, decimals, booleans, null.
> 
> D. Stack layers:
> backend direct
> backend direct + stream:true
> backend direct + MTP/speculative
> proxy
> coding agent
> full UI
> 
> ```
> 
> If it only fails after adding a layer, that layer is the first suspect.

* * *

## Bottom line

I would not frame the durable fix as:

> “Make Gemma 4 learn tool calling via LoRA.”

I would frame it as:

> “Every layer that claims Gemma 4 support needs to handle Gemma 4’s native tool protocol losslessly, and agent loops need safeguards for malformed-output cases.”

Durable fix surface:

```plaintext
official template
  + artifact freshness
  + native runtime parser
  + streaming parser
  + OpenAI-compatible adapter
  + client event parser
  + transcript re-renderer
  + agent-loop healing

```

Immediate user workaround:

```plaintext
update everything
  -> verify artifact/template freshness
  -> test backend direct
  -> disable streaming for tool calls if needed
  -> disable MTP/speculative if needed
  -> inspect prompt dumps
  -> fix tool_response mapping
  -> prevent poisoned retries

```

So yes: this looks pretty badly broken across the ecosystem, but not necessarily because the Gemma 4 weights are fundamentally unable to do tools. It looks more like the ecosystem is still settling around a non-JSON native tool protocol that does not fit cleanly into existing OpenAI-compatible agent assumptions.

---

<div class="post-metadata">

**Author:** ![Zek-Takai](https://avatars.discourse-cdn.com/v4/letter/z/c5a1d2/32.png) [@Zek-Takai](https://discuss.huggingface.co/u/Zek-Takai)\
**Post date:** [June 20, 2026, 4:04am UTC](https://discuss.huggingface.co/t/gemma-4-bug-fixes-and-research-request/176979/3 "2026-06-20T04:04:02Z")

</div>

yes and i removed my page. VLLM v23 finally fixed the issues. It now runs via hermes naturally.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [June 20, 2026, 9:27pm UTC](https://discuss.huggingface.co/t/gemma-4-bug-fixes-and-research-request/176979/5 "2026-06-20T21:27:24Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.