# Can this model run well on a Mac mini with OpenCode? **URL:** \ **Category:** Beginners\ **Created:** [September 1, 2026, 9:57am UTC](https://discuss.huggingface.co/t/can-this-model-run-well-on-a-mac-mini-with-opencode/179660 "2026-09-01T09:57:51Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![sophie-webber](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sophie-webber](https://discuss.huggingface.co/u/sophie-webber)\ **Post date:** [September 1, 2026, 9:57am UTC](https://discuss.huggingface.co/t/can-this-model-run-well-on-a-mac-mini-with-opencode/179660/1 "2026-09-01T09:57:52Z")
### Hi everyone, I’m considering buying a Mac mini specifically to run **Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated** locally for software development. My main use case is **agentic coding with OpenCode**. I would like to use the model as the main coding agent: reading a repository, modifying multiple files, running tests, debugging errors, and iterating through tool calls. I’m currently considering a **Mac mini with 64 GB of unified memory** (Apple Silicon). My questions are: 1. Can this model run reliably on a Mac mini with 64 GB unified memory? 2. Which quantization would you recommend for this use case — Q4, Q5, Q6, or something else? 3. Is **llama.cpp/Metal** the recommended way to run it on Apple Silicon, or would you recommend another backend? 4. Does the model work well with **OpenCode’s tool calling / agentic workflow**? 5. What kind of generation speed (tokens/sec) could I realistically expect on a 64 GB Mac mini? 6. Would a Mac mini be a reasonable choice for this model, or would renting a cloud GPU provide a significantly better experience? I’m primarily interested in **coding quality and agentic tool use** , rather than maximum benchmark scores. If anyone is already running this model (or a similar 30–40B MoE model) on a Mac mini, I would really appreciate your experience, including your Mac configuration, quantization, backend, context size, and approximate tokens/sec. Thanks! ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [September 1, 2026, 9:28pm UTC](https://discuss.huggingface.co/t/can-this-model-run-well-on-a-mac-mini-with-opencode/179660/2 "2026-09-01T21:28:01Z")
I don’t own any Apple hardware, but I think this **should probably be quite usable** without too much trouble: * * * Short version: **yes, I think a 64 GB Mac mini is a reasonable machine for this model and this use case.** I would worry much less about whether the model fits, and much more about whether the exact checkpoint + runtime + OpenCode combination remains reliable during a long tool-using coding session. If I were setting this up, my default starting point would probably be: - **64 GB unified memory** - **Q5\_K\_M or Q6\_K** - **llama.cpp / Metal + `llama-server`** - **OpenCode through its OpenAI-compatible llama.cpp provider** - **~64K context initially** , rather than immediately maximizing context - leave reasoning/tool behavior close to the model/runtime defaults at first - only change quantization/backend after verifying that the tool loop itself is stable The six questions look roughly like this to me: | Question | My current guess | | --- | --- | | 1. 64 GB enough? | **Yes, comfortably enough for the useful quants.** | | 2. Quant? | **Q5\_K\_M as a conservative default; Q6\_K if quality is the priority; Q4\_K\_M is also quite plausible.** | | 3. Backend? | **llama.cpp/Metal is a very sensible starting point. MLX/oMLX is also worth considering on Apple Silicon.** | | 4. OpenCode/tool use? | **Promising, but this is the part I would validate before treating the setup as production-stable.** | | 5. Speed? | Expect **tens of tokens/sec** , not single-digit speeds, but context/prefill/backend matter a lot. | | 6. Mac vs cloud? | **Mac makes sense for a single-user, always-available local agent; cloud wins for burst throughput, very large prefills, or parallel agents.** | ## Memory / quantization The [available GGUFs for this exact Huihui model](https://huggingface.co/shennguyen/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-GGUF/tree/main) are approximately: | Quant | File size | | --- | --- | | Q4\_K\_M | 21.2 GB | | Q5\_K\_M | 24.7 GB | | Q6\_K | 28.5 GB | | Q8\_0 | 36.9 GB | So 64 GB is not a “Q4 or it will not fit” situation. I would probably start with **Q5\_K\_M** because it leaves a lot of headroom for macOS, context/cache state, OpenCode, an editor, test processes, etc. If coding quality is much more important than maximizing context, **Q6\_K** is also very reasonable. I would not automatically reject Q4 for agentic work, though. I did a small sanity test with the **exact Q4\_K\_M Huihui checkpoint** , and its basic tool protocol behaved cleanly in that test; more on that below. So I would not assume that a tool-calling problem seen with Q4 is necessarily caused by quantization. Q8 also fits in 64 GB, but “the weights fit” is not the same as “I still have comfortable room for a long agent session”. I would only move to Q8 if an A/B test showed an actual coding-quality benefit worth the reduced memory margin. ## Backend: llama.cpp first, MLX/oMLX as the other obvious option For OpenCode specifically, **llama.cpp is probably the easiest baseline** because OpenCode [documents `llama-server` as a local provider](https://opencode.ai/docs/providers/) using an OpenAI-compatible endpoint. A minimal starting point could look roughly like: ```bash llama-server \ --hf-repo shennguyen/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-GGUF \ --hf-file huihui-qwen3.6-35b-a3b-claude-4.7-opus-abliterated-q5_k_m.gguf \ --alias huihui-qwen36 \ --jinja \ -c 65536 \ --host 127.0.0.1 \ --port 8080 ``` Current llama.cpp builds have Metal enabled by default on macOS, and current `llama-server` also defaults GPU-layer selection to `auto`; the current server options are documented [here](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md). Then something along these lines in OpenCode: ```json { "$schema": "https://opencode.ai/config.json", "provider": { "llama.cpp": { "npm": "@ai-sdk/openai-compatible", "name": "llama-server (local)", "options": { "baseURL": "http://127.0.0.1:8080/v1" }, "models": { "huihui-qwen36": { "name": "Huihui Qwen3.6 35B-A3B", "limit": { "context": 65536, "output": 32768 } } } } } } ``` I would treat those context/output numbers as a starting configuration, not sacred values. **MLX/oMLX is the other backend I would keep on the shortlist.** Apple Silicon is exactly the environment MLX is designed for, and there are already some very useful performance measurements for this exact checkpoint. For example, oMLX reports this exact Huihui derivative on an **M4 Pro 16-core GPU / 64 GB** system at 8-bit: - ~54.9 generation tok/s at 4K context - ~36.2 GB peak memory [Benchmark](https://omlx.ai/benchmarks/performance/4sslge3j) And on an **M5 Pro 20-core GPU / 64 GB** system at 8-bit: - ~44.8 generation tok/s at 64K - ~34.9 generation tok/s at 128K - ~41.2 GB peak at 64K - ~47.9 GB peak at 128K [Benchmark](https://omlx.ai/benchmarks/performance/ecgl0xff) Those are **oMLX numbers, not llama.cpp/Metal numbers** , so I would not use them as a direct llama.cpp performance prediction. But they are good evidence that the model itself is quite practical on a 64 GB Apple Silicon system. Apple’s newly announced [M5 Pro Mac mini](https://www.apple.com/newsroom/2026/08/apple-unveils-a-more-powerful-mac-mini-featuring-the-all-new-m6-and-m5-pro/) supports up to 64 GB unified memory and 307 GB/s memory bandwidth, so if that is the configuration you are considering, the hardware side looks particularly plausible. ## What I would expect for speed I would mentally budget for **“tens of tokens/sec”** , rather than trying to predict one exact number. There are really at least two different speeds that matter for a coding agent: 1. **prompt processing / prefill** 2. **token generation** Generation can remain quite fast even when the agent experience starts feeling slow because the growing repository/tool history has to be processed. The M5 Pro oMLX measurement above is a useful example: at 128K it still generates at ~34.9 tok/s, but the measured time-to-first-token for a full 128K prompt was about **158 seconds**. That is why I would not optimize the setup around maximum advertised context on day one. For an agent, good prefix/cache reuse can matter as much as raw decode speed. A practical sequence might be: ```plaintext 32K or 64K ↓ normal real repository work ↓ only increase context if you actually need it ``` rather than: ```plaintext "the model says 262K" ↓ set 262K immediately ``` The [base Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) has a default 262,144-token context and Qwen recommends keeping at least 128K for its full long-context reasoning behavior. But this derivative is not literally the base checkpoint. The [reasoning-distilled parent](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled) was trained on 4096-token sequences, and its author describes 64K as usable at inference while also warning that hard problems may generate tens of thousands of reasoning tokens. So I would treat extremely long-context quality on this derivative as something to validate, not something automatically inherited just because the underlying config permits it. ## OpenCode / agentic coding is the main thing I would test There is actually encouraging evidence here. The **base Qwen3.6-35B-A3B is explicitly evaluated as an agentic model**. Its [official model card](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) includes SWE-bench-family results, Terminal-Bench, MCP-style evaluations, etc. More directly relevant to your question: **Qwen’s SkillsBench evaluation was run through OpenCode** , on 78 self-contained tasks, averaged over five runs. So I would not be worried that Qwen3.6-35B-A3B is fundamentally the wrong kind of model for OpenCode. The uncertainty is narrower: > Does this exact `reasoning-distilled -> abliterated -> quantized -> local runtime` derivative preserve that behavior reliably over a long OpenCode trajectory? That part is much less directly benchmarked. The lineage is: ```plaintext Qwen3.6-35B-A3B ↓ Claude Opus 4.7 reasoning distillation ↓ Huihui abliteration ↓ GGUF quantization ``` The [reasoning-distilled model](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled) was trained on roughly 7,800 Claude Opus reasoning conversations with an attention-only LoRA. Then the [Huihui checkpoint](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated) applies abliteration. Huihui describes that implementation itself as a “crude, proof-of-concept implementation”. That does **not** mean it will be bad at tools. It only means I would not directly copy the base model’s OpenCode benchmark onto the derivative and call the question settled. ### A small sanity check on the exact checkpoint I tried a deliberately small tool-protocol sanity test with the exact: ```plaintext Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated Q4_K_M ``` The purpose was not to benchmark a Mac, but simply to see whether the checkpoint itself could still produce the tool syntax defined by its GGUF chat template. I tested: - thinking enabled - thinking disabled - repeated calls after an explicit reset - repeated calls reusing the same model context - a case where no tool should be called - a longer system prompt - 20 irrelevant/distractor tools - more than one tool call in one response - a short multi-turn sequence where tool results were fed back to the model In that small test: - the required tool was selected consistently - the no-tool case did not spuriously invoke one - multiple tool calls were emitted correctly - the returned XML was balanced and parseable - thinking-on and thinking-off both worked - the short multi-turn sequence completed correctly So I would move this hypothesis quite far down the list: > “The Huihui Q4 checkpoint itself simply cannot do tool calling.” At least in a short controlled test, it clearly can. That still leaves a separate integration question, though. > **Why a model can emit correct tools and OpenCode can still fail** > > A local coding agent has several layers: > > ```plaintext > model > ↓ > chat template > ↓ > raw tool-call syntax > ↓ > runtime parser > ↓ > OpenAI-compatible tool_calls JSON > ↓ > OpenCode / AI SDK > ↓ > tool execution > ↓ > tool-result history > ↓ > next turn > > ``` > > A failure in one layer can look like a failure in another. > > Qwen3.6’s template can use an XML-like intermediate representation for tool calls. A serving runtime such as llama.cpp parses that representation and exposes it as structured OpenAI-style `message.tool_calls`. > > So seeing literal: > > ```plaintext > > ... > > > ``` > > in an agent UI does not necessarily mean “Qwen decided to use the wrong API format”. It can also mean the conversion step did not happen correctly. > > There is a particularly relevant **open OpenCode issue** where Qwen3.6 + llama.cpp on macOS eventually exposes a naked tool call during a long coding session: > > [OpenCode #24316 — Qwen3.6 progress halts with naked tool call](https://github.com/anomalyco/opencode/issues/24316) > > The report shows the failure around **94.5K context** during an actual long-running repository task. > > Importantly, the reporter explicitly does **not** establish whether the root cause is Qwen, llama.cpp, or OpenCode. > > That is a useful failure example, but not evidence that every Qwen3.6/OpenCode session fails. > > There have also been llama.cpp-side parser edge cases. For example: > > [llama.cpp #26763](https://github.com/ggml-org/llama.cpp/issues/26763) > > documents a Qwen3.6 case where a tiny formatting variation around `` caused the parser to consume following tool-call text as part of an argument. That issue is now **closed as not planned** , so I would treat it as evidence about the type of boundary failure that can occur, not as an “open blocker”. > > There is also a currently open issue involving this combination: > > ```plaintext > Qwen3 template > + enable_thinking=false > + tool_choice=required/named tool > > ``` > > where the forced tool constraint is not enforced: > > [llama.cpp #27767](https://github.com/ggml-org/llama.cpp/issues/27767) > > That is one reason I would initially leave reasoning/tool behavior close to the normal template defaults instead of immediately forcing a special no-thinking configuration. > > The important practical point is just: > > **tool reliability belongs to the model + template + runtime parser + client combination, not the model alone.** ## A very cheap diagnostic if OpenCode misbehaves I would not build a giant evaluation harness first. If a real OpenCode task fails, the highest-information first comparison is probably: ```plaintext same model same llama-server same prompt/task OpenCode vs direct /v1/chat/completions request ``` If the direct API produces a valid structured tool call but OpenCode fails, the search space gets much smaller. If both fail, then test: ```plaintext cache/reuse ON vs fresh/reset path ``` If the problem is still present, then: ```plaintext base Qwen3.6 vs Huihui derivative ``` Only after that would I spend much time on: ```plaintext Q4 vs Q5 vs Q6 ``` In other words, I would diagnose in roughly this order: ```plaintext 1. Client boundary OpenCode vs direct llama-server 2. Runtime/state boundary cache/reuse vs fresh state 3. Model/checkpoint boundary base Qwen3.6 vs Huihui 4. Quantization Q4 vs Q5 vs Q6 ``` That order gives a lot of information without requiring a large benchmark. ## One OpenCode-specific performance trap If OpenCode feels much slower than calling the local model directly, do not immediately blame Apple Silicon or the model. There is an open report where a local llama.cpp request took about **2.5 seconds directly** , but about **22 seconds through OpenCode** ; supplying a fixed session title reduced the OpenCode path to roughly 5 seconds. [OpenCode #33126 — local llama.cpp models slow with session title generation](https://github.com/anomalyco/opencode/issues/33126) That does not mean you will reproduce those exact numbers. It is just a good example of why I would measure: ```plaintext direct llama-server latency vs OpenCode end-to-end latency ``` before deciding the local hardware is too slow. ## Mac mini vs cloud GPU For **one developer running one main coding agent** , I think the Mac mini is a very defensible choice. The Mac is attractive if you value: - always-on local availability - no per-hour inference bill - privacy/local repositories - quiet operation - simple unified-memory access to a fairly large model - long daily usage Cloud is more attractive if you need: - very fast cold-prefill of huge contexts - multiple simultaneous agents - very high throughput - experimentation with much larger models - occasional burst use rather than continuous use - the ability to change GPU size on demand The trade-off is therefore not really: ```plaintext Mac = slow cloud = fast ``` It is closer to: ```plaintext Mac: persistent, private, predictable, single-user Cloud: elastic, high-throughput, high-prefill, parallel ``` For the workload you described — one primary coding agent reading a repository, editing, testing, debugging, and iterating — I would not consider a 64 GB Mac mini an unreasonable compromise at all. I would probably only prefer cloud by default if you already know that you routinely need huge contexts with fast first-token latency, several simultaneous agents, or models substantially larger than this one. > **A little more context on why this 35B model is relatively practical** > > The [base model](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) is nominally a 35B model, but it is an MoE: > > - 35B total parameters > - ~3B activated per token > - 40 layers > - hybrid Gated DeltaNet / attention architecture > > So its runtime characteristics are quite different from a dense 35B model. > > That is one reason generation throughput can be surprisingly good on memory-bandwidth-oriented machines such as Apple Silicon. > > The same hybrid/recurrent architecture also means that long-session state/cache handling is more complicated than the familiar “ordinary Transformer KV cache” mental model. > > That is another reason I would pay attention to prefix reuse and long-session behavior instead of using only a simple tok/s benchmark. ## One non-performance caution Because this is an **abliterated** checkpoint, its refusal/safety behavior has intentionally been reduced; the [model card explicitly warns about this](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated). For ordinary chat that may simply be a model preference. For an agent that can: ```plaintext write files run shell commands delete/rename things access credentials ``` it matters more. I would therefore run it with the same basic precautions I would use for any autonomous local coding model: - work on a git branch/worktree - keep production credentials out of the environment - review destructive shell actions - give it only the filesystem permissions it needs - keep tests/builds isolated when practical That does not change whether the Mac is suitable; it just keeps “uncensored model” and “unrestricted machine access” as two separate decisions. ## My default recommendation If I were buying/configuring specifically for this use case, I would probably do: ```plaintext 64 GB Mac mini ↓ Q5_K_M first ↓ current llama.cpp / Metal ↓ llama-server OpenAI-compatible endpoint ↓ OpenCode ↓ 64K context initially ↓ one real repository task with ~20–30 tool turns ``` If that is stable, I would simply use it. Then optimize based on what is actually limiting you: ```plaintext Need more quality? → try Q6 Need more memory/context headroom? → try Q4 Need more Apple-Silicon throughput? → compare oMLX/MLX Tool calls break? → compare direct llama-server vs OpenCode Long sessions degrade? → inspect cache/state/context behavior Cold 128K+ prompts are too slow? → cloud GPU becomes more attractive ``` So overall: **I think your plan is technically reasonable.** The part I would validate before making strong claims is not “can 64 GB run the weights?” — that part looks fairly comfortable. It is: > **Can this exact derivative remain a reliable OpenCode agent over the length and tool complexity of your real software-development sessions?** The base Qwen3.6 evidence is encouraging, the exact checkpoint fits comfortably, the available Apple Silicon performance data looks quite usable, and the exact Q4 checkpoint can at least produce clean short-horizon tool calls. That is enough that I would proceed with the Mac idea rather than reject it because it is a “35B” model — I would just keep the model/runtime/OpenCode layers separate when diagnosing anything odd.