# Failed to load model **URL:** \ **Category:** Beginners\ **Created:** [April 19, 2026, 3:33pm UTC](https://discuss.huggingface.co/t/failed-to-load-model/175391 "2026-04-19T15:33:04Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![zuimeng001](https://avatars.discourse-cdn.com/v4/letter/z/4af34b/32.png) [@zuimeng001](https://discuss.huggingface.co/u/zuimeng001)\ **Post date:** [April 19, 2026, 3:33pm UTC](https://discuss.huggingface.co/t/failed-to-load-model/175391/1 "2026-04-19T15:33:04Z")
```auto 🥲 加载模型失败 Failed to load model. Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language_model.model.layers.0.self_attn.q_proj.weight ``` ---
**Author:** ![zuimeng001](https://avatars.discourse-cdn.com/v4/letter/z/4af34b/32.png) [@zuimeng001](https://discuss.huggingface.co/u/zuimeng001)\ **Post date:** [April 19, 2026, 3:38pm UTC](https://discuss.huggingface.co/t/failed-to-load-model/175391/2 "2026-04-19T15:38:49Z")
The mlx model I’m using keeps failing to load on my Mac. Why won’t it work? ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [April 19, 2026, 11:55pm UTC](https://discuss.huggingface.co/t/failed-to-load-model/175391/3 "2026-04-19T23:55:57Z")
If this is a Gemma 4 family device, it could be explained by the fact that Gemma 4 was released only recently and software support isn’t yet fully established; however, if that’s not the case, the issue is a different matter. In any case, the simplest solution is to update the software (if that doesn’t resolve the issue, you’ll need to look for another solution or wait): * * * For the **Gemma 4 JANG MLX model** , the most likely reason it will not load is this: **the app/runtime that is reading the model does not agree with the model file format about how one of the weight tensors is stored.** In other words, this is most likely a **compatibility problem** , not a “your Mac is too weak” problem. The strongest clue is the exact error itself, and there is already a public LM Studio bug report showing the **same shape mismatch** on the **same parameter** while loading `gemma-4-31b-jang_4m-crack` on Mac with LM Studio MLX. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) ## What the error is saying This part matters most: `Expected shape (8192, 672) but received shape (8192, 1344)` A model is a collection of tensors. Each tensor has a fixed shape. When the loader reaches `language_model.model.layers.0.self_attn.q_proj.weight`, it expects one layout, but the checkpoint contains another. Because the mismatch is **exact and structural** , the loader stops immediately. This kind of failure happens **before inference even begins** , which is why it points to a format/layout mismatch rather than a normal runtime slowdown or memory pressure issue. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) The fact that `1344` is exactly **2×** `672` is also a strong clue. That pattern usually suggests the tensor is being interpreted with the **wrong packing/layout assumption** rather than being randomly corrupted. I cannot prove the exact internal packing rule from the error alone, but the clean 2× difference strongly suggests “loader and checkpoint disagree on representation,” not “file is slightly damaged.” The recent `mlx-lm` release notes make that interpretation more plausible because they include a Gemma 4–specific fix for **quantized per-layer projection loading** , and your error is on a projection tensor (`q_proj.weight`). ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) ## Why your case is especially prone to this Your model is not just a plain stock MLX conversion. The model card identifies it as **JANG v2 (MLX-native safetensors)**, with **actual average 5.1 bits** , a **dense hybrid sliding/global attention** Gemma 4 architecture, and a recommendation to use **vMLX** for the best experience. The same page presents vMLX as the recommended path and says standard `mlx_lm` / `mlx_vlm` do not support this setup at the versions it lists. ([Hugging Face](https://huggingface.co/skyocean/Gemma-4-31B-JANG_4M-CRACK "skyocean/Gemma-4-31B-JANG\_4M-CRACK")) A related conversion page for the GGUF version explains the compatibility point even more directly: the original model uses **JANG v2 mixed-precision MLX quantization** and says standard tools such as **LM Studio, llama.cpp, oMLX, and mlx-lm** cannot load that original format because of its **mixed per-layer bit widths**. That lines up very well with your shape-mismatch error. ([Hugging Face](https://huggingface.co/douyamv/Gemma-4-31B-JANG_4M-CRACK-GGUF/blob/main/README.md "douyamv/Gemma-4-31B-JANG\_4M-CRACK-GGUF at main")) So the likely story is: 1. the runtime knows enough to recognize the model as Gemma 4, 2. it starts building the attention layer, 3. then it reaches a quantized projection tensor whose stored layout matches **JANG/vMLX expectations** , 4. but the loader is assuming a different MLX layout, 5. and load fails with the shape mismatch. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) ## Why this is probably **not** a memory problem If this were mostly a RAM or unified-memory issue, the usual symptoms would be different: out-of-memory messages, Metal allocation failures, crashing later in loading, or trouble once generation starts. Your failure happens earlier and more cleanly: the loader names a specific tensor and says its shape is wrong. That is a **schema/format mismatch** type of error. Also, a recent user writeup shows that at least some official Gemma 4 MLX models can run in LM Studio on a **32 GB Mac** after runtime updates, which further suggests that the main blocker here is not simply Mac memory. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) ## Why Gemma 4 makes this easier to trip over Gemma 4 is a newer and more specialized model family than older plain text-only local models. Google describes Gemma 4 as a four-size family built for reasoning and agentic workflows, and Gemma 4 support only landed recently in the MLX ecosystem. The `mlx-lm` release notes show that Gemma 4 support was added recently and immediately followed by Gemma 4–specific fixes, including the projection-loading fix mentioned earlier. That is the pattern you see when support is still stabilizing: some models load, some do not, and custom formats are more likely to break first. ([blog.google](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/ "Gemma 4: Our most capable open models to date")) LM Studio’s timeline shows the same thing. On **April 3, 2026** , users were still hitting `Model type gemma4 not supported` with `No module named 'mlx_vlm.models.gemma4'`. On **April 13, 2026** , there was still a public issue showing `Gemma 4 support is not ready yet`. LM Studio’s changelog then shows Gemma 4-related updates on **April 2** , **April 9** , and **April 10** —but those entries are about tool-call reliability and the updated Gemma 4 chat template, not a fix for this exact JANG tensor-layout mismatch. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1741 "[Bug] MLX version of Gemma 4 26B fails to load: "Model ...")) That context matters because it means your experience is not strange. It fits a broader pattern: **Gemma 4 support on Mac/MLX was moving quickly, and custom JANG MLX checkpoints sit near the edge of compatibility.** ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1741 "[Bug] MLX version of Gemma 4 26B fails to load: "Model ...")) ## What I think is happening in your case My best explanation is this: **The model file is probably okay, but your current loader path is not the right one for this checkpoint format.** More specifically, I think you are trying to load a **JANG v2 mixed-precision MLX checkpoint** in a runtime path that can handle **some Gemma 4 MLX models** , but not this particular weight layout. That interpretation is strongly supported by: - the exact same public bug report for the same family of model, ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) - the model card’s vMLX-first guidance, ([Hugging Face](https://huggingface.co/skyocean/Gemma-4-31B-JANG_4M-CRACK "skyocean/Gemma-4-31B-JANG\_4M-CRACK")) - the GGUF conversion page explicitly saying the original JANG format is not for standard tools, ([Hugging Face](https://huggingface.co/douyamv/Gemma-4-31B-JANG_4M-CRACK-GGUF/blob/main/README.md "douyamv/Gemma-4-31B-JANG\_4M-CRACK-GGUF at main")) - and the recent Gemma 4 projection-loading fixes in `mlx-lm`. ([GitHub](https://github.com/ml-explore/mlx-lm/releases "Releases · ml-explore/mlx-lm · GitHub")) ## What it is **less likely** to be It is **less likely** that: - your Mac is simply too weak, because the error is structural rather than resource-related, ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) - the model is randomly corrupted, because the mismatch is clean and reproducible rather than chaotic, and the same error exists publicly on another machine, ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) - or your prompt/settings are wrong, because those matter **after** loading, not during tensor-shape validation. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) A mixed or stale local snapshot is still possible, though. If your local cache combines `config.json`, shard files, or index files from different revisions, that can also create “expected A, got B” errors. It is not my top guess here, but it is worth cleaning up because it is easy to test. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) ## The safest way to fix it ### 1. If you are in LM Studio, update both the app and the runtimes In LM Studio, check **Settings → Runtime** and update **LM Studio MLX** and **Metal llama.cpp**. The changelog shows Gemma 4-related updates in early April, and a recent user writeup says updating the runtime was what allowed an official MLX Gemma 4 model to run in LM Studio. ([LM Studio](https://lmstudio.ai/changelog "LM Studio Changelog | LM Studio")) This is worth doing even though it may not fully solve the JANG model, because older LM Studio builds had explicit Gemma 4 support gaps. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1741 "[Bug] MLX version of Gemma 4 26B fails to load: "Model ...")) ### 2. Test a **standard** Gemma 4 MLX model This is the most informative next step. Try a more standard Gemma 4 MLX model, such as `mlx-community/gemma-4-26b-a4b-it-4bit`, which a recent writeup says now runs in LM Studio after runtime updates. ([Qiita](https://qiita.com/youtoy/items/4afdcd73748cc7be766b "LMStudio で MLX版の Gemma 4 が動くようになったので軽く動作確認〖ローカルLLM〗 #mlx - Qiita")) This gives you a clean diagnostic split: - **If a standard Gemma 4 MLX model loads, but your JANG model fails** , then your problem is almost certainly **checkpoint-format compatibility**. ([Qiita](https://qiita.com/youtoy/items/4afdcd73748cc7be766b "LMStudio で MLX版の Gemma 4 が動くようになったので軽く動作確認〖ローカルLLM〗 #mlx - Qiita")) - **If standard Gemma 4 MLX models also fail** , then your problem is broader: app/runtime versions, environment mismatch, or incomplete Gemma 4 support on your current stack. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1741 "[Bug] MLX version of Gemma 4 26B fails to load: "Model ...")) ### 3. For this JANG model, use the runtime it was built around: **vMLX** This is the most likely actual solution for **this** model family. The model page explicitly recommends **vMLX** , and the GGUF conversion page says the original JANG mixed-precision MLX format is only compatible with vMLX while standard tools cannot load it. That makes vMLX the natural first choice for the original JANG checkpoint. ([Hugging Face](https://huggingface.co/skyocean/Gemma-4-31B-JANG_4M-CRACK "skyocean/Gemma-4-31B-JANG\_4M-CRACK")) ### 4. If you want to stay in LM Studio, use the **GGUF** conversion instead of the original JANG MLX checkpoint The GGUF conversion exists specifically because the original JANG MLX format is not broadly compatible. The conversion page says it provides standard GGUF quantizations for use with **llama.cpp, LM Studio, Ollama, and other GGUF-compatible engines**. So if LM Studio is your preferred app, the GGUF path is likely the smoother path than trying to force the original JANG MLX checkpoint to work there. ([Hugging Face](https://huggingface.co/douyamv/Gemma-4-31B-JANG_4M-CRACK-GGUF/blob/main/README.md "douyamv/Gemma-4-31B-JANG\_4M-CRACK-GGUF at main")) ### 5. Delete the local model folder and re-download it cleanly This is a good hygiene step. If your local copy is stale or mixed, redownloading fixes that. It may not be the root cause, but it is easy to rule out and worth doing before deeper debugging. The model repo has had recent updates, including README and capability metadata changes, so a clean snapshot is safer than relying on an older local cache. ([Hugging Face](https://huggingface.co/dealignai/Gemma-4-31B-JANG_4M-CRACK/tree/main "dealignai/Gemma-4-31B-JANG\_4M-CRACK at main")) ### 6. If you are loading from Python, update MLX packages If you are not in LM Studio and instead use Python directly, make sure `mlx-lm` is recent enough to include **Gemma 4 support** and the **Gemma 4 quantized per-layer projection loading** fix. Those fixes are in the release notes, so older installations are a real risk. ([GitHub](https://github.com/ml-explore/mlx-lm/releases "Releases · ml-explore/mlx-lm · GitHub")) ## A simple decision tree Here is the beginner-safe version: **Case A: official MLX Gemma 4 works, JANG MLX fails** That means your app can handle Gemma 4 in general, but **not this custom checkpoint format**. Use **vMLX** for the JANG model, or use the **GGUF** version in LM Studio. ([Qiita](https://qiita.com/youtoy/items/4afdcd73748cc7be766b "LMStudio で MLX版の Gemma 4 が動くようになったので軽く動作確認〖ローカルLLM〗 #mlx - Qiita")) **Case B: official MLX Gemma 4 also fails** That means your Gemma 4 support is still not correct at the runtime/app level. Update LM Studio + runtimes, or update your MLX Python packages. ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1741 "[Bug] MLX version of Gemma 4 26B fails to load: "Model ...")) **Case C: everything still fails after updating** Then do a clean re-download of the model files and retest. If the JANG model still fails but official models load, the answer is still “wrong runtime for this checkpoint.” ([Hugging Face](https://huggingface.co/dealignai/Gemma-4-31B-JANG_4M-CRACK/tree/main "dealignai/Gemma-4-31B-JANG\_4M-CRACK at main")) ## My bottom line The clearest explanation is: **Your MLX model will not load because the checkpoint format and the loader are mismatched, and this is especially likely because you are using a JANG v2 mixed-precision Gemma 4 checkpoint that is meant for vMLX rather than a standard MLX loader path.** ([GitHub](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1800 "Error when loading model: ValueError: Expected shape (8192, 672) but received shape (8192, 1344) for parameter language\_model.model.layers.0.self\_attn.q\_proj.weight · Issue #1800 · lmstudio-ai/lmstudio-bug-tracker · GitHub")) So the practical fix order is: 1. update LM Studio and runtimes if you use LM Studio, ([LM Studio](https://lmstudio.ai/changelog "LM Studio Changelog | LM Studio")) 2. test a standard `mlx-community` Gemma 4 MLX model, ([Qiita](https://qiita.com/youtoy/items/4afdcd73748cc7be766b "LMStudio で MLX版の Gemma 4 が動くようになったので軽く動作確認〖ローカルLLM〗 #mlx - Qiita")) 3. use **vMLX** for the original JANG MLX checkpoint, ([Hugging Face](https://huggingface.co/skyocean/Gemma-4-31B-JANG_4M-CRACK "skyocean/Gemma-4-31B-JANG\_4M-CRACK")) 4. or use the **GGUF** conversion if you want LM Studio compatibility, ([Hugging Face](https://huggingface.co/douyamv/Gemma-4-31B-JANG_4M-CRACK-GGUF/blob/main/README.md "douyamv/Gemma-4-31B-JANG\_4M-CRACK-GGUF at main")) 5. and re-download the model cleanly to rule out cache issues. ([Hugging Face](https://huggingface.co/dealignai/Gemma-4-31B-JANG_4M-CRACK/tree/main "dealignai/Gemma-4-31B-JANG\_4M-CRACK at main"))