# 500 Internal Server Error with Ollama

**URL:** <https://discuss.huggingface.co/t/500-internal-server-error-with-ollama/174735>\
**Category:** Beginners\
**Created:** [March 28, 2026, 1:44pm UTC](https://discuss.huggingface.co/t/500-internal-server-error-with-ollama/174735 "2026-03-28T13:44:20Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![thebski](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thebski/32/58987_2.png) [@thebski](https://discuss.huggingface.co/u/thebski)\
**Post date:** [March 28, 2026, 1:44pm UTC](https://discuss.huggingface.co/t/500-internal-server-error-with-ollama/174735/1 "2026-03-28T13:44:20Z")

</div>

Hey guys!

I’m new here to HF and trying to utilize local LLMs in general. I have been playing around with Claude Code using Ollama. It’s been working fine but I just installed a second GPU so am wanting to play with more models. I’m new to using HF to download. I’m trying to run the following model in Ollama and ultimately Claude Code:

> **[Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF · Hugging Face](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

I used the command provided by the ‘use this model’ drop down to run Ollama and pull the manifest.

`ollama run ``hf.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q8_0`

It downloaded fine, and I can confirm the file is in the directory and named as the model is trying to call. However, I’m still getting this error when I try to launch Ollama with it.

`Error loading model: 500 Internal Server Error: unable to load model: /usr/share/ollama/.ollama/models/blobs/sha256-9093fa002d4e64576b0724bd67010de8f87d5ec284fa71d8394fc47e2d9c65be`

I’m on Ubuntu 24.04 and have two RTX 4090s so it shouldn’t be a size problem. Does anyone have any ideas on what to look for?

Thank you!

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [March 29, 2026, 12:36am UTC](https://discuss.huggingface.co/t/500-internal-server-error-with-ollama/174735/2 "2026-03-29T00:36:05Z")

</div>

Maybe an Ollama + Qwen 3.5 series specific issue?

* * *

What I would look at first is **model compatibility, not disk contents and not raw VRAM**. Your symptom pattern is: the Hugging Face repo resolves, the GGUF blob downloads, Ollama stores it under a SHA256 blob path, and then the model fails during the **load/init** phase with a local `500 Internal Server Error`. Recent Ollama issues show the same pattern for Hugging Face Qwen3.5 GGUFs, including reports on 0.17.5 and 0.17.6. ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))

## What the error means

That blob path error does **not** mean “the file is missing.” It means Ollama found the local blob and then failed to initialize it. In other words:

- download worked
- manifest write worked
- model open/decode/load failed

That distinction matters because it points away from Hugging Face transport problems and toward **runtime incompatibility inside Ollama**. The live Qwen3.5 issue thread in Ollama says HF-downloaded `qwen35/qwen35moe` models can fail because their metadata layout differs from Ollama’s packaged variants, which causes decode failure and then fallback failure. ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))

## Why your exact repo is a higher-risk case

Your target repo is not a simple text-only single-file package. The Hugging Face file tree labels it `Image-Text-to-Text`, includes `mmproj-BF16.gguf`, and ships separate quantized GGUF files like `Q4_K_M`, `Q5_K_M`, `Q6_K`, and `Q8_0`. The `Q8_0` file is listed as 28.6 GB and the projector file is about 931 MB. That means you are dealing with a **multimodal-flavored package** , not just “one plain text GGUF.” ([Hugging Face](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF/tree/main "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF at main"))

That matters because current Ollama issue reports around HF Qwen3.5 are not just about “large model too big.” They are about **Qwen3.5 architecture handling** and, in neighboring reports, split text/vision style packaging that does not cleanly load through the current HF import path. ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))

## Why I do not think your main problem is “two 4090s are not enough”

Ollama’s FAQ says that when a model fits on one GPU, it prefers one GPU. If it does not fit on one GPU, it can spread the model across all available GPUs. So dual 4090s are a valid setup for larger models, and your hardware does not immediately point to “this should hard-fail before starting.” ([Ollama](https://docs.ollama.com/faq "FAQ"))

Also, your failure is happening at the **load step** , not after a long prompt or a giant context window. That makes a metadata or architecture problem more likely than ordinary context-memory pressure. Context settings still matter later, but they are probably not the first thing breaking here. Ollama documents that parallelism and context length scale memory use, and that Flash Attention plus KV-cache quantization are later levers for memory reduction. ([Ollama](https://docs.ollama.com/faq "FAQ"))

## The strongest technical clue

The most important current Ollama issue for your case is the one stating that HF `qwen35/qwen35moe` models can have a different `attention.head_count_kv` representation from the Ollama library models. The issue says that causes `NewTextProcessor()` to fail during decode, and then the fallback path fails because support there was not merged yet. That is a very direct explanation for “download succeeds, then load fails.” ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))

There is also a March 2026 duplicate report showing actual loader output that ends with:

`error loading model architecture: unknown model architecture: 'qwen35'`

on Ollama 0.17.6, again from a Hugging Face Qwen3.5 GGUF run. That lines up closely with your symptom family. ([GitHub](https://github.com/ollama/ollama/issues/14631 "500 Internal Server Error when loading hf.co/HauhauCS/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive:Q4\_K\_M via ollama run · Issue #14631 · ollama/ollama · GitHub"))

## One subtle background point

Ollama’s official import docs say GGUF import is supported and show both local GGUF import and adapter workflows, but the architecture list on that page names Llama, Mistral, Gemma, and Phi3. At the same time, Ollama’s own model library now has an official `qwen3.5:27b` page whose metadata shows `arch qwen35`. That combination suggests something important: **Ollama’s own packaged qwen3.5 models may work before arbitrary HF-imported qwen3.5 GGUFs do**. In other words, “Ollama supports qwen3.5” and “Ollama supports every HF qwen3.5 GGUF repo through `hf.co/...`” are not the same statement. ([Ollama](https://docs.ollama.com/import "Importing a Model - Ollama"))

## What I think is happening in your case

My ranking would be:

1. **Most likely:** current Ollama incompatibility with this HF Qwen3.5 GGUF import path. ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))
2. **Also likely:** the repo’s multimodal `mmproj` packaging makes the load path more fragile. ([Hugging Face](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF/tree/main "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF at main"))
3. **Less likely:** a true VRAM-size problem. ([Ollama](https://docs.ollama.com/faq "FAQ"))
4. **Much less likely:** bad filename or failed download, because the loader got far enough to attempt model initialization from the local blob. ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))

## What to check next

### 1. Look at the actual Ollama server log

On Ubuntu with systemd, Ollama’s troubleshooting page says to use:

```bash
journalctl -u ollama --no-pager --follow --pager-end

```

That is the single highest-value next step, because it will tell you whether the internal failure is `unknown model architecture: 'qwen35'`, a decode error, or a GPU-init problem. ([Ollama](https://docs.ollama.com/troubleshooting "Troubleshooting - Ollama"))

### 2. Turn on debug logging

Ollama’s Linux docs say you can add this systemd override:

```ini
[Service]
Environment="OLLAMA_DEBUG=1"

```

Then restart the service and re-run the model. That will give you better loader diagnostics than the CLI’s short `500` message. ([Ollama](https://docs.ollama.com/linux "Linux - Ollama"))

### 3. Sanity-check Ollama with the official packaged Qwen3.5

Try:

```bash
ollama run qwen3.5:27b

```

Ollama’s official `qwen3.5:27b` page shows it is a packaged `qwen35` model, updated recently, with a 17 GB Q4\_K\_M artifact. If this works on your machine, then your CUDA path, service, and multi-GPU environment are probably fine, and the problem is specifically the Hugging Face imported model packaging. ([Ollama](https://ollama.com/library/qwen3.5%3A27b "qwen3.5:27b"))

### 4. Do not use `Q8_0` as your first test

Your target repo offers `Q4_K_M` at 16.5 GB, `Q5_K_M` at 19.2 GB, `Q6_K` at 22.1 GB, and `Q8_0` at 28.6 GB. Even if size is not the root cause, starting with `Q4_K_M` or `Q5_K_M` removes one extra variable. If the smaller quant fails with the same architecture-style error, that is more evidence that the core problem is compatibility, not memory pressure. ([Hugging Face](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF/tree/main "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF at main"))

### 5. Separate “HF import path” from “GGUF loader path”

Ollama’s import docs say you can import a local GGUF via a `Modelfile` using:

```plaintext
FROM /path/to/file.gguf

```

and then build it with `ollama create my-model`. I do **not** expect this to magically fix a true `qwen35` loader incompatibility, because the same runtime still has to parse the model, but it is a useful diagnostic because it removes the HF fetch layer from the equation. ([Ollama](https://docs.ollama.com/import "Importing a Model - Ollama"))

## What not to chase first

I would **not** start by deleting and redownloading the model repeatedly. The evidence points away from a bad transfer and toward a load-time compatibility problem. ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))

I would also **not** spend much time on the broken BF16 multipart discussion in that repo unless you specifically need BF16. Your current attempt is `Q8_0`, which is a different file. There is a repo discussion showing BF16 part 2 problems, but that is a separate issue and probably not your immediate blocker. ([Hugging Face](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF/discussions/4 "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF · BF16 part 2 of 2?"))

## My bottom line

Your setup is probably fine. Your command syntax was accepted, the download worked, and the file landed where Ollama stores model blobs. The most likely problem is that **this particular Hugging Face Qwen3.5 GGUF package is hitting a current Ollama loader/runtime gap** , and the repo’s multimodal-style packaging increases the odds of that. ([GitHub](https://github.com/ollama/ollama/issues/14503 "qwen35/qwen35moe models downloaded from HuggingFace are unsupported. · Issue #14503 · ollama/ollama · GitHub"))

So the best next move is:

- inspect `journalctl` logs
- enable `OLLAMA_DEBUG=1`
- test `ollama run qwen3.5:27b`
- retry with `Q4_K_M` or `Q5_K_M`
- if you need this exact Jackrong model and Ollama still fails, move it to a backend with stronger direct GGUF support

---

<div class="post-metadata">

**Author:** ![PhilippLinux](https://avatars.discourse-cdn.com/v4/letter/p/e68b1a/32.png) [@PhilippLinux](https://discuss.huggingface.co/u/PhilippLinux)\
**Post date:** [April 6, 2026, 4:19am UTC](https://discuss.huggingface.co/t/500-internal-server-error-with-ollama/174735/3 "2026-04-06T04:19:07Z")

</div>

Had the same issue, 3090.

> ollama run [Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF · Hugging Face](http://hf.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF) “4+18”  
> Error: 500 Internal Server Error: unable to load model:  
> /home/user/LLMs/blobs/sha256-a15622267316636d22575f8ab25a61347eab59def966f257f572914812348c61

## **What fixed it for me, was ‘import’ it from the download:**

! ollama rm [Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF · Hugging Face](http://hf.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF)

> ! ls ~/Downloads/\*.gguf
> 
> Then create a Modelfile and import it:
> 
> ! echo “FROM /home/philipp/Downloads/your-model-name.gguf” \> /tmp/Modelfile  
> ! ollama create qwen3.5-27b-claude --file /tmp/Modelfile
> 
> Replace your-model-name.gguf with the actual filename and qwen3.5-27b-claude with whatever name you want to give it in Ollama.

im a noob. not sure whats up. but i guess the checksum got messed up somewhere so building it manually eliminates this. tried pulling and rming the desired model and some offshoots as well as removed any other gwen 3.5 related models without success.

---

<div class="post-metadata">

**Author:** ![Puele](https://avatars.discourse-cdn.com/v4/letter/p/8dc957/32.png) [@Puele](https://discuss.huggingface.co/u/Puele)\
**Post date:** [April 11, 2026, 10:46am UTC](https://discuss.huggingface.co/t/500-internal-server-error-with-ollama/174735/4 "2026-04-11T10:46:46Z")

</div>

Same issue here - it doens’t really matter which model or which quantisation:

```auto
D:\dev\ollama-benchmark>ollama run hf.co/Jackrong/Qwopus3.5-9B-v3-GGUF:Q4_K_Mpulling manifestpulling 19d52ddc3343: 100% 
sha256 digestwriting manifest success

Error: 500 Internal Server Error: unable to load model: C:\Users\Paul.ollama\models\blobs\sha256-19d52ddc3343880a3999451c159c3564ba83e96ec24659c0041a83fe4e9c37ec

```

---

<div class="post-metadata">

**Author:** ![Puele](https://avatars.discourse-cdn.com/v4/letter/p/8dc957/32.png) [@Puele](https://discuss.huggingface.co/u/Puele)\
**Post date:** [April 11, 2026, 10:59am UTC](https://discuss.huggingface.co/t/500-internal-server-error-with-ollama/174735/5 "2026-04-11T10:59:06Z")

</div>

Yes this imports the model and it can be executed - but it seems broken - I tested it with [Jackrong/Qwopus3.5-9B-v3-GGUF at main](https://huggingface.co/Jackrong/Qwopus3.5-9B-v3-GGUF/tree/main)