# Inference Endpoint Initializing forever!

**URL:** <https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675>\
**Category:** Inference Endpoints on the Hub\
**Created:** [November 20, 2025, 3:37pm UTC](https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675 "2025-11-20T15:37:27Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [November 20, 2025, 5:31pm UTC](https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675/2 "2025-11-20T17:31:25Z")

</div>

Without knowing the specifics of your inference endpoint setup, I can only offer general advice…  
The most common issue is probably related to `torch.compile`.

* * *

Below is a structured, “from first principles” view of:

- What a **dedicated Hugging Face Inference Endpoint** is doing while it says `Initializing`
- The **main causes** of “Initializing forever”
- **Concrete solutions / tests** for each cause
- A short **decision checklist** at the end

I will repeat key ideas in different ways so the mental model is very clear.

* * *

## 1. Background: what “Initializing” means for a dedicated endpoint

A **dedicated endpoint** is basically:

- A reserved VM + GPU(s) + a load balancer + a container running your model.

When the status is `Initializing`, roughly this is happening:

1. **Scheduling / capacity**

2. **Container startup**

3. **Your code runs**

4. **Health checks**

Typical dedicated endpoint expectations:

- Docs and forum discussions imply a “normal” cold start is a **few minutes** (image pull + model load + light warmup). Staying in `Initializing` for tens of minutes or indefinitely is not normal. ([Hugging Face Forums](https://discuss.huggingface.co/t/stuck-inference-endpoint/65734 "Stuck Inference endpoint"))

So “Initializing forever” always means:

> The platform never sees your container as healthy, or it cannot schedule the container at all.

* * *

## 2. Cause 1 – Hardware capacity / scheduling issues (platform-side)

### Context / background

HF themselves have several threads where users suddenly cannot start an endpoint that was working before. The UI shows errors like:

- `Endpoint failed to start. Scheduling failure: not enough hardware capacity` ([Hugging Face Forums](https://discuss.huggingface.co/t/unable-to-start-inference-endpoint-not-enough-hardware-capacity/51805 "Unable to start inference endpoint: not enough hardware ..."))
- Or similar “unable to schedule” messages. ([Hugging Face Forums](https://discuss.huggingface.co/t/scheduling-failure-unable-to-schedule/160642 "Scheduling failure: unable to schedule"))

HF staff replies often say things like:

- “We had a minor issue in eu-west-1. This should be fixed.” ([Hugging Face Forums](https://discuss.huggingface.co/t/unable-to-start-inference-endpoint-not-enough-hardware-capacity/51805/2 "Unable to start inference endpoint: not enough hardware ..."))
- Or “This error is related to GPU availability; change instance or region if possible.” ([Hugging Face Forums](https://discuss.huggingface.co/t/scheduling-failure-unable-to-schedule/160642 "Scheduling failure: unable to schedule"))

This is a **pure infra problem** : the requested GPU node type is not available or a quota/cluster issue blocks scheduling.

### How it looks when you’re debugging

- The endpoint sits in `Initializing` or quickly flips to `Failed` with a scheduling error.
- A **tiny test endpoint** in the same region / instance type can also fail to start.
- The logs may not even show your code; the failure happens before your container fully starts.

### What you can do

1. **Test with a very small “known good” endpoint in the same region + instance type**

2. **Try a different GPU type or region**

3. **Contact HF support with concrete details**

If this cause is confirmed, **your code changes will not fix it** ; it must be resolved by Hugging Face or by changing the hardware/region.

* * *

## 3. Cause 2 – Load balancer / routing issues (platform-side)

### Context / background

Some HF forum topics show endpoints where:

- The message is:

- Other related posts group this together with “Inference Endpoints Issues” and “Dedicated endpoint stuck at Initializing.” ([Hugging Face Forums](https://discuss.huggingface.co/t/inference-endpoints-issues/90243 "Inference Endpoints Issues"))

This means the **front-end routing / load balancer** didn’t become ready, even if the backend container might be running.

### How it looks

- The status remains `Initializing` or “Load balancer not ready yet” for a long time.
- Logs for your container may look normal, or you might not see any direct error.
- Other users sometimes report similar symptoms around the same time, which hints at an incident.

### What to do

- Same strategy as capacity:

- If the problem appears across multiple endpoints, send HF support:

Again, this is **not fixed by editing your handler.py**.

* * *

## 4. Cause 3 – Health-check / port mapping issues (very common for dedicated + custom images)

This is one of the most common causes of “Initializing forever” when you use:

- **Custom Docker images** , or
- A Toolkit handler that starts its **own HTTP server**.

### Background from HF docs and forum

The official FAQ has an exact question:

> “I can see from the logs that my endpoint is running but the status is stuck at ‘initializing’.”

Answer:

> “This usually means that the port mapping is incorrect. Ensure your app is listening on port 80 and that the Docker container is exposing port 80 externally.” ([Hugging Face](https://huggingface.co/docs/inference-endpoints/support/faq "FAQs"))

A concrete forum case:

- A user deployed a ComfyUI-based custom image.
- Inside the container, ComfyUI ran on `127.0.0.1:8188`, and logs looked fine.
- The endpoint stayed in `initializing` until timeout, with no obvious error. ([Hugging Face Forums](https://discuss.huggingface.co/t/always-initializing-until-time-out-without-any-error-log/104006 "Always 【initializing】 until time out without any error log"))
- The cause fits the FAQ: the app listens on localhost/8188 rather than the **expected external port**.

### Why this breaks dedicated endpoints

The HF control plane:

- Provisions the VM
- Starts your container
- Probes **a specific port** from outside the container

If your app:

- Listens only on `127.0.0.1` instead of `0.0.0.0`, or
- Listens on port `8188` while HF probes port `80`,

then the health check never succeeds. HF sees “container up, but not healthy → keep initializing”.

### Symptoms

- Logs show your server starting (“Running on [http://0.0.0.0:8188”](http://0.0.0.0:8188%E2%80%9D), “ComfyUI started”, etc.).
- UI still shows `Initializing`.
- No clear Python exceptions, no OOM; just silence after the “server started” line.

### Solutions

1. **For custom Docker images**

2. **For Toolkit / custom handler endpoints**

3. **Add structured logging**

This cause is **very specific** : logs look “OK”, but the status never leaves `Initializing`. The FAQ line about port mapping is exactly about this situation. ([Hugging Face](https://huggingface.co/docs/inference-endpoints/support/faq "FAQs"))

* * *

## 5. Cause 4 – Crash loop while starting (OOM, import errors, misconfig)

### Background

Many HF forum topics have messages like:

- `Server message: Endpoint failed to start. Endpoint failed` ([Hugging Face Forums](https://discuss.huggingface.co/t/server-message-endpoint-failed-to-start/80578 "Server message:Endpoint failed to start"))
- `Inference Endpoint Fails to Start` and similar. ([Hugging Face Forums](https://discuss.huggingface.co/t/server-message-endpoint-failed-to-start-endpoint-failed/69876 "Server message:Endpoint failed to start. Endpoint failed"))

Typical reasons:

- The container starts, runs your code, **crashes** , and is restarted.
- If this happens repeatedly, you see an endpoint stuck in `Initializing` or quickly flipping between `Initializing` and `Failed`.

### Common crash causes

1. **Out-of-memory during model load**

2. **Missing / incompatible dependencies**

3. **Misconfigured inference engines**

### How to identify this cause

- In logs, you see **repeated stack traces** or the same error message over and over.
- The endpoint never reaches a stable `running` state.
- Sometimes the UI shows “Endpoint failed to start” instead of just `Initializing`. ([Hugging Face Forums](https://discuss.huggingface.co/t/server-message-endpoint-failed-to-start/80578 "Server message:Endpoint failed to start"))

### Solutions

1. **Read logs carefully**

2. **Test the exact same image / code locally**

3. **Reduce memory usage**

4. **Fix library versions**

Once you stop the crash loop, the endpoint should move from `Initializing` to `Running` relatively quickly.

* * *

## 6. Cause 5 – Startup too slow: big model, downloads, and `torch.compile`

This is where `torch.compile` becomes important.

### Background: what `torch.compile` is doing

- `torch.compile` is a **just-in-time compiler** that:

- Official PyTorch docs say:

- A full example tutorial reports total script time of several minutes when using `torch.compile` on a non-trivial model. ([PyTorch Docs](https://docs.pytorch.org/tutorials/intermediate/torch_compile_full_example.html "torch.compile End-to-End Tutorial"))

So for large LLMs/diffusion models, compile is **not cheap**. It is an extra heavy step on top of model loading.

### Why this breaks a dedicated endpoint

HF gives your container a limited window to:

- Start
- Load the model
- Answer health checks

If in ` __init__ ` you do:

```python
self.model = AutoModelForCausalLM.from_pretrained(path, ...)
self.model = torch.compile(self.model, mode="default" or "max-autotune")
# maybe with a warmup loop

```

then:

- HF is waiting for health checks during **download + load + compile + warmup**.
- If compile takes too long (which is realistic for large models), health checks may time out. HF kills and restarts the pod → you see `Initializing` forever.
- If compile crashes (unsupported op, driver issue), you get a crash loop similar to Cause 4.

This is also a known pain point in broader inference platforms: cold-start times become dominated by `torch.compile` and the first JIT run. ([PyTorch Docs](https://docs.pytorch.org/tutorials/recipes/regional_compilation.html "Reducing torch.compile cold start compilation time with ..."))

### Safer ways to use `torch.compile` on a dedicated endpoint

1. **Feature-flag compile (on/off via env variable)**

2. **Move compile from ` __init__ ` to first real request**

3. **Use appropriate mode (`reduce-overhead` for small batches)**

4. **Compile only the hot part of the model**

5. **Minimize warmup during startup**

6. **Accept that compile may not be worth it for some endpoints**

If you rearrange startup so that heavy compile and warmup are not part of the health-check window, you greatly reduce the chance of “Initializing forever”.

* * *

## 7. Putting it together: practical diagnosis flow

Here is a concise decision tree you can apply to your dedicated endpoint.

### Step 1 – Quick baseline

- Create a **small, vanilla** dedicated endpoint in the same region + GPU type.

- If that also fails or gets stuck:

### Step 2 – Examine logs

- If logs are empty or the log panel 5xx’s:

- If logs show your server running (“listening on 127.0.0.1:8188”, “server started”) but the status is still `Initializing`:

- If logs show repeated Python exceptions or OOMs:

- If logs show model loading messages and then nothing for a long time:

### Step 3 – Instrument phases (optional but very helpful)

Add explicit phase logs in ` __init__ `:

- `PHASE_START init`
- `PHASE_START load_model`
- `PHASE_START maybe_compile`
- `PHASE_END ...`

Then:

- If logs stop before `PHASE_START maybe_compile`, compile is probably not the problem; earlier steps are.
- If logs stop at `PHASE_START maybe_compile`, or you never see `PHASE_END maybe_compile`, that strongly implicates `torch.compile`.

### Step 4 – Toggle `torch.compile` and heavy work

- Disable compile and extra warmup (via env flags) and redeploy:

- If disabling compile doesn’t help, focus back on capacity, port mapping, and crash loops.

* * *

## 8. Summary: main causes and matching solutions

To make it extremely explicit:

1. **Capacity / scheduling issues (HF infra)**

2. **Load balancer / routing issues (HF infra)**

3. **Health-check / port mapping problems (custom images / servers)**

4. **Crash loop during startup (OOM, dependency, config errors)**

5. **Startup too slow due to heavy work and `torch.compile`**

---

_[View the full topic](https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675)._