# Inference Endpoint Initializing forever!

**URL:** <https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675>\
**Category:** Inference Endpoints on the Hub\
**Created:** [November 20, 2025, 3:37pm UTC](https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675 "2025-11-20T15:37:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![omarzouk55](https://avatars.discourse-cdn.com/v4/letter/o/e99b99/32.png) [@omarzouk55](https://discuss.huggingface.co/u/omarzouk55)\
**Post date:** [November 20, 2025, 3:37pm UTC](https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675/1 "2025-11-20T15:37:27Z")

</div>

Inference Endpoint Initializing forever !!!

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [November 20, 2025, 5:31pm UTC](https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675/2 "2025-11-20T17:31:25Z")

</div>

Without knowing the specifics of your inference endpoint setup, I can only offer general advice…  
The most common issue is probably related to `torch.compile`.

* * *

Below is a structured, “from first principles” view of:

- What a **dedicated Hugging Face Inference Endpoint** is doing while it says `Initializing`
- The **main causes** of “Initializing forever”
- **Concrete solutions / tests** for each cause
- A short **decision checklist** at the end

I will repeat key ideas in different ways so the mental model is very clear.

* * *

## 1. Background: what “Initializing” means for a dedicated endpoint

A **dedicated endpoint** is basically:

- A reserved VM + GPU(s) + a load balancer + a container running your model.

When the status is `Initializing`, roughly this is happening:

1. **Scheduling / capacity**

2. **Container startup**

3. **Your code runs**

4. **Health checks**

Typical dedicated endpoint expectations:

- Docs and forum discussions imply a “normal” cold start is a **few minutes** (image pull + model load + light warmup). Staying in `Initializing` for tens of minutes or indefinitely is not normal. ([Hugging Face Forums](https://discuss.huggingface.co/t/stuck-inference-endpoint/65734 "Stuck Inference endpoint"))

So “Initializing forever” always means:

> The platform never sees your container as healthy, or it cannot schedule the container at all.

* * *

## 2. Cause 1 – Hardware capacity / scheduling issues (platform-side)

### Context / background

HF themselves have several threads where users suddenly cannot start an endpoint that was working before. The UI shows errors like:

- `Endpoint failed to start. Scheduling failure: not enough hardware capacity` ([Hugging Face Forums](https://discuss.huggingface.co/t/unable-to-start-inference-endpoint-not-enough-hardware-capacity/51805 "Unable to start inference endpoint: not enough hardware ..."))
- Or similar “unable to schedule” messages. ([Hugging Face Forums](https://discuss.huggingface.co/t/scheduling-failure-unable-to-schedule/160642 "Scheduling failure: unable to schedule"))

HF staff replies often say things like:

- “We had a minor issue in eu-west-1. This should be fixed.” ([Hugging Face Forums](https://discuss.huggingface.co/t/unable-to-start-inference-endpoint-not-enough-hardware-capacity/51805/2 "Unable to start inference endpoint: not enough hardware ..."))
- Or “This error is related to GPU availability; change instance or region if possible.” ([Hugging Face Forums](https://discuss.huggingface.co/t/scheduling-failure-unable-to-schedule/160642 "Scheduling failure: unable to schedule"))

This is a **pure infra problem** : the requested GPU node type is not available or a quota/cluster issue blocks scheduling.

### How it looks when you’re debugging

- The endpoint sits in `Initializing` or quickly flips to `Failed` with a scheduling error.
- A **tiny test endpoint** in the same region / instance type can also fail to start.
- The logs may not even show your code; the failure happens before your container fully starts.

### What you can do

1. **Test with a very small “known good” endpoint in the same region + instance type**

2. **Try a different GPU type or region**

3. **Contact HF support with concrete details**

If this cause is confirmed, **your code changes will not fix it** ; it must be resolved by Hugging Face or by changing the hardware/region.

* * *

## 3. Cause 2 – Load balancer / routing issues (platform-side)

### Context / background

Some HF forum topics show endpoints where:

- The message is:

- Other related posts group this together with “Inference Endpoints Issues” and “Dedicated endpoint stuck at Initializing.” ([Hugging Face Forums](https://discuss.huggingface.co/t/inference-endpoints-issues/90243 "Inference Endpoints Issues"))

This means the **front-end routing / load balancer** didn’t become ready, even if the backend container might be running.

### How it looks

- The status remains `Initializing` or “Load balancer not ready yet” for a long time.
- Logs for your container may look normal, or you might not see any direct error.
- Other users sometimes report similar symptoms around the same time, which hints at an incident.

### What to do

- Same strategy as capacity:

- If the problem appears across multiple endpoints, send HF support:

Again, this is **not fixed by editing your handler.py**.

* * *

## 4. Cause 3 – Health-check / port mapping issues (very common for dedicated + custom images)

This is one of the most common causes of “Initializing forever” when you use:

- **Custom Docker images** , or
- A Toolkit handler that starts its **own HTTP server**.

### Background from HF docs and forum

The official FAQ has an exact question:

> “I can see from the logs that my endpoint is running but the status is stuck at ‘initializing’.”

Answer:

> “This usually means that the port mapping is incorrect. Ensure your app is listening on port 80 and that the Docker container is exposing port 80 externally.” ([Hugging Face](https://huggingface.co/docs/inference-endpoints/support/faq "FAQs"))

A concrete forum case:

- A user deployed a ComfyUI-based custom image.
- Inside the container, ComfyUI ran on `127.0.0.1:8188`, and logs looked fine.
- The endpoint stayed in `initializing` until timeout, with no obvious error. ([Hugging Face Forums](https://discuss.huggingface.co/t/always-initializing-until-time-out-without-any-error-log/104006 "Always 【initializing】 until time out without any error log"))
- The cause fits the FAQ: the app listens on localhost/8188 rather than the **expected external port**.

### Why this breaks dedicated endpoints

The HF control plane:

- Provisions the VM
- Starts your container
- Probes **a specific port** from outside the container

If your app:

- Listens only on `127.0.0.1` instead of `0.0.0.0`, or
- Listens on port `8188` while HF probes port `80`,

then the health check never succeeds. HF sees “container up, but not healthy → keep initializing”.

### Symptoms

- Logs show your server starting (“Running on [http://0.0.0.0:8188”](http://0.0.0.0:8188%E2%80%9D), “ComfyUI started”, etc.).
- UI still shows `Initializing`.
- No clear Python exceptions, no OOM; just silence after the “server started” line.

### Solutions

1. **For custom Docker images**

2. **For Toolkit / custom handler endpoints**

3. **Add structured logging**

This cause is **very specific** : logs look “OK”, but the status never leaves `Initializing`. The FAQ line about port mapping is exactly about this situation. ([Hugging Face](https://huggingface.co/docs/inference-endpoints/support/faq "FAQs"))

* * *

## 5. Cause 4 – Crash loop while starting (OOM, import errors, misconfig)

### Background

Many HF forum topics have messages like:

- `Server message: Endpoint failed to start. Endpoint failed` ([Hugging Face Forums](https://discuss.huggingface.co/t/server-message-endpoint-failed-to-start/80578 "Server message:Endpoint failed to start"))
- `Inference Endpoint Fails to Start` and similar. ([Hugging Face Forums](https://discuss.huggingface.co/t/server-message-endpoint-failed-to-start-endpoint-failed/69876 "Server message:Endpoint failed to start. Endpoint failed"))

Typical reasons:

- The container starts, runs your code, **crashes** , and is restarted.
- If this happens repeatedly, you see an endpoint stuck in `Initializing` or quickly flipping between `Initializing` and `Failed`.

### Common crash causes

1. **Out-of-memory during model load**

2. **Missing / incompatible dependencies**

3. **Misconfigured inference engines**

### How to identify this cause

- In logs, you see **repeated stack traces** or the same error message over and over.
- The endpoint never reaches a stable `running` state.
- Sometimes the UI shows “Endpoint failed to start” instead of just `Initializing`. ([Hugging Face Forums](https://discuss.huggingface.co/t/server-message-endpoint-failed-to-start/80578 "Server message:Endpoint failed to start"))

### Solutions

1. **Read logs carefully**

2. **Test the exact same image / code locally**

3. **Reduce memory usage**

4. **Fix library versions**

Once you stop the crash loop, the endpoint should move from `Initializing` to `Running` relatively quickly.

* * *

## 6. Cause 5 – Startup too slow: big model, downloads, and `torch.compile`

This is where `torch.compile` becomes important.

### Background: what `torch.compile` is doing

- `torch.compile` is a **just-in-time compiler** that:

- Official PyTorch docs say:

- A full example tutorial reports total script time of several minutes when using `torch.compile` on a non-trivial model. ([PyTorch Docs](https://docs.pytorch.org/tutorials/intermediate/torch_compile_full_example.html "torch.compile End-to-End Tutorial"))

So for large LLMs/diffusion models, compile is **not cheap**. It is an extra heavy step on top of model loading.

### Why this breaks a dedicated endpoint

HF gives your container a limited window to:

- Start
- Load the model
- Answer health checks

If in ` __init__ ` you do:

```python
self.model = AutoModelForCausalLM.from_pretrained(path, ...)
self.model = torch.compile(self.model, mode="default" or "max-autotune")
# maybe with a warmup loop

```

then:

- HF is waiting for health checks during **download + load + compile + warmup**.
- If compile takes too long (which is realistic for large models), health checks may time out. HF kills and restarts the pod → you see `Initializing` forever.
- If compile crashes (unsupported op, driver issue), you get a crash loop similar to Cause 4.

This is also a known pain point in broader inference platforms: cold-start times become dominated by `torch.compile` and the first JIT run. ([PyTorch Docs](https://docs.pytorch.org/tutorials/recipes/regional_compilation.html "Reducing torch.compile cold start compilation time with ..."))

### Safer ways to use `torch.compile` on a dedicated endpoint

1. **Feature-flag compile (on/off via env variable)**

2. **Move compile from ` __init__ ` to first real request**

3. **Use appropriate mode (`reduce-overhead` for small batches)**

4. **Compile only the hot part of the model**

5. **Minimize warmup during startup**

6. **Accept that compile may not be worth it for some endpoints**

If you rearrange startup so that heavy compile and warmup are not part of the health-check window, you greatly reduce the chance of “Initializing forever”.

* * *

## 7. Putting it together: practical diagnosis flow

Here is a concise decision tree you can apply to your dedicated endpoint.

### Step 1 – Quick baseline

- Create a **small, vanilla** dedicated endpoint in the same region + GPU type.

- If that also fails or gets stuck:

### Step 2 – Examine logs

- If logs are empty or the log panel 5xx’s:

- If logs show your server running (“listening on 127.0.0.1:8188”, “server started”) but the status is still `Initializing`:

- If logs show repeated Python exceptions or OOMs:

- If logs show model loading messages and then nothing for a long time:

### Step 3 – Instrument phases (optional but very helpful)

Add explicit phase logs in ` __init__ `:

- `PHASE_START init`
- `PHASE_START load_model`
- `PHASE_START maybe_compile`
- `PHASE_END ...`

Then:

- If logs stop before `PHASE_START maybe_compile`, compile is probably not the problem; earlier steps are.
- If logs stop at `PHASE_START maybe_compile`, or you never see `PHASE_END maybe_compile`, that strongly implicates `torch.compile`.

### Step 4 – Toggle `torch.compile` and heavy work

- Disable compile and extra warmup (via env flags) and redeploy:

- If disabling compile doesn’t help, focus back on capacity, port mapping, and crash loops.

* * *

## 8. Summary: main causes and matching solutions

To make it extremely explicit:

1. **Capacity / scheduling issues (HF infra)**

2. **Load balancer / routing issues (HF infra)**

3. **Health-check / port mapping problems (custom images / servers)**

4. **Crash loop during startup (OOM, dependency, config errors)**

5. **Startup too slow due to heavy work and `torch.compile`**

---

<div class="post-metadata">

**Author:** ![meganariley](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/meganariley/32/56966_2.png) [@meganariley](https://discuss.huggingface.co/u/meganariley)\
**Post date:** [November 20, 2025, 7:52pm UTC](https://discuss.huggingface.co/t/inference-endpoint-initializing-forever/170675/3 "2025-11-20T19:52:15Z")

</div>

Hi @omarzouk55! This can happen if there’s low availability of the hardware selected and it’s not available to use just yet. If you’d like to get started right away, you can try another instance. As soon as an instance is available, your Endpoint will be up and running 🚀