# HF Inference API: 503/504 Server Error

**URL:** <https://discuss.huggingface.co/t/hf-inference-api-503-504-server-error/148267>\
**Category:** Inference Endpoints on the Hub\
**Created:** [March 31, 2025, 7:40pm UTC](https://discuss.huggingface.co/t/hf-inference-api-503-504-server-error/148267 "2025-03-31T19:40:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![applied-ai-concordia](https://avatars.discourse-cdn.com/v4/letter/a/e47c2d/32.png) [@applied-ai-concordia](https://discuss.huggingface.co/u/applied-ai-concordia)\
**Post date:** [March 31, 2025, 7:40pm UTC](https://discuss.huggingface.co/t/hf-inference-api-503-504-server-error/148267/1 "2025-03-31T19:40:34Z")

</div>

Hello,  
I am trying to run inference as shown in the code snippet below, but always get a 504 or 503 server error. I’ve tried different models and different ways of calling the Inference API, but always run into the same problem. However, it appears that this problem is specific to my HF account since I’ve checked that others have been able to run the exact same code without issues. Does anyone know what could be going wrong?

CODE SNIPPET

```auto
from huggingface_hub import InferenceClient

# Initialize Hugging Face InferenceClient
client = InferenceClient(
   model="facebook/opt-1.3b",
   token="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
)

result = client.text_generation(
   prompt="Hello you are a chatbot, answer this ",
   model="facebook/opt-1.3b",
)
result

```

## ERROR

HTTPError Traceback (most recent call last)  
/usr/local/lib/python3.11/dist-packages/huggingface\_hub/utils/\_http.py in hf\_raise\_for\_status(response, endpoint\_name)  
408 try:  
 → 409 response.raise\_for\_status()  
410 except HTTPError as e:

6 frames  
HTTPError: 504 Server Error: Gateway Time-out for url: [https://router.huggingface.co/hf-inference/models/facebook/opt-1.3b](https://router.huggingface.co/hf-inference/models/facebook/opt-1.3b)

The above exception was the direct cause of the following exception:

HfHubHTTPError Traceback (most recent call last)  
/usr/local/lib/python3.11/dist-packages/huggingface\_hub/utils/\_http.py in hf\_raise\_for\_status(response, endpoint\_name)  
479 # Convert `HTTPError` into a `HfHubHTTPError` to display request information  
480 # as well (request id and/or server error message)  
 → 481 raise \_format(HfHubHTTPError, str(e), response) from e  
482  
483

HfHubHTTPError: 504 Server Error: Gateway Time-out for url: [https://router.huggingface.co/hf-inference/models/facebook/opt-1.3b](https://router.huggingface.co/hf-inference/models/facebook/opt-1.3b)

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 1, 2025, 11:08am UTC](https://discuss.huggingface.co/t/hf-inference-api-503-504-server-error/148267/2 "2025-04-01T11:08:37Z")

</div>

The API seems to be in a bad state at the moment.

```py
#model_id = "facebook/opt-1.3b" # No response for a long time...
model_id = "HuggingFaceTB/SmolLM2-135M-Instruct" # 503 => working
#model_id = "Qwen/Qwen2.5-3B-Instruct" # 503 => no response for a long time...

HF_TOKEN = "hf_my_pro_token***"

# Initialize Hugging Face InferenceClient
client = InferenceClient(
   model=model_id,
   token=HF_TOKEN,
   provider="hf-inference",
   timeout=600,
)

result = client.text_generation(
   prompt="Hello you are a chatbot, answer this ",
   model=model_id,
)

print(result)

```

---

<div class="post-metadata">

**Author:** ![saranjothi](https://avatars.discourse-cdn.com/v4/letter/s/8491ac/32.png) [@saranjothi](https://discuss.huggingface.co/u/saranjothi)\
**Post date:** [September 4, 2025, 6:46pm UTC](https://discuss.huggingface.co/t/hf-inference-api-503-504-server-error/148267/3 "2025-09-04T18:46:10Z")

</div>

Hello,  
I am getting 504 Gateway Timeout errors on API’s, i have tried more other API’s too, getting same error.  
@John6666 please take a look

curl --location --request POST ‘[https://api-inference.huggingface.co/models/facebook/bart-large-mnli’](https://api-inference.huggingface.co/models/facebook/bart-large-mnli%E2%80%99) \

--header ‘Authorization: Bearer xxxxxxxxxxxx’ \

--header ‘Content-Type: application/json’ \

--data-raw '{

“inputs”: “I love using n8n for workflows!”,

“parameters”: {

“candidate\_labels”: [“automation”, “sports”, “finance”]

}

}

’

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [September 4, 2025, 11:26pm UTC](https://discuss.huggingface.co/t/hf-inference-api-503-504-server-error/148267/4 "2025-09-04T23:26:30Z")

</div>

I tried and also got 504 timeout for both `https://api-inference.huggingface.co/models/facebook/bart-large-mnli` and `https://router.huggingface.co/hf-inference/models/facebook/bart-large-mnli` for now.  
It [doesn’t seem to be a server glitch](https://status.huggingface.co/). @tomaarsen

---

<div class="post-metadata">

**Author:** ![dissentwatch](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dissentwatch/32/53412_2.png) [@dissentwatch](https://discuss.huggingface.co/u/dissentwatch)\
**Post date:** [September 5, 2025, 3:56pm UTC](https://discuss.huggingface.co/t/hf-inference-api-503-504-server-error/148267/5 "2025-09-05T15:56:38Z")

</div>

Same here, has been failing for two days on my end. HF Enterprise support has been looking into it since then but have apparently been unable to resolve it.

> **[sentence-transformers/all-MiniLM-L6-v2 · API error for model...](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/discussions/131#68b8b603f2940040d6fb0c52)**
>
> Hello,