# Huggingface\_hub.client giving error on list\_deployed\_models()

**URL:** <https://discuss.huggingface.co/t/huggingface-hub-client-giving-error-on-list-deployed-models/143678>\
**Category:** Intermediate\
**Created:** [March 2, 2025, 3:38pm UTC](https://discuss.huggingface.co/t/huggingface-hub-client-giving-error-on-list-deployed-models/143678 "2025-03-02T15:38:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![acloudfan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/acloudfan/32/30634_2.png) [@acloudfan](https://discuss.huggingface.co/u/acloudfan)\
**Post date:** [March 2, 2025, 3:38pm UTC](https://discuss.huggingface.co/t/huggingface-hub-client-giving-error-on-list-deployed-models/143678/1 "2025-03-02T15:38:07Z")

</div>

Hi all ,  
I am getting the following error when I call the `list_deployed_models(`) on the `InferenceClient`.

> **[Inference](https://huggingface.co/docs/huggingface_hub/package_reference/inference_client#huggingface_hub.InferenceClient)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

```auto
framework = "text-generation-inference" # "text-to-speech", 
deployed_models = client.list_deployed_models([framework])
print(deployed_models)

```

Error:

```auto
BadRequestError: (Request ID: Root=1-67c47933-02bf30da3e80cb5307dc9184;4dbda49a-b0b7-48bc-b0f7-5f7e09fc28d6)

Bad request:
Not allowed to request framework/text-generation-inference for provider hf-inference

```

Any help is appreciated

Regards, Raj

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [March 2, 2025, 8:38pm UTC](https://discuss.huggingface.co/t/huggingface-hub-client-giving-error-on-list-deployed-models/143678/2 "2025-03-02T20:38:32Z")

</div>

I think the URL for the reference is no longer valid. Perhaps the library hasn’t been updated with the new URL?

There doesn’t seem to be an issue, but it might be fixed eventually.

> <https://github.com/huggingface/huggingface_hub/issues/1557>
>
> The new \`InferenceClient\` allows to run inference on the cloud. For each impleme…nted task, new users that don't know which model to use can use the recommended model on the (free) Inference API (implemented in https://github.com/huggingface/huggingface\_hub/pull/1510). Since the recommended model might not be suitable for all use cases, it is also possible to specify any model as input to the \`InferenceClient\`. While most models on the HuggingFace Hub is compatible with Inference API, waiting for the models to load can take a lot of time. Also, for large models the Inference API just cannot load them.
> 
> To ease user experience, it would be nice to have a utility \`list\_deployed\_models\` that would return a list of already deployed models for the user to try.
> 
> \`\`\`py
> class InferenceClient:
> ...
>    
> def list\_deployed\_models(self, \*, token: Optional\[str\] = None) -\> Dict\[str, List\[str\]\]:
> ...
> return {"audio-to-audio": \["microsoft/speecht5\_vc", ...\], "text-generation": \[...\], ...}
> \`\`\`
> 
> This helper should help with discoverability.
> 
> \---
> 
> Implementation-wise, this list of deployed models can be retrieved using the \`"https://api-inference.huggingface.co/framework/{framework\_name}"\` where "framework\_name" is a framework compatible with the Inference API. For example:
> \- https://api-inference.huggingface.co/framework/transformers
> \- https://api-inference.huggingface.co/framework/diffusers
> \- https://api-inference.huggingface.co/framework/text-generation-inference
> \- https://api-inference.huggingface.co/framework/sentence-transformers
> 
> Around ~30 frameworks are available but I think we should first focus on the main frameworks. Each url returns a list of dictionaries. Each item is a deployed model: \`{"compute\_type":"cpu","model\_id":"microsoft/speecht5\_vc","sha":"c418ba2144598f973d0fddc9fd5909a3af83de3d","task":"audio-to-audio"}\`. What is interesting for us is \`"model\_id"\` and \`"task"\`. What a \`list\_deployed\_models\` method would do is to loop through all main framework, loop though all items and build an output dictionary with 'task' as key and a list of model ids as value.
> 
> Example:
> 
> \`\`\`py
> \>\>\> from huggingface\_hub import InferenceClient
> 
> \>\>\> client = InferenceClient()
> \>\>\> client.deployed\_models()\["audio-to-audio"\]
> \["microsoft/speecht5\_vc", ...\]
> \`\`\`
> 
> Happy to discuss the API design and update it if needed :) We still have to figure out what to do if several "sha" are deployed for a single model.

> <https://github.com/huggingface/huggingface_hub/blob/v0.29.1/src/huggingface_hub/inference/_client.py#L3325>

---

<div class="post-metadata">

**Author:** ![nnilayy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nnilayy/32/39238_2.png) [@nnilayy](https://discuss.huggingface.co/u/nnilayy)\
**Post date:** [March 3, 2025, 12:00am UTC](https://discuss.huggingface.co/t/huggingface-hub-client-giving-error-on-list-deployed-models/143678/3 "2025-03-03T00:00:25Z")

</div>

Hey @acloudfan,  
I ran the code through `InferenceClient` and was able to replicate the same error you encountered. However, the error stack also includes an additional warning indicating that listing models via `InferenceClient` has been deprecated. Going forward, only warm models can be referenced, and this must be done using `HfApi`, not `InferenceClient`:  
Warning:

```bash
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_deployed_models' (from 'huggingface_hub.inference._client') is deprecated and will be removed from version '0.33.0'. HF Inference API is getting revamped and will only support warm models in the future (no cold start allowed). Use `HfApi.list_models(..., inference_provider='...')` to list warm models per provider.  
  warnings.warn(warning_message, FutureWarning)

```

As the warning suggests, modifying the implementation to use `HfApi` instead of `InferenceClient` allows model listing to work correctly. The following code successfully retrieves and lists the models:

```python
import os
os.environ["HF_TOKEN"] = "hf_yourhftoken"

```

```python
from huggingface_hub import HfApi
api = HfApi()
models = api.list_models(filter="text-generation-inference", limit=1000)
for model in models:
    print(model)

```

You can use the code above to list models without any issues. Feel free to modify or remove the limit argument if you need to retrieve all models for a specific task.

For reference, you can check the list of available tasks in this message from @Wauplin:

> <https://github.com/huggingface/huggingface_hub/issues/1557#issuecomment-1631002597>
>
> The new \`InferenceClient\` allows to run inference on the cloud. For each impleme…nted task, new users that don't know which model to use can use the recommended model on the (free) Inference API (implemented in https://github.com/huggingface/huggingface\_hub/pull/1510). Since the recommended model might not be suitable for all use cases, it is also possible to specify any model as input to the \`InferenceClient\`. While most models on the HuggingFace Hub is compatible with Inference API, waiting for the models to load can take a lot of time. Also, for large models the Inference API just cannot load them.
> 
> To ease user experience, it would be nice to have a utility \`list\_deployed\_models\` that would return a list of already deployed models for the user to try.
> 
> \`\`\`py
> class InferenceClient:
> ...
>    
> def list\_deployed\_models(self, \*, token: Optional\[str\] = None) -\> Dict\[str, List\[str\]\]:
> ...
> return {"audio-to-audio": \["microsoft/speecht5\_vc", ...\], "text-generation": \[...\], ...}
> \`\`\`
> 
> This helper should help with discoverability.
> 
> \---
> 
> Implementation-wise, this list of deployed models can be retrieved using the \`"https://api-inference.huggingface.co/framework/{framework\_name}"\` where "framework\_name" is a framework compatible with the Inference API. For example:
> \- https://api-inference.huggingface.co/framework/transformers
> \- https://api-inference.huggingface.co/framework/diffusers
> \- https://api-inference.huggingface.co/framework/text-generation-inference
> \- https://api-inference.huggingface.co/framework/sentence-transformers
> 
> Around ~30 frameworks are available but I think we should first focus on the main frameworks. Each url returns a list of dictionaries. Each item is a deployed model: \`{"compute\_type":"cpu","model\_id":"microsoft/speecht5\_vc","sha":"c418ba2144598f973d0fddc9fd5909a3af83de3d","task":"audio-to-audio"}\`. What is interesting for us is \`"model\_id"\` and \`"task"\`. What a \`list\_deployed\_models\` method would do is to loop through all main framework, loop though all items and build an output dictionary with 'task' as key and a list of model ids as value.
> 
> Example:
> 
> \`\`\`py
> \>\>\> from huggingface\_hub import InferenceClient
> 
> \>\>\> client = InferenceClient()
> \>\>\> client.deployed\_models()\["audio-to-audio"\]
> \["microsoft/speecht5\_vc", ...\]
> \`\`\`
> 
> Happy to discuss the API design and update it if needed :) We still have to figure out what to do if several "sha" are deployed for a single model.

Hope this helps! Let me know if it resolves your issue.