# Issue with ALLaM-7B Model in Inference API - Size Limitation Error

**URL:** <https://discuss.huggingface.co/t/issue-with-allam-7b-model-in-inference-api-size-limitation-error/144600>\
**Category:** Inference Endpoints on the Hub\
**Created:** [March 7, 2025, 10:30am UTC](https://discuss.huggingface.co/t/issue-with-allam-7b-model-in-inference-api-size-limitation-error/144600 "2025-03-07T10:30:58Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mehdidjemame](https://avatars.discourse-cdn.com/v4/letter/m/e99b99/32.png) [@mehdidjemame](https://discuss.huggingface.co/u/mehdidjemame)\
**Post date:** [March 7, 2025, 10:30am UTC](https://discuss.huggingface.co/t/issue-with-allam-7b-model-in-inference-api-size-limitation-error/144600/1 "2025-03-07T10:30:59Z")

</div>

Hi everyone,

I’m experiencing an issue with the ALLaM-7B model when trying to use it through the Hugging Face Inference API. Despite the model being listed as supported in the [HF Inference API documentation](https://huggingface.co/docs/api-inference/en/supported-models), I’m receiving an error message indicating that the model is too large to be loaded automatically.

Here’s the error message I’m seeing:

```auto
Cannot access content at: https://router.huggingface.co/hf-inference/models/ALLaM-AI/ALLaM-7B-Instruct-preview/v1/chat/completions. Make sure your token has the correct permissions. The model ALLaM-AI/ALLaM-7B-Instruct-preview is too large to be loaded automatically (14GB > 10GB).

```

Has anyone else encountered this issue? Are there any known workarounds or solutions for using the ALLaM-7B model with the Inference API? I’d appreciate any guidance or advice on how to resolve this issue.

Thanks

**Model details:**

- Model name: ALLaM-7B
- Endpoint: `https://api-inference.huggingface.co/models/ALLaM-AI/ALLaM-7B-Instruct-preview`
- Error message: Size limitation error (14GB \> 10GB)

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [March 7, 2025, 6:16pm UTC](https://discuss.huggingface.co/t/issue-with-allam-7b-model-in-inference-api-size-limitation-error/144600/2 "2025-03-07T18:16:11Z")

</div>

Models larger than 10GB will not be available through the Serverless Inference API unless Hugging Face gives individual permission.

Also, it seems that major changes are currently being made to the Serverless Inference API as a whole, so it is not known what will happen in the future…

If you really want to use it online, consider using Spaces, the Endpoint API (dedicated), or Colab Free.

> **[ALLaM-AI/ALLaM-7B-Instruct-preview · Hugging Face](https://huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

> This model is not currently available via any of the supported Inference Providers.