# Inference Requirements

**URL:** <https://discuss.huggingface.co/t/inference-requirements/175635>\
**Category:** Intermediate\
**Created:** [April 28, 2026, 8:20pm UTC](https://discuss.huggingface.co/t/inference-requirements/175635 "2026-04-28T20:20:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paul-Arista](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/paul-arista/32/59864_2.png) [@Paul-Arista](https://discuss.huggingface.co/u/Paul-Arista)\
**Post date:** [April 28, 2026, 8:20pm UTC](https://discuss.huggingface.co/t/inference-requirements/175635/1 "2026-04-28T20:20:36Z")

</div>

Does hugging face have anything like this - [LLM Directory: All Local LLMs List](https://apxml.com/models?modelType=open_weight&page=2)

which shows inference requirements

---

<div class="post-metadata">

**Author:** ![Lightcap](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lightcap/32/59918_2.png) [@Lightcap](https://discuss.huggingface.co/u/Lightcap)\
**Post date:** [May 2, 2026, 2:02pm UTC](https://discuss.huggingface.co/t/inference-requirements/175635/2 "2026-05-02T14:02:57Z")

</div>

I don’t think Hugging Face has one central directory exactly like that.

The closest thing I know of on HF is the model memory estimator / model-memory-usage Space. It can give you a decent first estimate for whether a model will fit in VRAM, but it is not really the same as a curated LLM requirements table.

The annoying part is that “requirements” are not one number. A 7B model can mean very different things depending on fp16, int8, 4-bit, GGUF, vLLM, Transformers, llama.cpp, context length, batch size, KV cache, CPU offload, etc.

So for now I usually treat HF model cards as the source for model details, then use a memory estimator or do the rough math myself. For LLMs, fp16/bf16 is roughly 2 GB per billion parameters just for weights, 8-bit around half of that, 4-bit around a quarter, plus overhead for runtime and context.

It would be nice if HF had this as a first-class field on model pages, even if it was only approximate. Right now it is scattered between model cards, discussions, Spaces, and external tools.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [May 3, 2026, 5:17am UTC](https://discuss.huggingface.co/t/inference-requirements/175635/3 "2026-05-03T05:17:06Z")

</div>

It’s not actually run by Hugging Face itself, it’s user-generated, but leaderboards might be similar? [Spaces - Hugging Face](https://huggingface.co/spaces?sort=trending&search=leader)

Also, `Eval Results` of models: [Models – Hugging Face](https://huggingface.co/models?other=eval-results&sort=trending)

---

<div class="post-metadata">

**Author:** ![Paul-Arista](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/paul-arista/32/59864_2.png) [@Paul-Arista](https://discuss.huggingface.co/u/Paul-Arista)\
**Post date:** [May 3, 2026, 11:12am UTC](https://discuss.huggingface.co/t/inference-requirements/175635/4 "2026-05-03T11:12:45Z")

</div>

Spaces - Hugging Face I select open LLM leaderboard and get error

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [May 3, 2026, 2:43pm UTC](https://discuss.huggingface.co/t/inference-requirements/175635/5 "2026-05-03T14:43:27Z")

</div>

Oh… Report it to the author via discussion: [open-llm-leaderboard/open\_llm\_leaderboard · Discussions](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions)