# Local code generation model (12GB VRAM, 32GB RAM)

**URL:** <https://discuss.huggingface.co/t/local-code-generation-model-12gb-vram-32gb-ram/169765>\
**Category:** Beginners\
**Created:** [October 31, 2025, 8:46pm UTC](https://discuss.huggingface.co/t/local-code-generation-model-12gb-vram-32gb-ram/169765 "2025-10-31T20:46:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jjoil](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jjoil](https://discuss.huggingface.co/u/jjoil)\
**Post date:** [October 31, 2025, 8:46pm UTC](https://discuss.huggingface.co/t/local-code-generation-model-12gb-vram-32gb-ram/169765/1 "2025-10-31T20:46:11Z")

</div>

I am trying to find a best model for my setup that i can run fully on GPU. I heard about vLLM that allows you to run models on RAM, that can adaptively load on VRAM only active weights, to run on GPU models that do not fit fully on VRAM. Do vLLM really work that well? If u have a recommendations about local models, i am very grateful for any help.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [November 1, 2025, 1:00am UTC](https://discuss.huggingface.co/t/local-code-generation-model-12gb-vram-32gb-ram/169765/2 "2025-11-01T01:00:52Z")

</div>

Quantization is essential when using practical models on consumer-grade GPUs, but [quantization and CPU offloading typically don’t mix well. GGUF is well-suited for GPU-CPU hybrid environments in this regard, so Ollama is likely a better choice than vLLM for handling larger models in this scenario](https://huggingface.co/datasets/John6666/forum2/blob/main/code_gen_model_12gb_1.md). For models that fit entirely within VRAM, backends like vLLM, TGI, or SGLang should be faster.

---

<div class="post-metadata">

**Author:** ![Rzkoohi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rzkoohi/32/55298_2.png) [@Rzkoohi](https://discuss.huggingface.co/u/Rzkoohi)\
**Post date:** [November 1, 2025, 5:34am UTC](https://discuss.huggingface.co/t/local-code-generation-model-12gb-vram-32gb-ram/169765/3 "2025-11-01T05:34:55Z")

</div>

agreed to john666. vllm is more for large GPUs. for local use in consumer grade go for ollama. safer. easier and more reliable.

---

<div class="post-metadata">

**Author:** ![jjoil](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jjoil](https://discuss.huggingface.co/u/jjoil)\
**Post date:** [November 1, 2025, 8:07pm UTC](https://discuss.huggingface.co/t/local-code-generation-model-12gb-vram-32gb-ram/169765/4 "2025-11-01T20:07:59Z")

</div>

Thanks for the info, you are so right, I tested Vllm anyway and it is such a pain in the ass

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [November 2, 2025, 9:18am UTC](https://discuss.huggingface.co/t/local-code-generation-model-12gb-vram-32gb-ram/169765/5 "2025-11-02T09:18:37Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.