# Serving AWQ models without a custom container

**URL:** <https://discuss.huggingface.co/t/serving-awq-models-without-a-custom-container/61351>\
**Category:** Inference Endpoints on the Hub\
**Created:** [November 7, 2023, 8:11am UTC](https://discuss.huggingface.co/t/serving-awq-models-without-a-custom-container/61351 "2023-11-07T08:11:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![p-christ](https://avatars.discourse-cdn.com/v4/letter/p/df705f/32.png) [@p-christ](https://discuss.huggingface.co/u/p-christ)\
**Post date:** [November 7, 2023, 8:11am UTC](https://discuss.huggingface.co/t/serving-awq-models-without-a-custom-container/61351/1 "2023-11-07T08:11:52Z")

</div>

Is it possible to serve AWQ models using huggingface’s inference endpoints without using a custom container?

---

<div class="post-metadata">

**Author:** ![meganariley](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/meganariley/32/56966_2.png) [@meganariley](https://discuss.huggingface.co/u/meganariley)\
**Post date:** [November 13, 2023, 1:18pm UTC](https://discuss.huggingface.co/t/serving-awq-models-without-a-custom-container/61351/2 "2023-11-13T13:18:52Z")

</div>

Hi @p-christ! Good news–AWQ has been added as a quantization option so you can now use it with Inference Endpoints 🤗

---

<div class="post-metadata">

**Author:** ![p-christ](https://avatars.discourse-cdn.com/v4/letter/p/df705f/32.png) [@p-christ](https://discuss.huggingface.co/u/p-christ)\
**Post date:** [November 13, 2023, 1:55pm UTC](https://discuss.huggingface.co/t/serving-awq-models-without-a-custom-container/61351/3 "2023-11-13T13:55:05Z")

</div>

oh great thanks a LOT!

Do you also know if there’s an easy way of using vLLM with inference endpoints?