# Training for GPTQ, possible?

**URL:** <https://discuss.huggingface.co/t/training-for-gptq-possible/46972>\
**Category:** Intermediate\
**Created:** [July 16, 2023, 7:24am UTC](https://discuss.huggingface.co/t/training-for-gptq-possible/46972 "2023-07-16T07:24:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![zinomx](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/zinomx/32/20273_2.png) [@zinomx](https://discuss.huggingface.co/u/zinomx)\
**Post date:** [July 16, 2023, 7:24am UTC](https://discuss.huggingface.co/t/training-for-gptq-possible/46972/1 "2023-07-16T07:24:08Z")

</div>

Apparently it’s not possible to train a Lora for a GPTQ model. If I’m incorrect please let me know. We’re using TheBloke/Wizard-Vicuna-30B-Superhot-8K-GPTQ which runs fast and does everything we need it to do on our Nividia A40 with 48GB of VRAM.

However, to train a Lora for this model, I’ve concluded we need to train for the original model, which is ehartford/wizard\_vicuna\_70k\_unfiltered, which is roughly 130GB

In order to train the original model, we’ll need to lease a GPU server with 130GB of VRAM, which is roughly $3500 a month, so…can someone please tell me if a Lora trained from the original model will FOR SURE work on the GPTQ version of the model, or do I have any of this wrong?

---

<div class="post-metadata">

**Author:** ![ntheden](https://avatars.discourse-cdn.com/v4/letter/n/e480ec/32.png) [@ntheden](https://discuss.huggingface.co/u/ntheden)\
**Post date:** [October 24, 2023, 10:11pm UTC](https://discuss.huggingface.co/t/training-for-gptq-possible/46972/2 "2023-10-24T22:11:34Z")

</div>

This [article](https://dsmonk.medium.com/training-and-deploying-of-quantized-llms-with-lora-and-gptq-part-2-2-ec7b54659c9e) says you can.