# QLoRA trained LLaMA2 13B deployment error on Sagemaker using text generation inference image **URL:** \ **Category:** Amazon SageMaker\ **Created:** [July 25, 2023, 9:57pm UTC](https://discuss.huggingface.co/t/qlora-trained-llama2-13b-deployment-error-on-sagemaker-using-text-generation-inference-image/48154 "2023-07-25T21:57:34Z")\ **Posts on this page:** 1\ **Showing post:** 12
**Author:** ![rycfung](https://avatars.discourse-cdn.com/v4/letter/r/7c8e57/32.png) [@rycfung](https://discuss.huggingface.co/u/rycfung)\ **Post date:** [August 5, 2023, 10:29pm UTC](https://discuss.huggingface.co/t/qlora-trained-llama2-13b-deployment-error-on-sagemaker-using-text-generation-inference-image/48154/12 "2023-08-05T22:29:12Z")
@elodium I ended up building the 0.9.3 image from scratch (which was half a day of work between actual compiling building and figuring out some build configurations to make the build stop freezing/blowing all of the memory even with 100GB of RAM on EC2). I ended up deploying with the TGI 0.9.3 image I built on a g5.4xlarge, and it worked. Only issue was even though I deployed the 4-bit QLoRA LLaMA 2 13B, generation was pretty slow, and often freeze on the sagemaker deployment or times out after 30s. That was super odd because I was expecting the 4bit quantized 13B to breeze thru the generation on a 4xlarge. Going to try the official 093 image to see if there’s a difference. --- _[View the full topic](https://discuss.huggingface.co/t/qlora-trained-llama2-13b-deployment-error-on-sagemaker-using-text-generation-inference-image/48154)._