# Error loading finetuned llama2 model while running inference

**URL:** <https://discuss.huggingface.co/t/error-loading-finetuned-llama2-model-while-running-inference/48850>\
**Category:** Amazon SageMaker\
**Created:** [July 31, 2023, 4:14pm UTC](https://discuss.huggingface.co/t/error-loading-finetuned-llama2-model-while-running-inference/48850 "2023-07-31T16:14:27Z")\
**Posts on this page:** 1\
**Showing post:** 10

<div class="post-metadata">

**Author:** ![guiba44](https://avatars.discourse-cdn.com/v4/letter/g/839c29/32.png) [@guiba44](https://discuss.huggingface.co/u/guiba44)\
**Post date:** [August 24, 2023, 8:57am UTC](https://discuss.huggingface.co/t/error-loading-finetuned-llama2-model-while-running-inference/48850/10 "2023-08-24T08:57:08Z")

</div>

Did you manage to run it with your own data ? I encountered the error reported in this thread when trying to deploy mine [QLoRA trained LLaMA2 13B deployment error on Sagemaker using text generation inference image - #12 by rycfung](https://discuss.huggingface.co/t/qlora-trained-llama2-13b-deployment-error-on-sagemaker-using-text-generation-inference-image/48154/12)…

[EDIT] I managed to run it on my own model: for a Llama2 13B, you need to deploy on an ml.g5.12xlarge (which is a bit weird considering you can run inference on a notebook deployed on ml.g5.2xlarge 🤷‍♂️).

---

_[View the full topic](https://discuss.huggingface.co/t/error-loading-finetuned-llama2-model-while-running-inference/48850)._