# Error loading Llama model

**URL:** <https://discuss.huggingface.co/t/error-loading-llama-model/67190>\
**Category:** Beginners\
**Created:** [December 27, 2023, 5:08pm UTC](https://discuss.huggingface.co/t/error-loading-llama-model/67190 "2023-12-27T17:08:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![XIAOXIAOFROZEN](https://avatars.discourse-cdn.com/v4/letter/x/c68b51/32.png) [@XIAOXIAOFROZEN](https://discuss.huggingface.co/u/XIAOXIAOFROZEN)\
**Post date:** [December 27, 2023, 5:08pm UTC](https://discuss.huggingface.co/t/error-loading-llama-model/67190/1 "2023-12-27T17:08:47Z")

</div>

I got a problem loading the Llama-2-13b-hf model with the following code,

LlamaForCausalLM.from\_pretrained(base\_model, trust\_remote\_code=True, device\_map = “cuda:0”, load\_in\_8bit = True),

A error returned as  
ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes`.

But I am pretty sure the two package are installed with version bitsandbytes-0.41.3, accelerate 0.25.0. Anyone has encountered the issue before and know how to resolve it, thank you very much!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [December 27, 2023, 6:38pm UTC](https://discuss.huggingface.co/t/error-loading-llama-model/67190/2 "2023-12-27T18:38:52Z")

</div>

Hi,

Are you running in Google Colab? If yes, restarting the runtime may help.

---

<div class="post-metadata">

**Author:** ![XIAOXIAOFROZEN](https://avatars.discourse-cdn.com/v4/letter/x/c68b51/32.png) [@XIAOXIAOFROZEN](https://discuss.huggingface.co/u/XIAOXIAOFROZEN)\
**Post date:** [December 27, 2023, 7:21pm UTC](https://discuss.huggingface.co/t/error-loading-llama-model/67190/3 "2023-12-27T19:21:14Z")

</div>

Hi, nielsr, I am running it in AWS sagemaker, I found a solution in forum by downgrading transformers to 4.30.0, the error disappears but a new error emerge: SafetensorError: Error while deserializing header: HeaderTooLarge, wonder if you have any clue on that, thank you!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [December 27, 2023, 7:35pm UTC](https://discuss.huggingface.co/t/error-loading-llama-model/67190/4 "2023-12-27T19:35:18Z")

</div>

It should also work with the latest Transformers version. Could you try creating a new environment?

---

<div class="post-metadata">

**Author:** ![XIAOXIAOFROZEN](https://avatars.discourse-cdn.com/v4/letter/x/c68b51/32.png) [@XIAOXIAOFROZEN](https://discuss.huggingface.co/u/XIAOXIAOFROZEN)\
**Post date:** [December 27, 2023, 8:14pm UTC](https://discuss.huggingface.co/t/error-loading-llama-model/67190/5 "2023-12-27T20:14:31Z")

</div>

Yes, latest version of transformers also pass that error, thank you very much!

---

<div class="post-metadata">

**Author:** ![akshat-kumar-akight](https://avatars.discourse-cdn.com/v4/letter/a/858c86/32.png) [@akshat-kumar-akight](https://discuss.huggingface.co/u/akshat-kumar-akight)\
**Post date:** [March 9, 2024, 8:14am UTC](https://discuss.huggingface.co/t/error-loading-llama-model/67190/6 "2024-03-09T08:14:43Z")

</div>

Hi, were you able to resolve the error?