# Cannot load tokenizer for llama2

**URL:** <https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046>\
**Category:** 🤗Tokenizers\
**Created:** [September 14, 2023, 5:12pm UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046 "2023-09-14T17:12:26Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](https://discuss.huggingface.co/u/saireddy)\
**Post date:** [September 14, 2023, 5:12pm UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046/1 "2023-09-14T17:12:26Z")

</div>

Hi,  
I am trying to load tokenizer for llama2 using AutoTokenizer but I am facing this issue

“”“OSError: Can’t load tokenizer for ‘meta-llama/Llama-2-7b-hf’. If you were trying to load it from ‘[Models - Hugging Face](https://huggingface.co/models)’, make sure you don’t have a local directory with the same name. Otherwise, make sure ‘meta-llama/Llama-2-7b-hf’ is the correct path to a directory containing all relevant files for a LlamaTokenizer tokenizer.  
“””

any help is appreciated!

Thanks

---

<div class="post-metadata">

**Author:** ![linying111](https://avatars.discourse-cdn.com/v4/letter/l/a9adbd/32.png) [@linying111](https://discuss.huggingface.co/u/linying111)\
**Post date:** [October 21, 2023, 7:47am UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046/2 "2023-10-21T07:47:45Z")

</div>

Hello, have you solved this problem,I’m having the same issue too?

---

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](https://discuss.huggingface.co/u/saireddy)\
**Post date:** [October 25, 2023, 7:56pm UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046/3 "2023-10-25T19:56:26Z")

</div>

yes, we need to pass access\_token and proxy(if applicable) for tokenizers as well

---

<div class="post-metadata">

**Author:** ![Hemanth05](https://avatars.discourse-cdn.com/v4/letter/h/d2c977/32.png) [@Hemanth05](https://discuss.huggingface.co/u/Hemanth05)\
**Post date:** [February 2, 2024, 3:27pm UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046/4 "2024-02-02T15:27:39Z")

</div>

hi, how did you do that?

---

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](https://discuss.huggingface.co/u/saireddy)\
**Post date:** [March 20, 2024, 2:14pm UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046/5 "2024-03-20T14:14:02Z")

</div>

tokenizer = AutoTokenizer.from\_pretrained(model\_id, token=access\_token)  
access\_token can be generated from huggingface UI

---

<div class="post-metadata">

**Author:** ![shinieaggarwal](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@shinieaggarwal](https://discuss.huggingface.co/u/shinieaggarwal)\
**Post date:** [September 7, 2024, 7:17pm UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046/6 "2024-09-07T19:17:27Z")

</div>

hi, what did you put in the variable model\_id?? i am trying it on colab and facing this same issue

---

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](https://discuss.huggingface.co/u/saireddy)\
**Post date:** [September 13, 2024, 3:48pm UTC](https://discuss.huggingface.co/t/cannot-load-tokenizer-for-llama2/55046/7 "2024-09-13T15:48:35Z")

</div>

model\_id should be the name from hf model repository  
for example : meta-llama/Meta-Llama-3-8B