# Running GGUF model files using Auto classes

**URL:** <https://discuss.huggingface.co/t/running-gguf-model-files-using-auto-classes/75589>\
**Category:** 🤗Transformers\
**Created:** [March 2, 2024, 12:25pm UTC](https://discuss.huggingface.co/t/running-gguf-model-files-using-auto-classes/75589 "2024-03-02T12:25:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rpiyush](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@rpiyush](https://discuss.huggingface.co/u/rpiyush)\
**Post date:** [March 2, 2024, 12:25pm UTC](https://discuss.huggingface.co/t/running-gguf-model-files-using-auto-classes/75589/1 "2024-03-02T12:25:48Z")

</div>

HI,  
I am new llm and hugging face. I tried to use Auto classes with llama 2(heBloke/CodeLlama-7B-GGUF) model gguf file(codellama-7b.q4\_K\_M.gguf) and was unable to get tokenizer or model using following code:  
AutoModelForCausalLM.from\_pretrained(“TheBloke/CodeLlama-7B-GGUF”, model\_file=“codellama-7b.q4\_K\_M.gguf”, model\_type=“llama”, gpu\_layers=50)  
tokenizer = AutoTokenizer.from\_pretrained(‘TheBloke/Llama-2-13B-GGUF’,model\_file=‘llama-2-13b.Q4\_K\_M.gguf’)

I am getting value error like : ValueError: Model file ‘codellama-7b.q4\_K\_M.gguf’ not found in ‘/root/.cache/huggingface/hub/models–TheBloke–CodeLlama-7B-GGUF/snapshots/98596f7f6c318118824bcbee4b0e20010ec510ec’

Can you please help, how can i get tokenizer or model instance using Auto classes ? Using llama cpp I can load model but not sure about tokenizer. Is this like gguf model files cannot work with Auto classes?

---

<div class="post-metadata">

**Author:** ![x0BT](https://avatars.discourse-cdn.com/v4/letter/x/c37758/32.png) [@x0BT](https://discuss.huggingface.co/u/x0BT)\
**Post date:** [March 2, 2024, 12:51pm UTC](https://discuss.huggingface.co/t/running-gguf-model-files-using-auto-classes/75589/2 "2024-03-02T12:51:40Z")

</div>

Why don’t you use llama-cpp-python? As far as I know, you can use the model tokenizer something like `model.tokenizer()` or `model.tokenize("text".encode("utf-8"))`

---

<div class="post-metadata">

**Author:** ![rpiyush](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@rpiyush](https://discuss.huggingface.co/u/rpiyush)\
**Post date:** [March 2, 2024, 1:39pm UTC](https://discuss.huggingface.co/t/running-gguf-model-files-using-auto-classes/75589/3 "2024-03-02T13:39:59Z")

</div>

As I said, I am learning and I want get basics write. looking at code on internet and examples allows you to use different model and using AutoClasses is super easy. However, there is some concept that I am missing and hence unable to use Auto class with specific gguf file. Thanks for your reply.