# ValueError: cannot reshape array of size (GGUF)

**URL:** <https://discuss.huggingface.co/t/valueerror-cannot-reshape-array-of-size-gguf/99994>\
**Category:** 🤗Transformers\
**Created:** [July 29, 2024, 11:17pm UTC](https://discuss.huggingface.co/t/valueerror-cannot-reshape-array-of-size-gguf/99994 "2024-07-29T23:17:49Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![H4rryM3ss](https://avatars.discourse-cdn.com/v4/letter/h/e19b73/32.png) [@H4rryM3ss](https://discuss.huggingface.co/u/H4rryM3ss)\
**Post date:** [July 29, 2024, 11:17pm UTC](https://discuss.huggingface.co/t/valueerror-cannot-reshape-array-of-size-gguf/99994/1 "2024-07-29T23:17:49Z")

</div>

Hey guys,

I’m trying to load this GGUF model [dolphin-2.0-mistral-7b.Q4\_K\_M.gguf](https://huggingface.co/TheBloke/dolphin-2.0-mistral-7B-GGUF/blob/main/dolphin-2.0-mistral-7b.Q4_K_M.gguf) with this Python code:

```auto
model_name = "TheBloke/dolphin-2.0-mistral-7B-GGUF"
model_file = "dolphin-2.0-mistral-7b.Q4_K_M.gguf"
tokenizer = AutoTokenizer.from_pretrained(model_name, gguf_file=model_file)
model = AutoModelForCausalLM.from_pretrained(model_name, gguf_file=model_file).to(device)

```

It loads the tokenizer, but when it tries to load the model I get:

```auto
ValueError: cannot reshape array of size 36864000 into shape (222,72)

```

Which breaks the application.

Am I loading it wrongly? I was following [this guide](https://huggingface.co/docs/transformers/en/gguf).

---

<div class="post-metadata">

**Author:** ![jpuser1](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@jpuser1](https://discuss.huggingface.co/u/jpuser1)\
**Post date:** [July 31, 2024, 1:59pm UTC](https://discuss.huggingface.co/t/valueerror-cannot-reshape-array-of-size-gguf/99994/2 "2024-07-31T13:59:48Z")

</div>

I also see the same type of error.

```auto
filename = "Meta-Llama-3.1-8B-Instruct-Q8_0.gguf"
tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
model = AutoModel.from_pretrained(model_id, gguf_file=filename, local_files_only=True)

```

Converting and de-quantizing GGUF tensors…: 0%| | 1/292 [00:00\<00:00, 2328.88it/s]  
Traceback (most recent call last):  
File “./tokenize\_text.py”, line 15, in   
model = AutoModel.from\_pretrained(model\_id, gguf\_file=filename, local\_files\_only=True)  
File “/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/auto/auto\_factory.py”, line 564, in from\_pretrained  
return model\_class.from\_pretrained(  
File “/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/modeling\_utils.py”, line 3661, in from\_pretrained  
state\_dict = load\_gguf\_checkpoint(gguf\_path, return\_tensors=True)[“tensors”]  
File “/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/modeling\_gguf\_pytorch\_utils.py”, line 148, in load\_gguf\_checkpoint  
weights = load\_dequant\_gguf\_tensor(shape=shape, ggml\_type=tensor.tensor\_type, data=tensor.data)  
File “/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/integrations/ggml.py”, line 493, in load\_dequant\_gguf\_tensor  
values = dequantize\_q8\_0(data)  
File “/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/integrations/ggml.py”, line 335, in dequantize\_q8\_0  
scales = np.frombuffer(data, dtype=np.float16).reshape(num\_blocks, 1 + 16)[:, :1].astype(np.float32)  
ValueError: cannot reshape array of size 279085056 into shape (3772,17)

Same symptoms, the tokenizer loads fine but then I get this reshape array error when trying to create the model. I have tried various different GGUF files with 4 bit quantization and 8 bit quantization but it doesn’t seem to matter.  
I’m not sure what I am doing wrong.

---

<div class="post-metadata">

**Author:** ![jpuser1](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@jpuser1](https://discuss.huggingface.co/u/jpuser1)\
**Post date:** [July 31, 2024, 7:20pm UTC](https://discuss.huggingface.co/t/valueerror-cannot-reshape-array-of-size-gguf/99994/3 "2024-07-31T19:20:25Z")

</div>

Possibly due to this bug:

> <https://github.com/huggingface/transformers/issues/32331>
>
> \### System Info
> 
> Python 3.11.2
> Debian 12
> ctransformers 0.2.27
> tran…sformers 4.42.4
> 
> \### Who can help?
> 
> @ArthurZucker 
> 
> \### Information
> 
> \- \[X\] The official example scripts
> \- \[\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> Following the example on Hugging Face: https://huggingface.co/docs/transformers/en/gguf
> 
> \`\`\`
> import torch
> from transformers import AutoModelForCausalLM, AutoTokenizer
> from huggingface\_hub import hf\_hub\_download
> from llama\_cpp import Llama
> 
> \# Ensure the model is loaded on the GPU
> device = torch.device('cuda' if torch.cuda.is\_available() else 'cpu')
> print(f"device set to: {device}")
> 
> \# Load the model and tokenizer
> model\_name = "TheBloke/dolphin-2.0-mistral-7B-GGUF"
> model\_file = "dolphin-2.0-mistral-7b.Q4\_K\_M.gguf"
> print(f"Loading tokenizer from {model\_name}.")
> tokenizer = AutoTokenizer.from\_pretrained(model\_name, gguf\_file=model\_file)
> print(f"Loading model from repo {model\_name}. Model file name {model\_file}.")
> model = AutoModelForCausalLM.from\_pretrained(model\_name, gguf\_file=model\_file)
> \`\`\`
> I get
> \`\`\`
> ValueError: cannot reshape array of size 36864000 into shape (222,72)
> \`\`\`
> 
> \### Expected behavior
> 
> Model should be loaded.

---

<div class="post-metadata">

**Author:** ![jpuser1](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@jpuser1](https://discuss.huggingface.co/u/jpuser1)\
**Post date:** [July 31, 2024, 9:51pm UTC](https://discuss.huggingface.co/t/valueerror-cannot-reshape-array-of-size-gguf/99994/4 "2024-07-31T21:51:09Z")

</div>

@H4rryM3ss I was able to get past this error by cloning the huggingface/transformers repository and running the latest code from there. At least the model loads now.

---

<div class="post-metadata">

**Author:** ![H4rryM3ss](https://avatars.discourse-cdn.com/v4/letter/h/e19b73/32.png) [@H4rryM3ss](https://discuss.huggingface.co/u/H4rryM3ss)\
**Post date:** [July 31, 2024, 10:04pm UTC](https://discuss.huggingface.co/t/valueerror-cannot-reshape-array-of-size-gguf/99994/5 "2024-07-31T22:04:05Z")

</div>

Hey @jpuser1, thanks for letting me know. I had workaround this by loading the model with Llama. But I will give a try to what you have described then.

Regards.