# Flash Attention 2 Error on Mistral Based Model

**URL:** <https://discuss.huggingface.co/t/flash-attention-2-error-on-mistral-based-model/66167>\
**Category:** Beginners\
**Created:** [December 18, 2023, 7:05am UTC](https://discuss.huggingface.co/t/flash-attention-2-error-on-mistral-based-model/66167 "2023-12-18T07:05:26Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![luisfrentzen](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/luisfrentzen/32/22208_2.png) [@luisfrentzen](https://discuss.huggingface.co/u/luisfrentzen)\
**Post date:** [December 18, 2023, 7:05am UTC](https://discuss.huggingface.co/t/flash-attention-2-error-on-mistral-based-model/66167/1 "2023-12-18T07:05:26Z")

</div>

Hi, I am trying to enable flash attention 2 on a model yet I got this error:

```auto
ValueError: past key much have a shape of (`batch_size, num_heads, self.config.sliding_window-1, head_dim`), got torch.Size([4, 8, 3968, 128])

```

I am using openchat’s openchat\_3.5 7B model which I believe is based on mistral [openchat/openchat\_3.5 · Hugging Face](https://huggingface.co/openchat/openchat_3.5). I am loading the model as such:

```auto
model = AutoModelForCausalLM.from_pretrained(model_name, 
                             device_map="auto", 
                             torch_dtype=torch.bfloat16,
                             use_flash_attention_2="flash_attention_2",
                             low_cpu_mem_usage=True)

```

Can someone explain the error for me? Or point me to a resource which can help me understand the problem, thank you.