# CUDA Assertion Error

**URL:** <https://discuss.huggingface.co/t/cuda-assertion-error/128035>\
**Category:** Beginners\
**Created:** [November 27, 2024, 4:26am UTC](https://discuss.huggingface.co/t/cuda-assertion-error/128035 "2024-11-27T04:26:24Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![asharsha30](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@asharsha30](https://discuss.huggingface.co/u/asharsha30)\
**Post date:** [November 27, 2024, 4:26am UTC](https://discuss.huggingface.co/t/cuda-assertion-error/128035/1 "2024-11-27T04:26:24Z")

</div>

Good Evening,

I am trying to finetune meta-llama/Llama-3.2-1B for QA tasks. Before that I am trying to understand the model in general by feeding in certain prompts to understand how tokenization is carried out. However, I am facing CUDA Assertion error. I researched and understood that it was due to the issue related to GPU serialization. Could someone please help me to debug.

My Code:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model and tokenizer

model\_name = “meta-llama/Llama-3.2-1B”  
tokenizer = AutoTokenizer.from\_pretrained(model\_name, use\_auth\_token=True)  
model = AutoModelForCausalLM.from\_pretrained(model\_name, use\_auth\_token=True).to(“cuda”)

special\_tokens\_dict = {  
“pad\_token”: DEFAULT\_PAD\_TOKEN,  
“eos\_token”: DEFAULT\_EOS\_TOKEN,  
“bos\_token”: DEFAULT\_BOS\_TOKEN,  
“unk\_token”: DEFAULT\_UNK\_TOKEN,  
}  
tokenizer.add\_special\_tokens(special\_tokens\_dict)  
tokenizer.model\_max\_length = 2048  
model.resize\_token\_embeddings(len(tokenizer))

# Example prompt

prompt = “Explain the importance of machine learning in today’s world.”

# Tokenize input

inputs = tokenizer(  
prompt,  
return\_tensors=“pt”,  
truncation=True,  
max\_length=512, # Start with a manageable length  
padding=“max\_length”,  
).to(“cuda”)

# Generate output

outputs = model.generate(  
inputs[“input\_ids”],  
max\_length=256,  
temperature=0.7,  
top\_p=0.9,  
)

# Decode and print response

response = tokenizer.decode(outputs[0], skip\_special\_tokens=True)  
print(“Generated Response:”, response)

---

<div class="post-metadata">

**Author:** ![asharsha30](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@asharsha30](https://discuss.huggingface.co/u/asharsha30)\
**Post date:** [November 27, 2024, 4:37am UTC](https://discuss.huggingface.co/t/cuda-assertion-error/128035/2 "2024-11-27T04:37:30Z")

</div>

Able to rectify the case. Running on CPU provided clear error message.

if tokenizer.pad\_token is None:  
tokenizer.pad\_token = tokenizer.eos\_token

model.resize\_token\_embeddings(len(tokenizer))

After adding these lines, error got resolved