# LLama pad token

**URL:** <https://discuss.huggingface.co/t/llama-pad-token/48001>\
**Category:** Beginners\
**Created:** [July 25, 2023, 5:03am UTC](https://discuss.huggingface.co/t/llama-pad-token/48001 "2023-07-25T05:03:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![hungchiayu](https://avatars.discourse-cdn.com/v4/letter/h/dec6dc/32.png) [@hungchiayu](https://discuss.huggingface.co/u/hungchiayu)\
**Post date:** [July 25, 2023, 5:03am UTC](https://discuss.huggingface.co/t/llama-pad-token/48001/1 "2023-07-25T05:03:30Z")

</div>

I notice that the llama tokenizer does not have a pad token and I tried to add the pad token manually. This is what I have done.

`tokenizer.add_special_tokens({"pad_token":"<pad>"}) model.resize_token_embeddings(len(tokenizer)) model.model.embed_tokens.padding_idx =32000`  
Is this the correct way of adding pad tokens?  
After adding the pad token in my tokenizer, my finetuned llama model stop generation with `</s><s>. `  
Is this behaviour expected? I thought the the model should stop with an eos token only, but I dont understand why it stops with an eos token and bos token

---

<div class="post-metadata">

**Author:** ![TonyN2060](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@TonyN2060](https://discuss.huggingface.co/u/TonyN2060)\
**Post date:** [February 17, 2025, 10:01pm UTC](https://discuss.huggingface.co/t/llama-pad-token/48001/2 "2025-02-17T22:01:46Z")

</div>

currently experiencing the same error, any updates on a solution?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [February 18, 2025, 5:57am UTC](https://discuss.huggingface.co/t/llama-pad-token/48001/3 "2025-02-18T05:57:27Z")

</div>

It’s not really a bug, but it seems to be an unsolved issue.

> <https://github.com/lm-sys/FastChat/issues/3266>
>
> I attempted a workaround, but the output from finetuning doesn't look quite righ…t. Has anyone made a working fix for this issue?

> <https://github.com/huggingface/transformers/issues/22312>
>
> \### System Info
> 
> System info:
> 
> \- Code: Current \`main\` branch, installed via: \`…pip install git+https://github.com/huggingface/transformers\` on 22nd March 2023
> 
> \### Who can help?
> 
> @ArthurZucker @sgugger @zphang 
> 
> \### Information
> 
> \- \[X\] The official example scripts
> \- \[\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[X\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> \- Code to reproduce:
> \`\`\`
> from transformers import LlamaTokenizer
> tokenizer = LlamaTokenizer.from\_pretrained("decapoda-research/llama-7b-hf")
> 
> print(repr(tokenizer.pad\_token)) ## None
> print(repr(tokenizer.bos\_token)) ## ''
> print(repr(tokenizer.eos\_token)) ## ''
> \`\`\`
> 
> \- Where this causes an issue:
> \`\`\`
> batch = tokenizer(
> \[
> "Singer Billy Joel yesterday ",
> "The primary use of LLaMA is research on large language "
> \],
> return\_tensors="pt",
> padding=True
> )
> \`\`\`
> 
> The above statement raises an issue:
> 
> \`\`\`
> Using pad\_token, but it is not set yet.
> \---------------------------------------------------------------------------
> ValueError Traceback (most recent call last)
> Cell In\[53\], line 1
> \----\> 1 batch = tokenizer(
> 2 \[
> 3 "Singer Billy Joel yesterday ",
> 4 "The primary use of LLaMA is research on large language "
> 5 \],
> 6 return\_tensors="pt",
> 7 padding=True
> 8 )
> 
> File /home/ec2-user/anaconda3/envs/llm-gen/lib/python3.10/site-packages/transformers/tokenization\_utils\_base.py:2531, in PreTrainedTokenizerBase.\_\_call\_\_(self, text, text\_pair, text\_target, text\_pair\_target, add\_special\_tokens, padding, truncation, max\_length, stride, is\_split\_into\_words, pad\_to\_multiple\_of, return\_tensors, return\_token\_type\_ids, return\_attention\_mask, return\_overflowing\_tokens, return\_special\_tokens\_mask, return\_offsets\_mapping, return\_length, verbose, \*\*kwargs)
> 2529 if not self.\_in\_target\_context\_manager:
> 2530 self.\_switch\_to\_input\_mode()
> \-\> 2531 encodings = self.\_call\_one(text=text, text\_pair=text\_pair, \*\*all\_kwargs)
> 2532 if text\_target is not None:
> 2533 self.\_switch\_to\_target\_mode()
> 
> File /home/ec2-user/anaconda3/envs/llm-gen/lib/python3.10/site-packages/transformers/tokenization\_utils\_base.py:2617, in PreTrainedTokenizerBase.\_call\_one(self, text, text\_pair, add\_special\_tokens, padding, truncation, max\_length, stride, is\_split\_into\_words, pad\_to\_multiple\_of, return\_tensors, return\_token\_type\_ids, return\_attention\_mask, return\_overflowing\_tokens, return\_special\_tokens\_mask, return\_offsets\_mapping, return\_length, verbose, \*\*kwargs)
> 2612 raise ValueError(
> 2613 f"batch length of \`text\`: {len(text)} does not match batch length of \`text\_pair\`:"
> 2614 f" {len(text\_pair)}."
> 2615 )
> 2616 batch\_text\_or\_text\_pairs = list(zip(text, text\_pair)) if text\_pair is not None else text
> \-\> 2617 return self.batch\_encode\_plus(
> 2618 batch\_text\_or\_text\_pairs=batch\_text\_or\_text\_pairs,
> 2619 add\_special\_tokens=add\_special\_tokens,
> 2620 padding=padding,
> 2621 truncation=truncation,
> 2622 max\_length=max\_length,
> 2623 stride=stride,
> 2624 is\_split\_into\_words=is\_split\_into\_words,
> 2625 pad\_to\_multiple\_of=pad\_to\_multiple\_of,
> 2626 return\_tensors=return\_tensors,
> 2627 return\_token\_type\_ids=return\_token\_type\_ids,
> 2628 return\_attention\_mask=return\_attention\_mask,
> 2629 return\_overflowing\_tokens=return\_overflowing\_tokens,
> 2630 return\_special\_tokens\_mask=return\_special\_tokens\_mask,
> 2631 return\_offsets\_mapping=return\_offsets\_mapping,
> 2632 return\_length=return\_length,
> 2633 verbose=verbose,
> 2634 \*\*kwargs,
> 2635 )
> 2636 else:
> 2637 return self.encode\_plus(
> 2638 text=text,
> 2639 text\_pair=text\_pair,
> (...)
> 2655 \*\*kwargs,
> 2656 )
> 
> File /home/ec2-user/anaconda3/envs/llm-gen/lib/python3.10/site-packages/transformers/tokenization\_utils\_base.py:2799, in PreTrainedTokenizerBase.batch\_encode\_plus(self, batch\_text\_or\_text\_pairs, add\_special\_tokens, padding, truncation, max\_length, stride, is\_split\_into\_words, pad\_to\_multiple\_of, return\_tensors, return\_token\_type\_ids, return\_attention\_mask, return\_overflowing\_tokens, return\_special\_tokens\_mask, return\_offsets\_mapping, return\_length, verbose, \*\*kwargs)
> 2782 """
> 2783 Tokenize and prepare for the model a list of sequences or a list of pairs of sequences.
> 2784 
> (...)
> 2795 details in \`encode\_plus\`).
> 2796 """
> 2798 # Backward compatibility for 'truncation\_strategy', 'pad\_to\_max\_length'
> \-\> 2799 padding\_strategy, truncation\_strategy, max\_length, kwargs = self.\_get\_padding\_truncation\_strategies(
> 2800 padding=padding,
> 2801 truncation=truncation,
> 2802 max\_length=max\_length,
> 2803 pad\_to\_multiple\_of=pad\_to\_multiple\_of,
> 2804 verbose=verbose,
> 2805 \*\*kwargs,
> 2806 )
> 2808 return self.\_batch\_encode\_plus(
> 2809 batch\_text\_or\_text\_pairs=batch\_text\_or\_text\_pairs,
> 2810 add\_special\_tokens=add\_special\_tokens,
> (...)
> 2825 \*\*kwargs,
> 2826 )
> 
> File /home/ec2-user/anaconda3/envs/llm-gen/lib/python3.10/site-packages/transformers/tokenization\_utils\_base.py:2436, in PreTrainedTokenizerBase.\_get\_padding\_truncation\_strategies(self, padding, truncation, max\_length, pad\_to\_multiple\_of, verbose, \*\*kwargs)
> 2434 # Test if we have a padding token
> 2435 if padding\_strategy != PaddingStrategy.DO\_NOT\_PAD and (not self.pad\_token or self.pad\_token\_id \< 0):
> \-\> 2436 raise ValueError(
> 2437 "Asking to pad but the tokenizer does not have a padding token. "
> 2438 "Please select a token to use as \`pad\_token\` \`(tokenizer.pad\_token = tokenizer.eos\_token e.g.)\` "
> 2439 "or add a new pad token via \`tokenizer.add\_special\_tokens({'pad\_token': '\[PAD\]'})\`."
> 2440 )
> 2442 # Check that we will truncate to a multiple of pad\_to\_multiple\_of if both are provided
> 2443 if (
> 2444 truncation\_strategy != TruncationStrategy.DO\_NOT\_TRUNCATE
> 2445 and padding\_strategy != PaddingStrategy.DO\_NOT\_PAD
> (...)
> 2448 and (max\_length % pad\_to\_multiple\_of != 0)
> 2449 ):
> 
> ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as \`pad\_token\` \`(tokenizer.pad\_token = tokenizer.eos\_token e.g.)\` or add a new pad token via \`tokenizer.add\_special\_tokens({'pad\_token': '\[PAD\]'})\`.
> \`\`\`
> 
> 
> \### Expected behavior
> 
> The following code should work:
> 
> \`\`\`
> from transformers import LlamaTokenizer
> tokenizer = LlamaTokenizer.from\_pretrained("decapoda-research/llama-7b-hf")
> 
> batch = tokenizer(
> \[
> "Singer Billy Joel yesterday ",
> "The primary use of LLaMA is research on large language "
> \],
> return\_tensors="pt",
> padding=True
> )
> \`\`\`

---

<div class="post-metadata">

**Author:** ![TonyN2060](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@TonyN2060](https://discuss.huggingface.co/u/TonyN2060)\
**Post date:** [February 18, 2025, 12:32pm UTC](https://discuss.huggingface.co/t/llama-pad-token/48001/4 "2025-02-18T12:32:59Z")

</div>

ValueError: Cannot handle batch sizes \> 1 if no padding token is defined., the error persists despite running ,  
tokenizer.pad\_token = tokenizer.eos\_token

model.config.pad\_token\_id = tokenizer.pad\_token\_id