# Tokenizer.pad\_token=what?

**URL:** <https://discuss.huggingface.co/t/tokenizer-pad-token-what/24367>\
**Category:** 🤗Tokenizers\
**Created:** [October 13, 2022, 12:49am UTC](https://discuss.huggingface.co/t/tokenizer-pad-token-what/24367 "2022-10-13T00:49:42Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![MLEnthusiast](https://avatars.discourse-cdn.com/v4/letter/m/85e7bf/32.png) [@MLEnthusiast](https://discuss.huggingface.co/u/MLEnthusiast)\
**Post date:** [October 13, 2022, 12:49am UTC](https://discuss.huggingface.co/t/tokenizer-pad-token-what/24367/1 "2022-10-13T00:49:42Z")

</div>

Hey everyone! Trying out some fine-tuning and I’m not exactly sure how I fix this error:

ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as ‘pad\_token’ ‘(tokenizer.pad\_token = tokenizer.eos\_token e.g.)’ or add a new pad token via ‘tokenizer.add\_special\_tokens ({‘pad\_token’: ‘[PAD]’})’

I’m trying to fine-tune [openai-gpt](https://huggingface.co/openai-gpt) when receiving this error.

I’m relatively new to FT and not really understanding the concept so far behind tokenizers and padding. Would love some clarification with any examples/solutions!

(I’ve found Huggingface tutorials on this to be too high level for a beginner).

Cheers!

---

<div class="post-metadata">

**Author:** ![lianghsun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lianghsun/32/57774_2.png) [@lianghsun](https://discuss.huggingface.co/u/lianghsun)\
**Post date:** [October 25, 2022, 8:14pm UTC](https://discuss.huggingface.co/t/tokenizer-pad-token-what/24367/2 "2022-10-25T20:14:43Z")

</div>

Actually there is no short cut for you to learn HF Tokenizers library… I’ll suggest you should take a look at HF Course and HF Tokenizers documentation to learn how to use it. However I’ll give you a hint how to deal with padding: first of all, you should specify `[PAD]` token in `BpeTrainer`

```auto
trainer = BpeTrainer(special_tokens=[..., '[PAD]', ...])

```

Second, after training the tokenizer, you can specify `tokenzier.enable_padding(pad_id=tokenizer.token_to_id('[PAD]'))`. Then you encode any sentence may contain padding follow by.

Please note why we set `.enable_padding()` after training? This because when training to _learn_ BPE vocab, it’s no need to consider padding in progress. Padding is the thing that to consider after tokenizing the sentense (truncation as well).

---

<div class="post-metadata">

**Author:** ![MLEnthusiast](https://avatars.discourse-cdn.com/v4/letter/m/85e7bf/32.png) [@MLEnthusiast](https://discuss.huggingface.co/u/MLEnthusiast)\
**Post date:** [November 8, 2022, 11:20am UTC](https://discuss.huggingface.co/t/tokenizer-pad-token-what/24367/3 "2022-11-08T11:20:46Z")

</div>

Thanks very much for your response Lianghsun 🙂