# Update encode function slowTokenizer vs FastTokenizer

**URL:** <https://discuss.huggingface.co/t/update-encode-function-slowtokenizer-vs-fasttokenizer/96946>\
**Category:** 🤗Tokenizers\
**Created:** [July 12, 2024, 8:04am UTC](https://discuss.huggingface.co/t/update-encode-function-slowtokenizer-vs-fasttokenizer/96946 "2024-07-12T08:04:40Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![hathyzazda](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@hathyzazda](https://discuss.huggingface.co/u/hathyzazda)\
**Post date:** [July 12, 2024, 8:04am UTC](https://discuss.huggingface.co/t/update-encode-function-slowtokenizer-vs-fasttokenizer/96946/1 "2024-07-12T08:04:40Z")

</div>

Hello,

I am trying to build a personalized tokenizer on top on GPT2 Tokenizer, so that’s why I am modifying the code. I want to build encode words following a specific manner.

The code for the slow tokenizer is here:

> <https://github.com/huggingface/transformers/blob/a695c18649fc6ab4b1fb1d9c8cfa9258c5908e2a/src/transformers/models/gpt2/tokenization_gpt2.py>

The code for the fast tokenizer is here:

> <https://github.com/huggingface/transformers/blob/a695c18649fc6ab4b1fb1d9c8cfa9258c5908e2a/src/transformers/models/gpt2/tokenization_gpt2_fast.py>

To do pre-tokenization for the slowTokenizer is use “\_tokenize()” method.

My question is how to do the same for the fast tokenizer?