# Transformers Tokenizer on GPU?

**URL:** <https://discuss.huggingface.co/t/transformers-tokenizer-on-gpu/2850>\
**Category:** 🤗Transformers\
**Created:** [December 16, 2020, 2:39pm UTC](https://discuss.huggingface.co/t/transformers-tokenizer-on-gpu/2850 "2020-12-16T14:39:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jaecha](https://avatars.discourse-cdn.com/v4/letter/j/d9b06d/32.png) [@jaecha](https://discuss.huggingface.co/u/jaecha)\
**Post date:** [December 16, 2020, 2:39pm UTC](https://discuss.huggingface.co/t/transformers-tokenizer-on-gpu/2850/1 "2020-12-16T14:39:53Z")

</div>

Hi, I am finding the tokenizing takes long time when I have large text data. There may be some documentation about this somewhere, but I could not find any that address how to use multiple GPUs to process the tokenization. Any help will be much appreciated. Thanks!

---

<div class="post-metadata">

**Author:** ![clem](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/clem/32/2838_2.png) [@clem](https://discuss.huggingface.co/u/clem)\
**Post date:** [December 17, 2020, 3:29am UTC](https://discuss.huggingface.co/t/transformers-tokenizer-on-gpu/2850/2 "2020-12-17T03:29:49Z")

</div>

@Narsil might help be able to help here

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [December 17, 2020, 8:39am UTC](https://discuss.huggingface.co/t/transformers-tokenizer-on-gpu/2850/3 "2020-12-17T08:39:49Z")

</div>

Hi @jaecha.

Tokenization does not happen on GPU (and won’t anytime soon). If you can show your tokenizer config that could help understand why it takes a long time ? tokenizers version, what kind of model and maybe how large your data ?  
If you see a warning about TOKENIZERS\_PARALLELISM in your console:

- You use multiple threads (like with `DataLoader`) then it’s better to create a tokenizer instance on each thread rather than before the fork otherwise we can’t use multiple cores (because of GIL)

Having a good pre\_tokenizer is important (usually `Whitespace` splitting for languages that allow it) at least.

You can find more information about your options here: [https://huggingface.co/docs/tokenizers/python/latest/pipeline.html#pre-tokenization](https://huggingface.co/docs/tokenizers/python/latest/pipeline.html#pre-tokenization)

Does that help?

Cheers  
Nicolas

---

<div class="post-metadata">

**Author:** ![jaecha](https://avatars.discourse-cdn.com/v4/letter/j/d9b06d/32.png) [@jaecha](https://discuss.huggingface.co/u/jaecha)\
**Post date:** [December 17, 2020, 2:28pm UTC](https://discuss.huggingface.co/t/transformers-tokenizer-on-gpu/2850/4 "2020-12-17T14:28:16Z")

</div>

Thank you for your replies! I think I know what to do next. Thanks!