# Pegasus Tokenizer Error

**URL:** <https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509>\
**Category:** 🤗Transformers\
**Created:** [December 4, 2020, 5:30am UTC](https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509 "2020-12-04T05:30:49Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![shailesh897](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shailesh897/32/751_2.png) [@shailesh897](https://discuss.huggingface.co/u/shailesh897)\
**Post date:** [December 4, 2020, 5:30am UTC](https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509/1 "2020-12-04T05:30:49Z")

</div>

While working with pegasus(using hugging face tranformer implimentation) I got this error… also tried to install sentencepiece library using pip… still showing the same error can anyone help? this is my code

**model\_name = ‘google/pegasus-aeslc’#'google/pegasus-xsum’**  
**torch\_device = ‘cuda’ if torch.cuda.is\_available() else 'cpu’**  
**tokenizer1 = PegasusTokenizer.from\_pretrained(model\_name)**  
**model = PegasusForConditionalGeneration.from\_pretrained(model\_name).to(torch\_device)**

Error:  
PegasusTokenizer requires the SentencePiece library but it was not found in your environment. Checkout the instructions on the  
installation page of its repo: [https://github.com/google/sentencepiece#installation](https://github.com/google/sentencepiece#installation) and follow the ones  
that match your environment.

---

<div class="post-metadata">

**Author:** ![marcoabrate](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcoabrate/32/441_2.png) [@marcoabrate](https://discuss.huggingface.co/u/marcoabrate)\
**Post date:** [December 4, 2020, 8:52am UTC](https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509/2 "2020-12-04T08:52:14Z")

</div>

You are not showing the error, but I am having the same problem with the SentencePiece library and PegasusTokenizer.

---

<div class="post-metadata">

**Author:** ![shailesh897](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shailesh897/32/751_2.png) [@shailesh897](https://discuss.huggingface.co/u/shailesh897)\
**Post date:** [December 4, 2020, 8:49pm UTC](https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509/3 "2020-12-04T20:49:09Z")

</div>

If you have same error as mine then

use  
**!pip install Transformers==3.2.0** instead of _!pip install Transformers_

---

<div class="post-metadata">

**Author:** ![marcoabrate](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcoabrate/32/441_2.png) [@marcoabrate](https://discuss.huggingface.co/u/marcoabrate)\
**Post date:** [December 7, 2020, 3:43pm UTC](https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509/4 "2020-12-07T15:43:04Z")

</div>

Opening an issue on GitHub  
[#8963](https://github.com/huggingface/transformers/issues/8963)

---

<div class="post-metadata">

**Author:** ![rohansheelvant](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@rohansheelvant](https://discuss.huggingface.co/u/rohansheelvant)\
**Post date:** [June 26, 2023, 10:30pm UTC](https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509/5 "2023-06-26T22:30:51Z")

</div>

Restarting the kernel and using  
**!pip install Transformers==3.2.0** instead of _!pip install Transformers_, worked for me

---

<div class="post-metadata">

**Author:** ![KayembaLuwaga](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@KayembaLuwaga](https://discuss.huggingface.co/u/KayembaLuwaga)\
**Post date:** [December 29, 2023, 3:27am UTC](https://discuss.huggingface.co/t/pegasus-tokenizer-error/2509/6 "2023-12-29T03:27:05Z")

</div>

> Use:

```auto
!pip install transformers==3.2.0
!pip install transformers[sentencepiece]
!pip install sentencepiece

```

> Make sure to restart your runtime before running again