# How are the inputs tokenized when model deployment?

**URL:** <https://discuss.huggingface.co/t/how-are-the-inputs-tokenized-when-model-deployment/9692>\
**Category:** Amazon SageMaker\
**Created:** [September 2, 2021, 1:39pm UTC](https://discuss.huggingface.co/t/how-are-the-inputs-tokenized-when-model-deployment/9692 "2021-09-02T13:39:10Z")\
**Posts on this page:** 1\
**Showing post:** 12

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [September 3, 2021, 9:13am UTC](https://discuss.huggingface.co/t/how-are-the-inputs-tokenized-when-model-deployment/9692/12 "2021-09-03T09:13:44Z")

</div>

> [@Oigres](#):
>
> `Token indices sequence length is longer than the specified maximum sequence length for this model (520 > 512). Running this sequence through the model will result in indexing errors"`

The Error is pretty clear. It looks like `truncate` is not applied correctly.  
Which model are you using?

---

_[View the full topic](https://discuss.huggingface.co/t/how-are-the-inputs-tokenized-when-model-deployment/9692)._