# Truncating sequence -- within a pipeline

**URL:** <https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336>\
**Category:** Beginners\
**Created:** [July 16, 2020, 11:25pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336 "2020-07-16T23:25:18Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![AlanFeder](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@AlanFeder](https://discuss.huggingface.co/u/AlanFeder)\
**Post date:** [July 16, 2020, 11:25pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/1 "2020-07-16T23:25:18Z")

</div>

Hi all,

Thanks for making this forum!

I have a list of tests, one of which apparently happens to be 516 tokens long. I have been using the feature-extraction pipeline to process the texts, just using the simple function:

```python
nlp = pipeline('feature-extraction')

```

When it gets up to the long text, I get an error:

```auto
Token indices sequence length is longer than the specified maximum sequence length for this model (516 > 512). Running this sequence through the model will result in indexing errors

```

Alternately, if I do the sentiment-analysis pipeline (created by `nlp2 = pipeline('sentiment-analysis')`, I did not get the error.

Is there a way for me put an argument in the `pipeline` function to make it truncate at the max model input length? I tried reading [this](https://huggingface.co/transformers/preprocessing.html#everything-you-always-wanted-to-know-about-padding-and-truncation), but I was not sure how to make everything else in `pipeline` the same/default, except for this truncation.

---

<div class="post-metadata">

**Author:** ![AlanFeder](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@AlanFeder](https://discuss.huggingface.co/u/AlanFeder)\
**Post date:** [July 20, 2020, 7:50pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/2 "2020-07-20T19:50:52Z")

</div>

One quick follow-up – I just realized that the message earlier is just a warning, and not an error, which comes from the tokenizer portion. I then get an error on the model portion:

```auto
IndexError: index out of range in self

```

So I have two questions:

1. Is there a way to just add an argument somewhere that does the truncation automatically?
2. Is there a way for me to split out the tokenizer/model, truncate in the tokenizer, and then run that truncated in the model?

Thank you!

---

<div class="post-metadata">

**Author:** ![cx00](https://avatars.discourse-cdn.com/v4/letter/c/c68b51/32.png) [@cx00](https://discuss.huggingface.co/u/cx00)\
**Post date:** [April 12, 2021, 2:59pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/3 "2021-04-12T14:59:25Z")

</div>

Hello, have you found a solution to this? I have also come across this problem and haven’t found a solution.

---

<div class="post-metadata">

**Author:** ![AlanFeder](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@AlanFeder](https://discuss.huggingface.co/u/AlanFeder)\
**Post date:** [April 13, 2021, 10:15pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/4 "2021-04-13T22:15:50Z")

</div>

I have not – I just moved out of the “pipeline” framework, and used the building blocks. It wasn’t too bad 😃

---

<div class="post-metadata">

**Author:** ![cx00](https://avatars.discourse-cdn.com/v4/letter/c/c68b51/32.png) [@cx00](https://discuss.huggingface.co/u/cx00)\
**Post date:** [April 27, 2021, 3:40pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/5 "2021-04-27T15:40:48Z")

</div>

## I see, do you think you could share a snippet of how you did that? I am unsure of what to do with the output of a model. Thanks again!

```
model_name = "distilbert-base-uncased-finetuned-sst-2-english"
pt_model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

pt_batch = tokenizer(
    "We are very happy to show you the 🤗 Transformers library.",
    truncation=True,
    max_length=10,
    return_tensors="pt"
)

pt_outputs = pt_model(**pt_batch)

pt_outputs

```

Which gives…

`SequenceClassifierOutput(loss=None, logits=tensor([[-4.2644, 4.6002]], grad_fn=<AddmmBackward>), hidden_states=None, attentions=None)`

---

<div class="post-metadata">

**Author:** ![AlanFeder](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@AlanFeder](https://discuss.huggingface.co/u/AlanFeder)\
**Post date:** [April 27, 2021, 7:15pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/6 "2021-04-27T19:15:05Z")

</div>

try adding something like the following:

```python
from torch.nn import Softmax

smax = Softmax(dim=-1)

probs0 = smax(pt_outputs.logits)
probs0 = probs0.flatten().detach().numpy()

prob_pos = probs[1]

```

Now `prob_pos` should be the probability that the sentence is positive.

Does that work/make sense?

---

<div class="post-metadata">

**Author:** ![ttj](https://avatars.discourse-cdn.com/v4/letter/t/dc4da7/32.png) [@ttj](https://discuss.huggingface.co/u/ttj)\
**Post date:** [July 15, 2021, 11:53am UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/7 "2021-07-15T11:53:03Z")

</div>

Passing `truncation=True` in ` __call__ ` seems to suppress the error.

---

<div class="post-metadata">

**Author:** ![Mohit2112](https://avatars.discourse-cdn.com/v4/letter/m/77aa72/32.png) [@Mohit2112](https://discuss.huggingface.co/u/Mohit2112)\
**Post date:** [May 3, 2024, 5:59pm UTC](https://discuss.huggingface.co/t/truncating-sequence-within-a-pipeline/336/8 "2024-05-03T17:59:03Z")

</div>

Just add “truncation=True” in your pipeline initiation  
`pipe = pipeline("text-classification", max_length=512, truncation=True)`