# Sentiment analysis for long sequences

**URL:** <https://discuss.huggingface.co/t/sentiment-analysis-for-long-sequences/516>\
**Category:** Beginners\
**Created:** [July 29, 2020, 9:52am UTC](https://discuss.huggingface.co/t/sentiment-analysis-for-long-sequences/516 "2020-07-29T09:52:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![adamh](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@adamh](https://discuss.huggingface.co/u/adamh)\
**Post date:** [July 29, 2020, 9:52am UTC](https://discuss.huggingface.co/t/sentiment-analysis-for-long-sequences/516/1 "2020-07-29T09:52:45Z")

</div>

I’m trying to use the sentiment analysis pipeline,. currently using:

```python
nlp = pipeline('sentiment-analysis')
nlp.tokenizer = transformers.DistilBertTokenizer.from_pretrained("distilbert-base-uncased")

```

to classify a large corpus of textual data. Most of my data points are \< 512 tokens, and the pipeline seems to be working well. However, some data points (~15% of the whole datasets) have more than 512 tokens. I’ve tried to split them into chunks of size 512 and aggregate the results, but this didn’t seem to work very well. Is there any principled/recommended approach for such situations? Perhaps using a different model/tokenizer? I’ve tried using XLNet but didn’t get very good results…

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [July 29, 2020, 12:25pm UTC](https://discuss.huggingface.co/t/sentiment-analysis-for-long-sequences/516/2 "2020-07-29T12:25:10Z")

</div>

Hi @adamh, if your context is really long then you can consider using the [longformer](https://huggingface.co/transformers/model_doc/longformer.html) model, it allows to use sequences with upto 4096 tokens. But you’ll need to fine-tune the model first.

---

<div class="post-metadata">

**Author:** ![adamh](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@adamh](https://discuss.huggingface.co/u/adamh)\
**Post date:** [July 29, 2020, 1:21pm UTC](https://discuss.huggingface.co/t/sentiment-analysis-for-long-sequences/516/3 "2020-07-29T13:21:56Z")

</div>

Thanks! Most of the data points that have more than 512 tokens are between 500 and 1000, but it can get up to 2000 or so (which is more “document sentiment analysis” than a “sentence sentiment analysis”).

Regarding Longformer - is there reasonable way to asses how many examples would be needed for fine tuning?

---

<div class="post-metadata">

**Author:** ![t-lochhead](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/t-lochhead/32/794_2.png) [@t-lochhead](https://discuss.huggingface.co/u/t-lochhead)\
**Post date:** [December 7, 2020, 5:46pm UTC](https://discuss.huggingface.co/t/sentiment-analysis-for-long-sequences/516/4 "2020-12-07T17:46:49Z")

</div>

@adamh - did you find a viable solution? I am looking to solve this myself.