# Longformer for sequenceclassification

**URL:** <https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496>\
**Category:** 🤗Transformers\
**Created:** [October 12, 2020, 7:00pm UTC](https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496 "2020-10-12T19:00:40Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\
**Post date:** [October 12, 2020, 7:00pm UTC](https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496/1 "2020-10-12T19:00:40Z")

</div>

I am using the Longformer for sequence classification task (taking example code from here), the model is running somehow on a small batch, the issue is with the wrong predictions, is there any example of the Longformer sequence classification for GLUE tasks.  
thanks

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [October 13, 2020, 7:20am UTC](https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496/2 "2020-10-13T07:20:31Z")

</div>

try the example script [here](https://github.com/huggingface/transformers/tree/master/examples/text-classification)

---

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\
**Post date:** [October 13, 2020, 10:28am UTC](https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496/3 "2020-10-13T10:28:09Z")

</div>

thanks, so instead of --model\_name\_or\_path bert-base-multilingual-uncased , I could use longformer?

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [October 13, 2020, 10:49am UTC](https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496/4 "2020-10-13T10:49:25Z")

</div>

Yes, all you’ll need to do is change the model name!

---

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\
**Post date:** [October 13, 2020, 12:34pm UTC](https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496/5 "2020-10-13T12:34:49Z")

</div>

thanks @valhalla appreciate yours response everytime

---

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\
**Post date:** [October 13, 2020, 4:59pm UTC](https://discuss.huggingface.co/t/longformer-for-sequenceclassification/1496/6 "2020-10-13T16:59:33Z")

</div>

can you check this notebook quickly, where am I going wrong, the predictions are too poor, I just take a sample of dataset during fine-tuning because of performance issue

> <https://github.com/shainaraza/BERT/blob/master/Text_Classification_longformer_1.ipynb>