# Using finBert for 7-class sequence classification

**URL:** <https://discuss.huggingface.co/t/using-finbert-for-7-class-sequence-classification/9118>\
**Category:** Beginners\
**Created:** [August 10, 2021, 9:10am UTC](https://discuss.huggingface.co/t/using-finbert-for-7-class-sequence-classification/9118 "2021-08-10T09:10:11Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Byskov11](https://avatars.discourse-cdn.com/v4/letter/b/c89c15/32.png) [@Byskov11](https://discuss.huggingface.co/u/Byskov11)\
**Post date:** [August 10, 2021, 9:10am UTC](https://discuss.huggingface.co/t/using-finbert-for-7-class-sequence-classification/9118/1 "2021-08-10T09:10:11Z")

</div>

Hi,

I am a beginner to Hugging Face/Transformers and a question of a more conceptual manner have risen.

My end goal is to use ProsusAI/finbert ([ProsusAI/finbert · Hugging Face](https://huggingface.co/ProsusAI/finbert)) for a Sequence Classification problem with 7 classes.

The two different statements yield a Bert-model with and without header:

```auto
from transformers import AutoModel, AutoModelForSequenceClassification
checkpoint = "ProsusAI/finbert"
model_pretrained = AutoModel.from_pretrained(checkpoint)
model_finetuned = AutoModelForSequenceClassification(checkpoint)

```

The pretrained model is loaded without a header (meaning, the “forward”-method does not include labels) - this is a BertModel. The finetuned model is loaded with the header that includes a pre-defined architecture, which involves only 3 class-prediction - this is a BertModelForClassificaiton.

My question is, how do I create a BertModelForClassification for a 7-class prediction problem using the pre-trained model and weights? Unfortunately, this does not work since the finetuned model comes with pre-specified architecture:

```auto
from transformers import AutoModel, AutoModelForSequenceClassification
checkpoint = "ProsusAI/finbert"
model_finetuned = AutoModelForSequenceClassification(checkpoint, num_labels = 7)

```

The only solution I was able to find myself is to code/modify the output-layer myself using pytorch. But it this the most straight-forward solution?

I hope it makes sense.

/Mathias

---

<div class="post-metadata">

**Author:** ![klauspar](https://avatars.discourse-cdn.com/v4/letter/k/f0a364/32.png) [@klauspar](https://discuss.huggingface.co/u/klauspar)\
**Post date:** [June 8, 2022, 12:34pm UTC](https://discuss.huggingface.co/t/using-finbert-for-7-class-sequence-classification/9118/2 "2022-06-08T12:34:07Z")

</div>

I have the same problem. I set num\_labels=2 but the model still tries to use 3 labels. Don’t know how to fix.