# Multi-class Classification Basics

**URL:** <https://discuss.huggingface.co/t/multi-class-classification-basics/9411>\
**Category:** Beginners\
**Created:** [August 23, 2021, 3:07pm UTC](https://discuss.huggingface.co/t/multi-class-classification-basics/9411 "2021-08-23T15:07:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![marlon89](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@marlon89](https://discuss.huggingface.co/u/marlon89)\
**Post date:** [August 23, 2021, 3:07pm UTC](https://discuss.huggingface.co/t/multi-class-classification-basics/9411/1 "2021-08-23T15:07:38Z")

</div>

Hello,

I got a really basic question on the whole BERT/finetune BERT for classification topic:

I got a dataset with customer reviews which consists of 7 different labels such as “Customer Service”, “Tariff”, “Provider related” etc. .

My dataset contains 12700 not labelled customer reviews and I labelled 1100 reviews for my classification task. Now to my questions:

- Could it be enough to take an existing BERT model and fine-tune it with AutoModelForSequenceClassification on my specific task?

- Are the 1100 labelled reviews (around 150 per each class) enough to train that?

- What are other approaches?

I am completely new to the whole NLP task and working on it for 3 weeks. I just understood that more traditional approaches are often outperformed by transformers…

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [August 23, 2021, 7:02pm UTC](https://discuss.huggingface.co/t/multi-class-classification-basics/9411/2 "2021-08-23T19:02:17Z")

</div>

> [@marlon89](#):
>
> Could it be enough to take an existing BERT model and fine-tune it with AutoModelForSequenceClassification on my specific task?

Yes. You can use “bert-base-uncased” for example.

> [@marlon89](#):
>
> - Are the 1100 labelled reviews (around 150 per each class) enough to train that?

Yes, that’s the power of transfer learning: we only need a few (like a hundred) examples per class.

> [@marlon89](#):
>
> What are other approaches?

Actually, that’s the most typical approach: fine-tune an `xxxForSequenceClassification` model using a pre-trained base. You can check out the official tutorial here: [https://github.com/huggingface/notebooks/blob/master/examples/text\_classification.ipynb](https://github.com/huggingface/notebooks/blob/master/examples/text_classification.ipynb)

---

<div class="post-metadata">

**Author:** ![marlon89](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@marlon89](https://discuss.huggingface.co/u/marlon89)\
**Post date:** [August 24, 2021, 8:28am UTC](https://discuss.huggingface.co/t/multi-class-classification-basics/9411/3 "2021-08-24T08:28:44Z")

</div>

Thank you so much for your reply and helping me to go on. For now it looks like I have to use MultiLabel Classification instead of MultiClass…does it make any difference in the way I set up the transformer?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [August 24, 2021, 8:40am UTC](https://discuss.huggingface.co/t/multi-class-classification-basics/9411/4 "2021-08-24T08:40:16Z")

</div>

> [@marlon89](#):
>
> MultiLabel

If a given review can have more than 1 label, then it’s a multi-label text classification problem indeed.

> [@marlon89](#):
>
> does it make any difference in the way I set up the transformer?

The only thing you’ll need to change is setting the `problem_type` to `multi_label_classification` when instantiating an `xxxForSequenceClassification` model. Suppose that we have 7 different labels and we want to do multi-label classification, then you can for example instantiate a BERT model as follows:

```auto
from transformers import BertForSequenceClassification

model = BertForSequenceClassification.from_pretrained("bert-base-uncased", problem_type="multi_label_classification", num_labels=7)

```

---

<div class="post-metadata">

**Author:** ![marlon89](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@marlon89](https://discuss.huggingface.co/u/marlon89)\
**Post date:** [August 24, 2021, 8:42am UTC](https://discuss.huggingface.co/t/multi-class-classification-basics/9411/5 "2021-08-24T08:42:32Z")

</div>

Thank you so much for getting back that fast!