# Multi class text classification tutorial: how does he get away with one out\_feature on linear layer?

**URL:** <https://discuss.huggingface.co/t/multi-class-text-classification-tutorial-how-does-he-get-away-with-one-out-feature-on-linear-layer/630>\
**Category:** Beginners\
**Created:** [August 7, 2020, 11:14pm UTC](https://discuss.huggingface.co/t/multi-class-text-classification-tutorial-how-does-he-get-away-with-one-out-feature-on-linear-layer/630 "2020-08-07T23:14:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dickdanieljr](https://avatars.discourse-cdn.com/v4/letter/d/96bed5/32.png) [@dickdanieljr](https://discuss.huggingface.co/u/dickdanieljr)\
**Post date:** [August 7, 2020, 11:14pm UTC](https://discuss.huggingface.co/t/multi-class-text-classification-tutorial-how-does-he-get-away-with-one-out-feature-on-linear-layer/630/1 "2020-08-07T23:14:18Z")

</div>

I’ve read this tutorial: [https://github.com/abhimishra91/transformers-tutorials/blob/master/transformers\_multiclass\_classification.ipynb](https://github.com/abhimishra91/transformers-tutorials/blob/master/transformers_multiclass_classification.ipynb)

As I see it, the dataset uses 4 labels. I thought that that would imply having 4 out\_features on your last linear layer. But when I check his model, he uses just 1.

Please help me to fix my misunderstanding 🙂

---

<div class="post-metadata">

**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\
**Post date:** [January 22, 2021, 7:56pm UTC](https://discuss.huggingface.co/t/multi-class-text-classification-tutorial-how-does-he-get-away-with-one-out-feature-on-linear-layer/630/2 "2021-01-22T19:56:05Z")

</div>

Hi @dickdanieljr, are you referring to this class?

```auto
class DistillBERTClass(torch.nn.Module):
    def __init__ (self):
        super(DistillBERTClass, self). __init__ ()
        self.l1 = DistilBertModel.from_pretrained("distilbert-base-uncased")
        self.pre_classifier = torch.nn.Linear(768, 768)
        self.dropout = torch.nn.Dropout(0.3)
        self.classifier = torch.nn.Linear(768, 4)

    def forward(self, input_ids, attention_mask):
        output_1 = self.l1(input_ids=input_ids, attention_mask=attention_mask)
        hidden_state = output_1[0]
        pooler = hidden_state[:, 0]
        pooler = self.pre_classifier(pooler)
        pooler = torch.nn.ReLU()(pooler)
        pooler = self.dropout(pooler)
        output = self.classifier(pooler)
        return output

```

Here you can see that the last layer is `torch.nn.Linear` of shape `[hidden_dim, num_labels]` - or am I missing something?

---

<div class="post-metadata">

**Author:** ![dickdanieljr](https://avatars.discourse-cdn.com/v4/letter/d/96bed5/32.png) [@dickdanieljr](https://discuss.huggingface.co/u/dickdanieljr)\
**Post date:** [January 22, 2021, 10:19pm UTC](https://discuss.huggingface.co/t/multi-class-text-classification-tutorial-how-does-he-get-away-with-one-out-feature-on-linear-layer/630/3 "2021-01-22T22:19:09Z")

</div>

Yeah, that looks ok. I don’t remember the issue I had after such a long time. I described it very poorly. Thank you for the message though!