# Always only a single Linear layer as the classification head?

**URL:** <https://discuss.huggingface.co/t/always-only-a-single-linear-layer-as-the-classification-head/32324>\
**Category:** 🤗Transformers\
**Created:** [February 23, 2023, 5:24am UTC](https://discuss.huggingface.co/t/always-only-a-single-linear-layer-as-the-classification-head/32324 "2023-02-23T05:24:07Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![incameet](https://avatars.discourse-cdn.com/v4/letter/i/c57346/32.png) [@incameet](https://discuss.huggingface.co/u/incameet)\
**Post date:** [February 23, 2023, 5:24am UTC](https://discuss.huggingface.co/t/always-only-a-single-linear-layer-as-the-classification-head/32324/1 "2023-02-23T05:24:07Z")

</div>

For example, in the class [BertForSequenceClassification definition](https://github.com/huggingface/transformers/blob/v4.26.1/src/transformers/models/bert/modeling_bert.py#L1518), only one Linear layer is used as the classifier. If just one Linear layer is used, doesn’t it just do linear project for pooled\_out? Will such a classifier produce good predictions? Why not use multiple Linear layers? Does transformers offer any option for using multiple Linear layers as the classification head?