# Trying to understand XForSequenceClassification heads

**URL:** <https://discuss.huggingface.co/t/trying-to-understand-xforsequenceclassification-heads/1153>\
**Category:** Intermediate\
**Created:** [September 15, 2020, 11:03pm UTC](https://discuss.huggingface.co/t/trying-to-understand-xforsequenceclassification-heads/1153 "2020-09-15T23:03:01Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](https://discuss.huggingface.co/u/BramVanroy)\
**Post date:** [September 23, 2020, 2:22pm UTC](https://discuss.huggingface.co/t/trying-to-understand-xforsequenceclassification-heads/1153/5 "2020-09-23T14:22:11Z")

</div>

When answering [this question](https://discuss.huggingface.co/t/xlnetforsqeuenceclassification-warnings/1096/13), I found that BertForSequenceClassification’s doesn’t actually use the pretrained linear weights for the classification layer. I kinda had expected that it did. It probably would not be useful for most tasks and needs finetuning anyway, but still.

Maybe the XForX models can have a line in their docs stating which of its heads are pretrained and which ones aren’t.

---

_[View the full topic](https://discuss.huggingface.co/t/trying-to-understand-xforsequenceclassification-heads/1153)._