# How was LlamaForSequenceClassification Pretrained

**URL:** <https://discuss.huggingface.co/t/how-was-llamaforsequenceclassification-pretrained/46877>\
**Category:** 🤗Transformers\
**Created:** [July 15, 2023, 12:51am UTC](https://discuss.huggingface.co/t/how-was-llamaforsequenceclassification-pretrained/46877 "2023-07-15T00:51:25Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![zhanwenchen](https://avatars.discourse-cdn.com/v4/letter/z/a6a055/32.png) [@zhanwenchen](https://discuss.huggingface.co/u/zhanwenchen)\
**Post date:** [July 15, 2023, 12:51am UTC](https://discuss.huggingface.co/t/how-was-llamaforsequenceclassification-pretrained/46877/1 "2023-07-15T00:51:25Z")

</div>

Hello!

We are relatively new to HuggingFace, and we are trying to access some form of document (not token) embedding for LLaMA or other LLMs. We noticed that `LlamaForSequenceClassification` has two outputs and uses the first token to represent hidden states (so perhaps the document embedding we are looking for?).

We could not find on which task LlamaForSequenceClassification was fine-tuned anywhere. We’d appreciate if you can point us in the right direction.