# Apply BertForTokenClassification on partially labeled input

**URL:** <https://discuss.huggingface.co/t/apply-bertfortokenclassification-on-partially-labeled-input/11875>\
**Category:** 🤗Transformers\
**Created:** [November 16, 2021, 2:05pm UTC](https://discuss.huggingface.co/t/apply-bertfortokenclassification-on-partially-labeled-input/11875 "2021-11-16T14:05:24Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![JakobDerPharao](https://avatars.discourse-cdn.com/v4/letter/j/bbce88/32.png) [@JakobDerPharao](https://discuss.huggingface.co/u/JakobDerPharao)\
**Post date:** [November 16, 2021, 2:05pm UTC](https://discuss.huggingface.co/t/apply-bertfortokenclassification-on-partially-labeled-input/11875/1 "2021-11-16T14:05:24Z")

</div>

I have a corpus where only some tokens (some adjective + noun paurs) are labeled for a specific task.

How do I use the BertForTokenClassification class to only calculate my updates on those labeled tokens and not on every single token? If I am correct I can’t unmask them in the attention\_mask because that would lead to the context being ignored. I also don’t see a way where just labeling every unlabeled token as 0 for example makes sense, since not all adjective + noun pairs are labeled.

Thanks for any help in advance.