# Token classification for a non-textual data

**URL:** <https://discuss.huggingface.co/t/token-classification-for-a-non-textual-data/33063>\
**Category:** 🤗Transformers\
**Created:** [March 5, 2023, 9:13am UTC](https://discuss.huggingface.co/t/token-classification-for-a-non-textual-data/33063 "2023-03-05T09:13:03Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![vitvit](https://avatars.discourse-cdn.com/v4/letter/v/b5e925/32.png) [@vitvit](https://discuss.huggingface.co/u/vitvit)\
**Post date:** [March 5, 2023, 9:13am UTC](https://discuss.huggingface.co/t/token-classification-for-a-non-textual-data/33063/1 "2023-03-05T09:13:03Z")

</div>

I’m looking for an implementation of an architecture that performs token classification, but the input is not an integer that represents the vocabulary but a vector of numbers.  
Basically, each token in the input is represented by a vector. Each token is already an embeddings vector.  
How can this be achieved?

### Expected behavior

Input vector of size 768 for each token. A sequence of such tokens of up to 512.  
Maybe it is as simple as removing the layer  
(word\_embeddings): Embedding(50265, 768, padding\_idx=1)?  
In any case a link to the solution would be most helpful.