# Image Features as Model Input

**URL:** <https://discuss.huggingface.co/t/image-features-as-model-input/782>\
**Category:** Beginners\
**Created:** [August 20, 2020, 6:05am UTC](https://discuss.huggingface.co/t/image-features-as-model-input/782 "2020-08-20T06:05:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![CMPE-PL](https://avatars.discourse-cdn.com/v4/letter/c/8797f3/32.png) [@CMPE-PL](https://discuss.huggingface.co/u/CMPE-PL)\
**Post date:** [August 20, 2020, 6:05am UTC](https://discuss.huggingface.co/t/image-features-as-model-input/782/1 "2020-08-20T06:05:37Z")

</div>

Hello All. I apologize for the basic question but for some reason I am having difficulty using image features as input to a huggingface model.

My data comes in the form of an image feature numpy array extracted by a 2D CNN but all the models seem to be built for text based input.

If anybody could point me in the right direction of an example or a code snippet I would greatly appreciate it!

---

<div class="post-metadata">

**Author:** ![rgwatwormhill](https://avatars.discourse-cdn.com/v4/letter/r/a9adbd/32.png) [@rgwatwormhill](https://discuss.huggingface.co/u/rgwatwormhill)\
**Post date:** [November 18, 2020, 12:43pm UTC](https://discuss.huggingface.co/t/image-features-as-model-input/782/2 "2020-11-18T12:43:05Z")

</div>

Hi CMPE-PL,

the transformers models are designed for Natural Language Processing, ie text. What makes you think they would be good for image features?

I expect you could bypass the tokenizers and input numbers directly, but I’m not sure it would do anything useful. If you did want to do that, you would need to ensure that your numbers were in the right format. For example, a BERT model expects input that is a vector of 768 real numbers for each word-token, or rather a matrix of 768 x N real numbers, where N is number of word-tokens in a text.

What size is your image feature array?

The main trick of transformers is to use Attention mechanisms. It is certainly possible to use Attention in image recognition models, but without using transformers. See this article for an example [https://arxiv.org/abs/2004.13621](https://arxiv.org/abs/2004.13621)

---

<div class="post-metadata">

**Author:** ![rgwatwormhill](https://avatars.discourse-cdn.com/v4/letter/r/a9adbd/32.png) [@rgwatwormhill](https://discuss.huggingface.co/u/rgwatwormhill)\
**Post date:** [November 18, 2020, 12:51pm UTC](https://discuss.huggingface.co/t/image-features-as-model-input/782/3 "2020-11-18T12:51:36Z")

</div>

If you haven’t seen them already, you might find these articles useful.

> **[Attention Is All You Need](https://arxiv.org/abs/1706.03762)**
>
> The dominant sequence transduction models are based on complex recurrent or
> convolutional neural networks in an encoder-decoder configuration. The best
> performing models also connect the encoder and decoder through an attention
> mechanism. We propose...

> **[The Illustrated Transformer](http://jalammar.github.io/illustrated-transformer/)**
>
> Discussions:
> Hacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)
> 
> 
> Translations: Chinese (Simplified), French, Japanese, Korean, Russian, Spanish
> 
> Watch: MIT’s Deep Learning State of the Art lecture referencing this...