# Using trasnsformer to get image features

**URL:** <https://discuss.huggingface.co/t/using-trasnsformer-to-get-image-features/17720>\
**Category:** 🤗Transformers\
**Created:** [May 9, 2022, 5:34pm UTC](https://discuss.huggingface.co/t/using-trasnsformer-to-get-image-features/17720 "2022-05-09T17:34:17Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![marcomameli01](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@marcomameli01](https://discuss.huggingface.co/u/marcomameli01)\
**Post date:** [May 9, 2022, 5:34pm UTC](https://discuss.huggingface.co/t/using-trasnsformer-to-get-image-features/17720/1 "2022-05-09T17:34:17Z")

</div>

Hello to everybody,  
I would like to use one transformers tro extract features from images at different level, as for example it is possible to do with convolutional network that we can take the ouput from convolutional layer at level 3 or 4 or 5.  
Can be a possibility to do something like that with tranformers?

Or, and that is more interesting, how I can get the features with the positional embending from the tranformers and use them as input in other kind of network with attention.

---

<div class="post-metadata">

**Author:** ![prithivida](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/prithivida/32/20872_2.png) [@prithivida](https://discuss.huggingface.co/u/prithivida)\
**Post date:** [May 9, 2022, 6:17pm UTC](https://discuss.huggingface.co/t/using-trasnsformer-to-get-image-features/17720/2 "2022-05-09T18:17:10Z")

</div>

Sure it can be done (for the 1st question), To extract features use the bare model, for instance, if we are using ViT the naming convention for the bare model is ViTModel & by default \*most models returns `last_hidden_state` (last layer) and `pooler_output`. To get all layers set `output_hidden_states=True` (line 10) in the forward pass. Now you can access all the layers, you can play with them with the index.

Consider this code

```auto
1. from transformers import ViTFeatureExtractor, ViTModel
2. import torch
3. from datasets import load_dataset

4. dataset = load_dataset("huggingface/cats-image")
5. image = dataset["test"]["image"][0]

6. feature_extractor = ViTFeatureExtractor.from_pretrained("google/vit-base-patch16-224-in21k")
7. model = ViTModel.from_pretrained("google/vit-base-patch16-224-in21k")

8. inputs = feature_extractor(image, return_tensors="pt")

9. with torch.no_grad():
10. outputs = model(**inputs, output_hidden_states=True)

```

---

<div class="post-metadata">

**Author:** ![marcomameli01](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@marcomameli01](https://discuss.huggingface.co/u/marcomameli01)\
**Post date:** [May 15, 2022, 10:16am UTC](https://discuss.huggingface.co/t/using-trasnsformer-to-get-image-features/17720/3 "2022-05-15T10:16:32Z")

</div>

I try this solution but when I pass an image as input which is a 3-channels image in batch of dimension 1 but I receive the error: keyError: ((1, 1, 224, 224), ‘|u1’) and I don’t find where the image become 1 channel

---

<div class="post-metadata">

**Author:** ![mae338](https://avatars.discourse-cdn.com/v4/letter/m/848f3c/32.png) [@mae338](https://discuss.huggingface.co/u/mae338)\
**Post date:** [March 20, 2024, 3:04pm UTC](https://discuss.huggingface.co/t/using-trasnsformer-to-get-image-features/17720/4 "2024-03-20T15:04:44Z")

</div>

> [@prithivida](#):
>
> `output_hidden_states`

Thank you for this valuable information. I just have a question. I tried it and I got a tuple comprising 13 tensors. Each tensor shape is 1x12x197x768. My question is: How can I find the class token feature vector?