# Swin transformer hidden states( feature map) different

**URL:** <https://discuss.huggingface.co/t/swin-transformer-hidden-states-feature-map-different/18096>\
**Category:** 🤗Transformers\
**Created:** [May 18, 2022, 10:52am UTC](https://discuss.huggingface.co/t/swin-transformer-hidden-states-feature-map-different/18096 "2022-05-18T10:52:40Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dreamer312](https://avatars.discourse-cdn.com/v4/letter/d/e0b2c6/32.png) [@Dreamer312](https://discuss.huggingface.co/u/Dreamer312)\
**Post date:** [May 18, 2022, 10:52am UTC](https://discuss.huggingface.co/t/swin-transformer-hidden-states-feature-map-different/18096/1 "2022-05-18T10:52:40Z")

</div>

```auto
from transformers import AutoFeatureExtractor, SwinModel
import torch
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = AutoFeatureExtractor.from_pretrained("microsoft/swin-base-patch4-window7-224")
model = SwinModel.from_pretrained("microsoft/swin-base-patch4-window7-224", output_hidden_states=True)

inputs = feature_extractor(image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

last_hidden_states = outputs.last_hidden_state
hidden_states = outputs.hidden_states

print(hidden_states[-1].equal(last_hidden_states))
False

```

Hi, I am using a Swin transformer to extract features or hidden states in NLP,  
I think the last\_hidden\_state should be same as the hidden\_state[-1], while it outputs False. Is there anything I missed?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [November 3, 2022, 9:02am UTC](https://discuss.huggingface.co/t/swin-transformer-hidden-states-feature-map-different/18096/2 "2022-11-03T09:02:40Z")

</div>

Hi,

In this case you haven’t specified `output_hidden_states=True` in the forward of the model. Hence `output.hidden_states` will be None.