# Getting embeddings from wav2vec2 models

**URL:** <https://discuss.huggingface.co/t/getting-embeddings-from-wav2vec2-models/10139>\
**Category:** Beginners\
**Created:** [September 20, 2021, 3:05pm UTC](https://discuss.huggingface.co/t/getting-embeddings-from-wav2vec2-models/10139 "2021-09-20T15:05:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![synapse](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/synapse/32/27883_2.png) [@synapse](https://discuss.huggingface.co/u/synapse)\
**Post date:** [September 20, 2021, 3:05pm UTC](https://discuss.huggingface.co/t/getting-embeddings-from-wav2vec2-models/10139/1 "2021-09-20T15:05:54Z")

</div>

Hi, there!

I am trying to get the embeddings from pre-trained wav2vec2 models (e.g., from jonatasgrosman/wav2vec2-large-xlsr-53-german) using my own dataset.

My aim is to use these features for a downstream task (not specifically speech recognition). Namely, since the dataset is relatively small, I would train an SVM with these embeddings for the final classification.

So far I have tried this:

```auto
model_name = "facebook/wav2vec2-large-xlsr-53-german"
feature_extractor = Wav2Vec2Processor.from_pretrained(model_name)
model = Wav2Vec2Model.from_pretrained(model_name)

input_values = feature_extractor(train_dataset[:10]["speech"], return_tensors="pt", padding=True, 
                                 feature_size=1, sampling_rate=16000 ).input_values 

```

Then, I am not sure whether the embeddings here correspond to the sequence of last\_hidden\_states:

```auto
hidden_states = model(input_values).last_hidden_state

```

or to the sequence of features of the last conv layer of the model:

```auto
features_last_cnn_layer = model(input_values).extract_features

```

Also, is this the correct way to extract features from a pre-trained model?  
How one can get embeddings from a specific layer?

---

<div class="post-metadata">

**Author:** ![sriniu](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sriniu/32/14806_2.png) [@sriniu](https://discuss.huggingface.co/u/sriniu)\
**Post date:** [March 28, 2023, 8:58am UTC](https://discuss.huggingface.co/t/getting-embeddings-from-wav2vec2-models/10139/2 "2023-03-28T08:58:57Z")

</div>

> [@synapse](#):
>
> `.extract_features`

Hi, Thank you for posting this question. Were you successful to get the hidden states or embeddings from Wav2Vec2 model?

---

<div class="post-metadata">

**Author:** ![synapse](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/synapse/32/27883_2.png) [@synapse](https://discuss.huggingface.co/u/synapse)\
**Post date:** [October 20, 2023, 9:27am UTC](https://discuss.huggingface.co/t/getting-embeddings-from-wav2vec2-models/10139/3 "2023-10-20T09:27:22Z")

</div>

Hi, yes. The way I described in the post was indeed the correct way for getting the embeddings.