# Labels in Audio Frame classification task (Wav2Vec2 For Audio Frame Classification)

**URL:** <https://discuss.huggingface.co/t/labels-in-audio-frame-classification-task-wav2vec2-for-audio-frame-classification/49824>\
**Category:** 🤗Transformers\
**Created:** [August 7, 2023, 1:10pm UTC](https://discuss.huggingface.co/t/labels-in-audio-frame-classification-task-wav2vec2-for-audio-frame-classification/49824 "2023-08-07T13:10:57Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![maureendss](https://avatars.discourse-cdn.com/v4/letter/m/b5ac83/32.png) [@maureendss](https://discuss.huggingface.co/u/maureendss)\
**Post date:** [August 7, 2023, 1:10pm UTC](https://discuss.huggingface.co/t/labels-in-audio-frame-classification-task-wav2vec2-for-audio-frame-classification/49824/1 "2023-08-07T13:10:57Z")

</div>

I’m working on fine-tuning a speech model (either Wav2Vec or HuBERT) to classify speech at the frame level (e.g., every 20ms of audio must be classified). Specifically, I’m looking to use the `Wav2Vec2ForAudioFrameClassification` method, but I’m uncertain about the shape and format of the labels required.

For this task, I’d like to input a torch tensor with a length corresponding to the audio length divided by the frame length (e.g., 20ms), containing binary values of 0s or 1s. Is this approach feasible? I’m having difficulty understanding the precise requirements for the labels.

Does anyone have an example of using `Wav2Vec2ForAudioFrameClassification`, or can someone guide me on the correct shape and formatting for the labels? Any help would be appreciated, thanks a lot!

---

<div class="post-metadata">

**Author:** ![bnestor](https://avatars.discourse-cdn.com/v4/letter/b/e68b1a/32.png) [@bnestor](https://discuss.huggingface.co/u/bnestor)\
**Post date:** [January 7, 2025, 3:12am UTC](https://discuss.huggingface.co/t/labels-in-audio-frame-classification-task-wav2vec2-for-audio-frame-classification/49824/2 "2025-01-07T03:12:29Z")

</div>

The final dimension of your labels when feeding them into the model should be shape `[batch_size, sequence_length*num_classes]`. Internally, the model is calling `torch.argmax(labels.view(-1, self.num_classes), axis=1).long()`

The snippet below will work for a binary classification problem.

```auto
check_labels = labels.long() # copy before modification.
labels = torch.nn.functional.one_hot(labels.long(), num_classes=2)

labels = labels.view(labels.shape[0], labels.shape[1]*labels.shape[2])

# this is the line used in the model.
num_classes=2
for i in range(len(check_labels)):
    check_against=labels[i]
    check_against = torch.argmax(check_against.view(-1, num_classes), axis=1).long()
    torch.testing.assert_close(check_labels[i], check_against, rtol=1e-5, atol=1e-5)

```