# Dataset loading script for an audio dataset

**URL:** <https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332>\
**Category:** 🤗Datasets\
**Created:** [August 31, 2022, 6:14am UTC](https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332 "2022-08-31T06:14:04Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![comodoro](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/comodoro/32/20251_2.png) [@comodoro](https://discuss.huggingface.co/u/comodoro)\
**Post date:** [August 31, 2022, 6:14am UTC](https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332/1 "2022-08-31T06:14:04Z")

</div>

I want to fix all metadata and loading for my audio datasets (like `comodoro/vystadial2016_asr`), but cannot find specific documentation for audio datasets, in particular how to load into an `Audio` column.

Looked at `librispeech_asr/blob/main/librispeech_asr.py`, but audio data there is only referenced in feature metadata and `_generate_examples`. I tried to emulate it at least, but I got stuck at [these lines](https://huggingface.co/datasets/comodoro/vystadial2016_asr/blob/main/vystadial2016_asr.py#L93) with `[Errno 2] No such file or directory: 'data_voip_cs_2016'`; I do not know what exactly the download manager returns and how to access it, not even looking at the [source](https://huggingface.co/docs/datasets/v1.0.1/_modules/datasets/utils/download_manager.html#DownloadManager).

Several questions:

- Is there any more documentation?
- Can I debug the script locally instead of adding a commit for every fix attempt?
- I would also like to add a loading script for some json (HF hosted) audio datasets like datasets/comodoro/pscr, is that even possible?

---

<div class="post-metadata">

**Author:** ![comodoro](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/comodoro/32/20251_2.png) [@comodoro](https://discuss.huggingface.co/u/comodoro)\
**Post date:** [September 1, 2022, 6:21am UTC](https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332/2 "2022-09-01T06:21:43Z")

</div>

AD 2, datases-cli, I suppose

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [September 1, 2022, 1:40pm UTC](https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332/3 "2022-09-01T13:40:01Z")

</div>

Maybe @polinaeterna ?

---

<div class="post-metadata">

**Author:** ![comodoro](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/comodoro/32/20251_2.png) [@comodoro](https://discuss.huggingface.co/u/comodoro)\
**Post date:** [September 1, 2022, 2:05pm UTC](https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332/4 "2022-09-01T14:05:36Z")

</div>

I have made some progress (DownloadManager returns different type for multiple and single downloads; not strictly speaking an error, but confusing). Now `The split is being processed. Retry later.` for two hours. Something is still wrong.

---

<div class="post-metadata">

**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\
**Post date:** [September 2, 2022, 9:37am UTC](https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332/5 "2022-09-02T09:37:44Z")

</div>

Hey @comodoro there’s a new `audiofolder` feature that let’s you load audio datasets with metadata: [Load audio data](https://huggingface.co/docs/datasets/main/en/audio_load#audiofolder)

You’ll need to install `datasets` from `main` to make use of it 🙂

---

<div class="post-metadata">

**Author:** ![comodoro](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/comodoro/32/20251_2.png) [@comodoro](https://discuss.huggingface.co/u/comodoro)\
**Post date:** [September 2, 2022, 10:23am UTC](https://discuss.huggingface.co/t/dataset-loading-script-for-an-audio-dataset/22332/6 "2022-09-02T10:23:06Z")

</div>

Thanks, I will try it with the next ones. In the meantime, [succes!](https://huggingface.co/datasets/comodoro/vystadial2016_asr). Last few errors were caused by me not following the dataset structure, different from Librispeech. Frankly if not for my strong determination, I would have given up a long time ago. Lessons learned:

- ALWAYS ensure `datasets-cli` runs OK before making a commit
- `generate_examples` indeed generates all samples, it is just badly named.
