# Speech recognition processing max\_length **URL:** \ **Category:** Beginners\ **Created:** [October 19, 2024, 11:31am UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799 "2024-10-19T11:31:12Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![goat123](https://avatars.discourse-cdn.com/v4/letter/g/bc8723/32.png) [@goat123](https://discuss.huggingface.co/u/goat123)\ **Post date:** [October 19, 2024, 11:31am UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/1 "2024-10-19T11:31:12Z")
Hello. I am working on an automatic speech recognition project. But most of the models say that the maximum processing time is 30 seconds. If the original data is longer than 30 seconds, I would like to know how to handle it. For example, is there a way to handle it in code other than manually dividing the original data? ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 19, 2024, 12:18pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/2 "2024-10-19T12:18:19Z")
This might have something to do with it. > [@Whisper on long audio files -- support for chunking?](https://discuss.huggingface.co/t/whisper-on-long-audio-files-support-for-chunking/24682): > > The transformer library supports chunking (concatenation of multiple segments) for transcribing long audio files with Wav2Vec2, as described here: [Making automatic speech recognition work on large files with Wav2Vec2 in hugs Transformers](https://huggingface.co/blog/asr-chunking) The OpenAI repository contains code for chunking with Whisper: [whisper/transcribe.py at main · openai/whisper · GitHub](https://github.com/openai/whisper/blob/main/whisper/transcribe.py) Is chunking with Whisper supported in the transformers library? > **[openai/whisper-large-v3 · Hugging Face](https://huggingface.co/openai/whisper-large-v3#chunked-long-form)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ---
**Author:** ![goat123](https://avatars.discourse-cdn.com/v4/letter/g/bc8723/32.png) [@goat123](https://discuss.huggingface.co/u/goat123)\ **Post date:** [October 19, 2024, 12:38pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/3 "2024-10-19T12:38:27Z")
Thanks for pointing out a good way. But I was referring to the preprocessing of the dataset before fine-tuning. So I was wondering how to divide the data with text and audio into answer chunks, if possible. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 19, 2024, 12:42pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/4 "2024-10-19T12:42:18Z")
Would it be a pre-processing thing? > **[Preprocessing an audio dataset - Hugging Face Audio Course](https://huggingface.co/learn/audio-course/chapter1/preprocessing#pre-processing-audio-data)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. > **[A Complete Guide to Audio Datasets](https://huggingface.co/blog/audio-datasets)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ---
**Author:** ![goat123](https://avatars.discourse-cdn.com/v4/letter/g/bc8723/32.png) [@goat123](https://discuss.huggingface.co/u/goat123)\ **Post date:** [October 19, 2024, 1:13pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/5 "2024-10-19T13:13:02Z")
Thanks for the great resources. These are correct, but they are things I’ve referenced in the past. According to the tutorial, 30 seconds is the maximum. I can’t use the data after 30 seconds because it returns false. It says to discard anything over 30 seconds. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 19, 2024, 2:22pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/6 "2024-10-19T14:22:20Z")
Would you simply split the file? > **[audio-splitter](https://pypi.org/project/audio-splitter/)** > > A CLI tool for splitting audio files into equally-sized chunks or based on periods of silence ---
**Author:** ![goat123](https://avatars.discourse-cdn.com/v4/letter/g/bc8723/32.png) [@goat123](https://discuss.huggingface.co/u/goat123)\ **Post date:** [October 19, 2024, 2:39pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/7 "2024-10-19T14:39:20Z")
That’s a nice package. So I guess the only way to divide text into chunks is to do it myself? ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 19, 2024, 2:44pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/8 "2024-10-19T14:44:13Z")
I think it would be best if there was an integrated dataset processing library, but if the current one doesn’t support it… Even though sound and text are bundled together, they are completely different information from the program’s point of view, so we need a program that ties the two together and finds the split point. We should have to do SR at the stage of creating the data set. It’s hard to do this manually. I wonder if anyone has made one… Edit: It’s close, but there’s no function for text splitting. > **[GitHub - davidmartinrius/speech-dataset-generator: 🔊 Create labeled datasets, enhance audio quality,...](https://github.com/davidmartinrius/speech-dataset-generator)** > > 🔊 Create labeled datasets, enhance audio quality, identify speakers, support diverse dataset types. 🎧👥📊 Advanced audio processing. Other famous libraries when doing it manually > **[Preprocessing the Audio Dataset - GeeksforGeeks](https://www.geeksforgeeks.org/preprocessing-the-audio-dataset/)** > > A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 19, 2024, 3:22pm UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/9 "2024-10-19T15:22:12Z")
Isn’t this one close? > **[GitHub - bluesignum/Audio-SentenceSplit: With one whole audio and corresponding text, the...](https://github.com/bluesignum/Audio-SentenceSplit)** > > With one whole audio and corresponding text, the audio can be split line by line and saved with exact sentence using comparison with the data from Google Speech Recognition API ---
**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\ **Post date:** [October 20, 2024, 3:23am UTC](https://discuss.huggingface.co/t/speech-recognition-processing-max-length/112799/10 "2024-10-20T03:23:01Z")
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.