# Wav2vec: how to run decoding with a language model? **URL:** \ **Category:** Beginners\ **Created:** [May 7, 2021, 10:51am UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055 "2021-05-07T10:51:08Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Su-Youn](https://avatars.discourse-cdn.com/v4/letter/s/e8c25b/32.png) [@Su-Youn](https://discuss.huggingface.co/u/Su-Youn)\ **Post date:** [May 7, 2021, 10:51am UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055/1 "2021-05-07T10:51:08Z")
Hello. I am finetuning wav2vec “wav2vec2-large-lv60 “ using my own dataset. I followed Patrick’s tutorial ([Fine-Tune Wav2Vec2 for English ASR in Hugging Face with 🤗 Transformers](https://huggingface.co/blog/fine-tune-wav2vec2-english)) and successfully finished the finetuning (thanks for very nice tutorial.) Now, I would like to run decoding with a language model and have a few questions. 1. Can we run decoding with a language model directly from huggingface? 2. If not, how can I get the wave2vec model compatible to the fairseq decoding script (fairseq/examples/speech\_recognition/infer.py)? I did the following steps, but it failed: 1. Create ‘.pt’ file from the finetuning checkpoint def save\_model(my\_checkpoint\_path): model = Wav2Vec2ForCTC.from\_pretrained(my\_checkpoint\_path) torch.save(model.state\_dict(), my\_model.pt) 2. Decoding I used the decoding step command from the following webpage [https://github.com/pytorch/fairseq/blob/master/examples/wav2vec/README.md#evaluating-a-ctc-model](https://github.com/pytorch/fairseq/blob/master/examples/wav2vec/README.md#evaluating-a-ctc-model) $subset=dev\_other python examples/speech\_recognition/infer.py /checkpoint/abaevski/data/speech/libri/10h/wav2vec/raw --task audio\_pretraining –nbest 1 --path /path/to/model --gen-subset $subset --results-path /path/to/save/results/for/sclite --w2l-decoder kenlm –lm-model /path/to/kenlm.bin --lm-weight 2 --word-score -1 --sil-weight 0 --criterion ctc --labels ltr --max-tokens 4000000 –post-process letter I replaced /path/to/model with “my\_model.pt”. Then, I am getting the following error message. Traceback (most recent call last): File “/mount/fairseq/examples/speech\_recognition/infer.py”, line 427, in cli\_main() File “/mount/fairseq/examples/speech\_recognition/infer.py”, line 423, in cli\_main main(args) File “/mount/fairseq/examples/speech\_recognition/infer.py”, line 229, in main models, saved\_cfg, task = checkpoint\_utils.load\_model\_ensemble\_and\_task( File “/mount/fairseq/fairseq/checkpoint\_utils.py”, line 370, in load\_model\_ensemble\_and\_task state = load\_checkpoint\_to\_cpu(filename, arg\_overrides) File “/mount/fairseq/fairseq/checkpoint\_utils.py”, line 304, in load\_checkpoint\_to\_cpu state = \_upgrade\_state\_dict(state) File “/mount/fairseq/fairseq/checkpoint\_utils.py”, line 456, in \_upgrade\_state\_dict {“criterion\_name”: “CrossEntropyCriterion”, “best\_loss”: state[“best\_loss”]} KeyError: ‘best\_loss’ When I googled it, this seems relevant to removal of the optimization history logs: > > This happens because we remove the useless optimization history logs from the model to reduce the file size. Only the desired model weights are kept to release. As a result, if you directly load the model, error will be reported that some logs are missed. So how can I save the finetuning model compatible to “fairseq”. Should I store the optimization history? If yes, how can I do it? Does anyone have same experience? If yes, could you please share it with me? Thank you always. ---
**Author:** ![Su-Youn](https://avatars.discourse-cdn.com/v4/letter/s/e8c25b/32.png) [@Su-Youn](https://discuss.huggingface.co/u/Su-Youn)\ **Post date:** [May 8, 2021, 2:17am UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055/2 "2021-05-08T02:17:45Z")
Oh, I found the following previous discussion from the forum. Sorry for missing this one. > [@Language model for wav2vec2.0 decoding](https://discuss.huggingface.co/t/language-model-for-wav2vec2-0-decoding/4434): > > Hello, I implemented [wav2vec2.0 code](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2) and a language model is not used for decoding. How can I add a language model (let’s say a language model which is trained with KenLM) for decoding @patrickvonplaten ? thanks in advance. Note: I also opened an [issue](https://github.com/huggingface/transformers/issues/10716), but redirected here. so i will check them out first. Thanks. ---
**Author:** ![qpazuzu](https://avatars.discourse-cdn.com/v4/letter/q/0ea827/32.png) [@qpazuzu](https://discuss.huggingface.co/u/qpazuzu)\ **Post date:** [June 5, 2021, 6:52pm UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055/3 "2021-06-05T18:52:49Z")
Hi @Su-Youn, Did you manage to add LM decoding? ---
**Author:** ![Su-Youn](https://avatars.discourse-cdn.com/v4/letter/s/e8c25b/32.png) [@Su-Youn](https://discuss.huggingface.co/u/Su-Youn)\ **Post date:** [June 6, 2021, 7:32am UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055/4 "2021-06-06T07:32:20Z")
Yes. I used the following code with some updates. > **[GitHub - Edresson/Wav2Vec-Wrapper: An easy way to fine-tune Wav2Vec 2.0 for...](https://github.com/Edresson/Wav2Vec-Wrapper)** > > An easy way to fine-tune Wav2Vec 2.0 for low-resource languages. - GitHub - Edresson/Wav2Vec-Wrapper: An easy way to fine-tune Wav2Vec 2.0 for low-resource languages. ---
**Author:** ![qpazuzu](https://avatars.discourse-cdn.com/v4/letter/q/0ea827/32.png) [@qpazuzu](https://discuss.huggingface.co/u/qpazuzu)\ **Post date:** [June 6, 2021, 5:08pm UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055/5 "2021-06-06T17:08:03Z")
Thank you! I will give this a try 🙂 ---
**Author:** ![patrickvonplaten](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/patrickvonplaten/32/2171_2.png) [@patrickvonplaten](https://discuss.huggingface.co/u/patrickvonplaten)\ **Post date:** [November 4, 2021, 5:28pm UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055/6 "2021-11-04T17:28:18Z")
This could also help: [GitHub - patrickvonplaten/Wav2Vec2\_PyCTCDecode: Small repo describing how to use Hugging Face's Wav2Vec2 with PyCTCDecode](https://github.com/patrickvonplaten/Wav2Vec2_PyCTCDecode) ---
**Author:** ![nihalbaig](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nihalbaig/32/5783_2.png) [@nihalbaig](https://discuss.huggingface.co/u/nihalbaig)\ **Post date:** [August 24, 2022, 8:29am UTC](https://discuss.huggingface.co/t/wav2vec-how-to-run-decoding-with-a-language-model/6055/7 "2022-08-24T08:29:41Z")
Hi @patrickvonplaten , I have trained a model using wave2vec2 and tried to use n-gram with it. But there are errors like input shape is not matching. Also my model didn’t generate any alphabet.json file. I have tried your blog [Boosting Wav2Vec2 with n-grams in 🤗 Transformers](https://huggingface.co/blog/wav2vec2-with-ngram) . But it didn’t worked on my model. Though it worked on some other models. Could you guide me what’s wrong with my model training? Here is my mode: [nihalbaig/wav2vec2-large-xlsr-bn · Hugging Face](https://huggingface.co/nihalbaig/wav2vec2-large-xlsr-bn) .