# Wav2vec with new LM causing CPU OOM

**URL:** <https://discuss.huggingface.co/t/wav2vec-with-new-lm-causing-cpu-oom/14069>\
**Category:** 🤗Transformers\
**Created:** [January 24, 2022, 8:29pm UTC](https://discuss.huggingface.co/t/wav2vec-with-new-lm-causing-cpu-oom/14069 "2022-01-24T20:29:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![othiele](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/othiele/32/19711_2.png) [@othiele](https://discuss.huggingface.co/u/othiele)\
**Post date:** [January 24, 2022, 8:29pm UTC](https://discuss.huggingface.co/t/wav2vec-with-new-lm-causing-cpu-oom/14069/1 "2022-01-24T20:29:35Z")

</div>

Hi @patrickvonplaten and all other wav2vec LM users,  
thanks for the new [LM addition](https://huggingface.co/blog/wav2vec2-with-ngram). When using the following code (also without GPU), the CPU will run out of memory. Looks like something should be freed up but isn’t. This is not happening without the LM, so I guess this has something to do with pyctcdecode. Any ideas or am I missing a step?

This code will trigger the OOM on a GPU. Maybe set the range higher if needed, but memory usage should increase anyways.

```auto
from datasets import load_dataset
from transformers import Wav2Vec2ProcessorWithLM, Wav2Vec2ForCTC
import torch

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
processor = Wav2Vec2ProcessorWithLM.from_pretrained("patrickvonplaten/wav2vec2-base-100h-with-lm")
model = Wav2Vec2ForCTC.from_pretrained("patrickvonplaten/wav2vec2-base-100h-with-lm").to("cuda")
print("* * * * * * loaded models")

for i in range(200):
    audio_sample = dataset[i]
    print(" * * * * Sample: ", i)
    inputs = processor(audio_sample["audio"]["array"], sampling_rate=audio_sample["audio"]["sampling_rate"], return_tensors="pt").to("cuda")
    with torch.no_grad():
      logits = model(**inputs).logits
    transcription = processor.batch_decode(logits.cpu().numpy()).text
    print(transcription[0].lower())

```

And a CPU version, will just run slower OOM 🙂

```auto
from datasets import load_dataset
from transformers import Wav2Vec2ProcessorWithLM, Wav2Vec2ForCTC
import torch

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
processor = Wav2Vec2ProcessorWithLM.from_pretrained("patrickvonplaten/wav2vec2-base-100h-with-lm")
model = Wav2Vec2ForCTC.from_pretrained("patrickvonplaten/wav2vec2-base-100h-with-lm")
print("* * * * * * loaded models")

for i in range(200):
    audio_sample = dataset[i]
    print(" * * * * Sample: ", i)
    inputs = processor(audio_sample["audio"]["array"], sampling_rate=audio_sample["audio"]["sampling_rate"], return_tensors="pt")
    with torch.no_grad():
      logits = model(**inputs).logits
    transcription = processor.batch_decode(logits.numpy()).text
    print(transcription[0].lower())

```

---

<div class="post-metadata">

**Author:** ![othiele](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/othiele/32/19711_2.png) [@othiele](https://discuss.huggingface.co/u/othiele)\
**Post date:** [January 26, 2022, 2:32pm UTC](https://discuss.huggingface.co/t/wav2vec-with-new-lm-causing-cpu-oom/14069/2 "2022-01-26T14:32:55Z")

</div>

Also [asked this on github](https://github.com/huggingface/transformers/issues/15344).

---

<div class="post-metadata">

**Author:** ![othiele](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/othiele/32/19711_2.png) [@othiele](https://discuss.huggingface.co/u/othiele)\
**Post date:** [January 27, 2022, 7:38am UTC](https://discuss.huggingface.co/t/wav2vec-with-new-lm-causing-cpu-oom/14069/3 "2022-01-27T07:38:41Z")

</div>

And [solved on Github](https://github.com/huggingface/transformers/issues/15344#issuecomment-1022722961), a missing pool.close() in the processor code.