# Trainer.train() is stuck **URL:** \ **Category:** 🤗Transformers\ **Created:** [February 17, 2021, 9:03pm UTC](https://discuss.huggingface.co/t/trainer-train-is-stuck/3702 "2021-02-17T21:03:33Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![saichandra](https://avatars.discourse-cdn.com/v4/letter/s/f07891/32.png) [@saichandra](https://discuss.huggingface.co/u/saichandra)\ **Post date:** [February 17, 2021, 9:03pm UTC](https://discuss.huggingface.co/t/trainer-train-is-stuck/3702/1 "2021-02-17T21:03:34Z")
Hi, I’m training roberta-base using HF Trainer, but it’s stuck at the starting itself. Here’s my code - ```auto train_dataset[0] {'input_ids': tensor([0, 100, 657, ..., 1, 1, 1]), 'attention_mask': tensor([1, 1, 1, ..., 0, 0, 0]), 'labels': tensor(0)} val_dataset[0] {'input_ids': tensor([0, 11094, 14, ..., 1, 1, 1]), 'attention_mask': tensor([1, 1, 1, ..., 0, 0, 0]), 'labels': tensor(0)} ## simple test model(train_dataset[:2]['input_ids'], attention_mask = train_dataset[:2]['attention_mask'], labels=train_dataset[:2]['labels']) SequenceClassifierOutput(loss=tensor(0.6995, grad_fn=), logits=tensor([[0.0438, -0.1893], [0.0530, -0.1786]], grad_fn=), hidden_states=None, attentions=None) train_args = transformers.TrainingArguments( output_dir='test_1', overwrite_output_dir=True, evaluation_strategy="epoch", per_device_train_batch_size=8, per_device_eval_batch_size=8, learning_rate=3e-5, weight_decay=0.01, num_train_epochs=2, load_best_model_at_end=True, ) trainer = transformers.Trainer( model=model, args=train_args, train_dataset=train_dataset, eval_dataset=val_dataset, tokenizer=tok, ) trainer.train() ``` I saw memory consumption and it is stuck at - ```auto +-----------------------------------------------------------------------------+ | NVIDIA-SMI 450.51.06 Driver Version: 450.51.06 CUDA Version: 11.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla V100-SXM2... On | 00000000:62:00.0 Off | 0 | | N/A 49C P0 60W / 300W | 1756MiB / 32510MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 1 Tesla V100-SXM2... On | 00000000:8A:00.0 Off | 0 | | N/A 50C P0 61W / 300W | 1376MiB / 32510MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| +-----------------------------------------------------------------------------+ ``` Plz let me know how to proceed further… ---
**Author:** ![masoudmzb](https://avatars.discourse-cdn.com/v4/letter/m/e68b1a/32.png) [@masoudmzb](https://discuss.huggingface.co/u/masoudmzb)\ **Post date:** [May 26, 2021, 11:37am UTC](https://discuss.huggingface.co/t/trainer-train-is-stuck/3702/2 "2021-05-26T11:37:17Z")
any progress since then? ---
**Author:** ![flowxd](https://avatars.discourse-cdn.com/v4/letter/f/838e76/32.png) [@flowxd](https://discuss.huggingface.co/u/flowxd)\ **Post date:** [February 1, 2022, 3:10pm UTC](https://discuss.huggingface.co/t/trainer-train-is-stuck/3702/3 "2022-02-01T15:10:25Z")
check if works with num\_worker=0 if you are using windows. It worked for me. here is reference > **[Cannot enumerate over Dataloader object](https://discuss.pytorch.org/t/cannot-enumerate-over-dataloader-object/44167/2)** > > Is your code running fine using num\_workers=0? If you are using multiple workers on Windows, you should add the if-clause protection as described in the Windows FAQ. ---
**Author:** ![hadiqa123](https://avatars.discourse-cdn.com/v4/letter/h/b9e5f3/32.png) [@hadiqa123](https://discuss.huggingface.co/u/hadiqa123)\ **Post date:** [August 31, 2022, 11:10am UTC](https://discuss.huggingface.co/t/trainer-train-is-stuck/3702/4 "2022-08-31T11:10:14Z")
I am also having the same problem. When training the trainer ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/a/aeca9daa466f90cc167b2df2a88af6171876f444.png) I have used the urdu dataset of mozilla-foundation/common\_voice\_7.0. The model which I have used is “facebook/wav2vec2-xls-r-300m” and the libraries and transformers are !pip install datasets ==2.4.0 !pip install transformers ==4.21.2 !pip install torchaudio = 0.11 !pip install jiwer" If anyone could help me, I would be very thankful. ---
**Author:** ![Isma](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/isma/32/5936_2.png) [@Isma](https://discuss.huggingface.co/u/Isma)\ **Post date:** [January 11, 2023, 10:59am UTC](https://discuss.huggingface.co/t/trainer-train-is-stuck/3702/5 "2023-01-11T10:59:04Z")
I ran into the same problem. I am using GCP and when I create notebook using the PyTorch, when I launch the trainer there is no progression. I manage to solve the problem by creating a choosing to create an environment without torch and by installing torch using the following line: ```auto pip install torch==1.13.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html ``` I hope it helps. ---
**Author:** ![borghives](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@borghives](https://discuss.huggingface.co/u/borghives)\ **Post date:** [May 1, 2023, 4:36am UTC](https://discuss.huggingface.co/t/trainer-train-is-stuck/3702/6 "2023-05-01T04:36:14Z")
I ran into a similar problem while I was using GCP. To resolved the problem I edit my instance to have more memory from 4GB to 30GB. I ran out of system memory and the only reason I realized it was because other tasks I was running at that time (non torch/GPU related) threw error. The Trainer.train() returns no error and just hung for hours (I first thought I setup my GPU instance wrong and didn’t have the correct driver)