# Trainer.evaluate() freezing

**URL:** <https://discuss.huggingface.co/t/trainer-evaluate-freezing/75067>\
**Category:** 🤗Transformers\
**Created:** [February 28, 2024, 12:03am UTC](https://discuss.huggingface.co/t/trainer-evaluate-freezing/75067 "2024-02-28T00:03:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![tylerkim](https://avatars.discourse-cdn.com/v4/letter/t/9d8465/32.png) [@tylerkim](https://discuss.huggingface.co/u/tylerkim)\
**Post date:** [February 28, 2024, 12:03am UTC](https://discuss.huggingface.co/t/trainer-evaluate-freezing/75067/1 "2024-02-28T00:03:27Z")

</div>

Hello, I’m trying to train a RoBERTa model for sequence classification. Previously, I was able to train it with the “test\_trainer” arguments. However, when I would subsequently run trainer.evaluate(), the process would complete and then stall out.

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/a/b/ab8df6159ad4ebf1833f1d18a44fb49427622680.png)  
^trainer.evaluate() freezes here

When I tried incorporating the evaluation step into training arguments (evaluation\_strategy), I would get the same error – after the first epoch/steps would finish training, the evaluation step would happen and then the whole process would freeze.

 ![Screenshot 2024-02-27 at 11.51.16 AM](https://us1.discourse-cdn.com/hellohellohello/original/3X/8/a/8aedb432fdf5b7c3101e0d6b1a5361228751e3f5.png)  
^evaluation\_strategy in training arguments freezes here

CODE:

```auto
training_args = TrainingArguments(
    output_dir="./output",
    num_train_epochs=3,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    evaluation_strategy="steps",
    logging_dir="./output/logs",
    logging_strategy="steps",
    logging_steps=10,
    learning_rate=5e-5,
    weight_decay=0.01,
    warmup_steps=500,
    save_strategy="steps",
    load_best_model_at_end=True,
    save_total_limit=2,
)

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return metric.compute(predictions=predictions, references=labels)

metric = Accuracy()

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_data,
    eval_dataset=eval_data,
    compute_metrics=compute_metrics
)

```

No error logs, just the progress bar stalling out.

On transformers v4.33.1

---

<div class="post-metadata">

**Author:** ![tylerkim](https://avatars.discourse-cdn.com/v4/letter/t/9d8465/32.png) [@tylerkim](https://discuss.huggingface.co/u/tylerkim)\
**Post date:** [March 6, 2024, 2:42am UTC](https://discuss.huggingface.co/t/trainer-evaluate-freezing/75067/3 "2024-03-06T02:42:38Z")

</div>

one more bump

---

<div class="post-metadata">

**Author:** ![Littleetree](https://avatars.discourse-cdn.com/v4/letter/l/3bc359/32.png) [@Littleetree](https://discuss.huggingface.co/u/Littleetree)\
**Post date:** [April 29, 2024, 11:25am UTC](https://discuss.huggingface.co/t/trainer-evaluate-freezing/75067/4 "2024-04-29T11:25:26Z")

</div>

I met the same issue as you and had no idea what was happening. Have you figured out a solution?

---

<div class="post-metadata">

**Author:** ![lowend1hz](https://avatars.discourse-cdn.com/v4/letter/l/ba8739/32.png) [@lowend1hz](https://discuss.huggingface.co/u/lowend1hz)\
**Post date:** [August 23, 2024, 9:57am UTC](https://discuss.huggingface.co/t/trainer-evaluate-freezing/75067/5 "2024-08-23T09:57:52Z")

</div>

I’m running into a similar issue, but things typically stop after several training epochs and consistently fail to complete 25 epochs. All of the sudden things just stop progressing and the GPU stops doing any computation, but still has memory reserved; the memory is typically between 4-6GB on a GPU with 24GB available. CPU doesn’t ever show significant memory pressure either and usually has around 32-40GB of RAM available.