# Multi-GPU eval in PyTorch training loop with generate method

**URL:** <https://discuss.huggingface.co/t/multi-gpu-eval-in-pytorch-training-loop-with-generate-method/13509>\
**Category:** 🤗Accelerate\
**Created:** [January 9, 2022, 4:15pm UTC](https://discuss.huggingface.co/t/multi-gpu-eval-in-pytorch-training-loop-with-generate-method/13509 "2022-01-09T16:15:47Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![VictorAtPL](https://avatars.discourse-cdn.com/v4/letter/v/9f8e36/32.png) [@VictorAtPL](https://discuss.huggingface.co/u/VictorAtPL)\
**Post date:** [January 9, 2022, 4:15pm UTC](https://discuss.huggingface.co/t/multi-gpu-eval-in-pytorch-training-loop-with-generate-method/13509/1 "2022-01-09T16:15:47Z")

</div>

Hi,

Thank you for your work. I really like the idea behind Transformers and Accelerate libraries.

I am experimenting with TrOCR fine-tuning and currently I train it on multi-gpu, but evaluate on single-gpu using following code:

```auto
   if accelerator.is_main_process:
     unwrapped_model = accelerator.unwrap_model(model).to(accelerator.device)
     unwrapped_model.eval()
    
     valid_cer = 0.0
     with torch.no_grad():
       for batch in tqdm(eval_dataloader):
         outputs = unwrapped_model.generate(batch["pixel_values"].to(accelerator.device))
         cer = compute_cer(pred_ids=outputs, label_ids=batch["labels"])
         valid_cer += cer 

     accelerator.print("Validation CER:", valid_cer / len(eval_dataloader))

```

Is it possible to use `generate` method on a parallelized model?

---

<div class="post-metadata">

**Author:** ![smangrul](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/smangrul/32/3791_2.png) [@smangrul](https://discuss.huggingface.co/u/smangrul)\
**Post date:** [August 30, 2022, 4:16am UTC](https://discuss.huggingface.co/t/multi-gpu-eval-in-pytorch-training-loop-with-generate-method/13509/3 "2022-08-30T04:16:21Z")

</div>

Hello, yes, you can use `generate` method in a multi-gpu setting. Refer to the official example script [transformers/run\_translation\_no\_trainer.py at main · huggingface/transformers (github.com)](https://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/run_translation_no_trainer.py#L664).