# Accelerate test stuck on training

**URL:** <https://discuss.huggingface.co/t/accelerate-test-stuck-on-training/34498>\
**Category:** 🤗Accelerate\
**Created:** [March 23, 2023, 5:04pm UTC](https://discuss.huggingface.co/t/accelerate-test-stuck-on-training/34498 "2023-03-23T17:04:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vican9000](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@vican9000](https://discuss.huggingface.co/u/vican9000)\
**Post date:** [March 23, 2023, 5:04pm UTC](https://discuss.huggingface.co/t/accelerate-test-stuck-on-training/34498/1 "2023-03-23T17:04:13Z")

</div>

I am using two A100 GPUs to train some audio models on Jupyter environment (Google Compute instance). Recently I got interested in the accelerate package and adjusted the code accordingly (it was rather straightforward) but it always gets stuck on accelerator.backward(loss).  
Today I tried “accelerate test” and it ALSO gets stuck.

```auto
Running: accelerate-launch /opt/conda/lib/python3.7/site-packages/accelerate/test_utils/scripts/test_script.py
stderr: WARNING:accelerate.commands.launch:The following values were not passed to `accelerate launch` and had defaults used instead:
stderr: `--dynamo_backend` was set to a value of `'no'`
stderr: To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
stdout: **Initialization**
stdout: Testing, testing. 1, 2, 3.
stdout: Distributed environment: MULTI_GPU Backend: nccl
stdout: Num processes: 2
stdout: Process index: 0
stdout: Local process index: 0
stdout: Device: cuda:0
stdout:
stdout: Mixed precision type: no
stdout:
stdout:
stdout: **Test process execution**
stdout: Distributed environment: MULTI_GPU Backend: nccl
stdout: Num processes: 2
stdout: Process index: 1
stdout: Local process index: 1
stdout: Device: cuda:1
stdout:
stdout: Mixed precision type: no
stdout:
stdout:
stdout: **Test random number generator synchronization**
stdout: All rng are properly synched.
stdout:
stdout: **DataLoader integration test**
stdout: 1 0 tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
stdout: 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
stdout: 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
stdout: 54, 55, 56, 57, 58, 59, 60, 61, 62, 63], device='cuda:1')tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
stdout: 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
stdout: 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
stdout: 54, 55, 56, 57, 58, 59, 60, 61, 62, 63], device='cuda:0') <class 'accelerate.data_loader.DataLoaderShard'><class 'accelerate.data_loader.DataLoaderShard'>
stdout:
stdout: Non-shuffled dataloader passing.
stdout: Shuffled dataloader passing.
stdout: Non-shuffled central dataloader passing.
stdout: Shuffled central dataloader passing.
stdout:
stdout: **Training integration test**
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout:

```

accelerate config is setup as multi-GPU (2 cards), utilizing all cards, NO to all other options.

---

<div class="post-metadata">

**Author:** ![miguelcarv](https://avatars.discourse-cdn.com/v4/letter/m/848f3c/32.png) [@miguelcarv](https://discuss.huggingface.co/u/miguelcarv)\
**Post date:** [January 11, 2024, 7:24pm UTC](https://discuss.huggingface.co/t/accelerate-test-stuck-on-training/34498/2 "2024-01-11T19:24:26Z")

</div>

Have you been able to find a solution to this issue?

---

<div class="post-metadata">

**Author:** ![marcsun13](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcsun13/32/19059_2.png) [@marcsun13](https://discuss.huggingface.co/u/marcsun13)\
**Post date:** [January 24, 2024, 5:13pm UTC](https://discuss.huggingface.co/t/accelerate-test-stuck-on-training/34498/3 "2024-01-24T17:13:57Z")

</div>

Hi ! Can you try to install the latest accelerate and reboot your env ? LMK how is goes !