# Accelerate TPU training

**URL:** <https://discuss.huggingface.co/t/accelerate-tpu-training/95721>\
**Category:** 🤗Accelerate\
**Created:** [July 5, 2024, 2:23pm UTC](https://discuss.huggingface.co/t/accelerate-tpu-training/95721 "2024-07-05T14:23:51Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![hannayukhymenko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hannayukhymenko/32/29011_2.png) [@hannayukhymenko](https://discuss.huggingface.co/u/hannayukhymenko)\
**Post date:** [July 5, 2024, 2:23pm UTC](https://discuss.huggingface.co/t/accelerate-tpu-training/95721/1 "2024-07-05T14:23:51Z")

</div>

Hi! I am using GCP v4-16 TPU Pod for training an LLM. I am learning how to do big distributed model training on TPU and I have several issues with using accelerate for TPU:

1. Accelerate fails to recognize XLA device, when I run the training script without accelerate I do not have this issue. I have tried setting up config using `accelerate config` and it still fails to find the TPU device.
2. Does accelerate support multi-worker/node TPU training, similarly to `Multi-GPU` in distributed\_type? I might be incorrect here since I haven’t used accelerate before - should I manually assign workers to acelerate or it handles this process by itself?

Thanks!