# Would PyTorch's FSDP work with a model loaded using device\_map='auto'?

**URL:** <https://discuss.huggingface.co/t/would-pytorchs-fsdp-work-with-a-model-loaded-using-device-map-auto/82143>\
**Category:** 🤗Transformers\
**Created:** [April 17, 2024, 3:24am UTC](https://discuss.huggingface.co/t/would-pytorchs-fsdp-work-with-a-model-loaded-using-device-map-auto/82143 "2024-04-17T03:24:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ChristopherTan](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@ChristopherTan](https://discuss.huggingface.co/u/ChristopherTan)\
**Post date:** [April 17, 2024, 3:24am UTC](https://discuss.huggingface.co/t/would-pytorchs-fsdp-work-with-a-model-loaded-using-device-map-auto/82143/1 "2024-04-17T03:24:38Z")

</div>

I’m guessing no because FSDP has a very specific way that it distributes the model over the GPUs and device\_map=‘auto’ might not align with that. Is my understanding correct? Is that why training usually does not work with device\_map=‘auto’?