# Model parallel with deepspeed integration

**URL:** <https://discuss.huggingface.co/t/model-parallel-with-deepspeed-integration/9980>\
**Category:** Beginners\
**Created:** [September 14, 2021, 12:39pm UTC](https://discuss.huggingface.co/t/model-parallel-with-deepspeed-integration/9980 "2021-09-14T12:39:23Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiami](https://avatars.discourse-cdn.com/v4/letter/x/dc4da7/32.png) [@xiami](https://discuss.huggingface.co/u/xiami)\
**Post date:** [September 14, 2021, 12:39pm UTC](https://discuss.huggingface.co/t/model-parallel-with-deepspeed-integration/9980/1 "2021-09-14T12:39:23Z")

</div>

If I use the deepspeed integration with multi-GPU, am I doing model parallel or just data parallel?  
I know the deepspeed library may be able of model parallel but I am still learning it. Is there a way to do model parallel in transformers?

And is there a doc or example for the parallelize method in T5(or any other model)?

Thank you for your time!