# Data Parallelism for multi-GPUs Inference

**URL:** <https://discuss.huggingface.co/t/data-parallelism-for-multi-gpus-inference/25003>\
**Category:** Intermediate\
**Created:** [October 26, 2022, 10:02am UTC](https://discuss.huggingface.co/t/data-parallelism-for-multi-gpus-inference/25003 "2022-10-26T10:02:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Giulietta](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/giulietta/32/12249_2.png) [@Giulietta](https://discuss.huggingface.co/u/Giulietta)\
**Post date:** [October 26, 2022, 10:02am UTC](https://discuss.huggingface.co/t/data-parallelism-for-multi-gpus-inference/25003/1 "2022-10-26T10:02:37Z")

</div>

Hi everyone,

I need to do inference on huge size of data and I would like to send the pre-trained HF model to multiple GPUs. Therefore my problem is a data parallel rather than a model parallel. I have seen that DP cannot support model.generate() method. Please, do you have any suggestion about Inference on multi-GPU?

Thanks a lot!