# Tensor parallelism for customized model

**URL:** <https://discuss.huggingface.co/t/tensor-parallelism-for-customized-model/104867>\
**Category:** 🤗Accelerate\
**Created:** [September 2, 2024, 12:48am UTC](https://discuss.huggingface.co/t/tensor-parallelism-for-customized-model/104867 "2024-09-02T00:48:18Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![zzxslp](https://avatars.discourse-cdn.com/v4/letter/z/e47774/32.png) [@zzxslp](https://discuss.huggingface.co/u/zzxslp)\
**Post date:** [September 2, 2024, 12:48am UTC](https://discuss.huggingface.co/t/tensor-parallelism-for-customized-model/104867/1 "2024-09-02T00:48:18Z")

</div>

Hi! If I want to do **model inference with a customized large model** (can’t fit into a single GPU), how do I enable tensor parallel to shard the model across multiple GPUs?

For my specific case, I have a multimodal-LLM with a ViT, projector, and an LLM. I am not sure what is the best library to start with. Should I use deepspeed, accelerate or something else?