# Tensor parallel in Pytorch 2.3

**URL:** <https://discuss.huggingface.co/t/tensor-parallel-in-pytorch-2-3/84909>\
**Category:** 🤗Transformers\
**Created:** [May 2, 2024, 11:17pm UTC](https://discuss.huggingface.co/t/tensor-parallel-in-pytorch-2-3/84909 "2024-05-02T23:17:19Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![proj-persona](https://avatars.discourse-cdn.com/v4/letter/p/f6c823/32.png) [@proj-persona](https://discuss.huggingface.co/u/proj-persona)\
**Post date:** [May 2, 2024, 11:17pm UTC](https://discuss.huggingface.co/t/tensor-parallel-in-pytorch-2-3/84909/1 "2024-05-02T23:17:19Z")

</div>

Tensor parallelism has been supported by Pytorch 2.3. Just wonder when Transformers could also officially support this feature, which would be very helpful to handle very large language models.