# T5 inference performance

**URL:** <https://discuss.huggingface.co/t/t5-inference-performance/15494>\
**Category:** Models\
**Created:** [March 8, 2022, 9:53am UTC](https://discuss.huggingface.co/t/t5-inference-performance/15494 "2022-03-08T09:53:41Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![YannAgora](https://avatars.discourse-cdn.com/v4/letter/y/c0e974/32.png) [@YannAgora](https://discuss.huggingface.co/u/YannAgora)\
**Post date:** [March 8, 2022, 1:31pm UTC](https://discuss.huggingface.co/t/t5-inference-performance/15494/2 "2022-03-08T13:31:02Z")

</div>

Hey 👋,

Did you try quantization ?

There is an example for pegasus model [here](https://discuss.huggingface.co/t/pegasus-model-weights-compression-pruning/6381/9). I tried and it performed pretty well for summarization with an inference time decrease by 2x or 3x

---

_[View the full topic](https://discuss.huggingface.co/t/t5-inference-performance/15494)._