# Any good code/tutorial that is shows how to do inference with Llama 2 70b on multiple GPUs with accelerate?

**URL:** <https://discuss.huggingface.co/t/any-good-code-tutorial-that-is-shows-how-to-do-inference-with-llama-2-70b-on-multiple-gpus-with-accelerate/60049>\
**Category:** 🤗Accelerate\
**Created:** [October 26, 2023, 3:25pm UTC](https://discuss.huggingface.co/t/any-good-code-tutorial-that-is-shows-how-to-do-inference-with-llama-2-70b-on-multiple-gpus-with-accelerate/60049 "2023-10-26T15:25:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vbachi](https://avatars.discourse-cdn.com/v4/letter/v/9d8465/32.png) [@vbachi](https://discuss.huggingface.co/u/vbachi)\
**Post date:** [October 26, 2023, 3:25pm UTC](https://discuss.huggingface.co/t/any-good-code-tutorial-that-is-shows-how-to-do-inference-with-llama-2-70b-on-multiple-gpus-with-accelerate/60049/1 "2023-10-26T15:25:05Z")

</div>

Do you know of any good code/tutorial that is shows how to do inference with Llama 2 70b on multiple GPUs with accelerate?

---

<div class="post-metadata">

**Author:** ![marcsun13](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcsun13/32/19059_2.png) [@marcsun13](https://discuss.huggingface.co/u/marcsun13)\
**Post date:** [November 27, 2023, 6:00pm UTC](https://discuss.huggingface.co/t/any-good-code-tutorial-that-is-shows-how-to-do-inference-with-llama-2-70b-on-multiple-gpus-with-accelerate/60049/2 "2023-11-27T18:00:42Z")

</div>

Hey @vbachi , you can check this doc: [Handling big models for inference](https://huggingface.co/docs/accelerate/usage_guides/big_modeling)