# Does pipline with accelerate use "with init\_empty\_weights():"?

**URL:** <https://discuss.huggingface.co/t/does-pipline-with-accelerate-use-with-init-empty-weights/80544>\
**Category:** 🤗Accelerate\
**Created:** [April 7, 2024, 11:37am UTC](https://discuss.huggingface.co/t/does-pipline-with-accelerate-use-with-init-empty-weights/80544 "2024-04-07T11:37:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vivek9840](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@vivek9840](https://discuss.huggingface.co/u/vivek9840)\
**Post date:** [April 7, 2024, 11:37am UTC](https://discuss.huggingface.co/t/does-pipline-with-accelerate-use-with-init-empty-weights/80544/1 "2024-04-07T11:37:43Z")

</div>

i am using pipeline to infer LLMs with 7B to 70B on kaggle and my local setup.

i saw the application of accelerate for infering large models. and i saw the base.py and utils of pipeline from huggigface.

i didn’t find the “with init\_empty\_weights():” which is used for infering big models.

does it not use it or it is used but nt written in those files.

if it not written, can i load model with “with init\_empty\_weights():” and pass that model in pipeline with same device as i load using load\_checkpoint\_and\_dispatch() and tie\_weight()?

---

<div class="post-metadata">

**Author:** ![marcsun13](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcsun13/32/19059_2.png) [@marcsun13](https://discuss.huggingface.co/u/marcsun13)\
**Post date:** [April 15, 2024, 12:18pm UTC](https://discuss.huggingface.co/t/does-pipline-with-accelerate-use-with-init-empty-weights/80544/2 "2024-04-15T12:18:41Z")

</div>

Hi @vivek9840, you can’t do the following:

> if it not written, can i load model with “with init\_empty\_weights():” and pass that model in pipeline with same device as i load using load\_checkpoint\_and\_dispatch() and tie\_weight()?

However, you can pass device\_map=“auto” in pipeline. It will use `init_empty_weights` and more to load your model on your gpu(s). With this arg, you will be able to perform big model inference as described [Handling big models for inference](https://huggingface.co/docs/accelerate/en/usage_guides/big_modeling).

---

<div class="post-metadata">

**Author:** ![vivek9840](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@vivek9840](https://discuss.huggingface.co/u/vivek9840)\
**Post date:** [April 15, 2024, 3:52pm UTC](https://discuss.huggingface.co/t/does-pipline-with-accelerate-use-with-init-empty-weights/80544/3 "2024-04-15T15:52:54Z")

</div>

thanks for your reply. i was also curious since i didn’t see the line where it uses `init_empty_weights`. i tried to see the dile in pipeline and accelerate but i didn’t saw the use of `init_empty_weights`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [April 16, 2024, 3:53am UTC](https://discuss.huggingface.co/t/does-pipline-with-accelerate-use-with-init-empty-weights/80544/4 "2024-04-16T03:53:31Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.