# Move model with device\_map="balanced" to CPU

**URL:** <https://discuss.huggingface.co/t/move-model-with-device-map-balanced-to-cpu/43932>\
**Category:** 🤗Transformers\
**Created:** [June 20, 2023, 2:00pm UTC](https://discuss.huggingface.co/t/move-model-with-device-map-balanced-to-cpu/43932 "2023-06-20T14:00:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![abhinavkulkarni](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/abhinavkulkarni/32/32750_2.png) [@abhinavkulkarni](https://discuss.huggingface.co/u/abhinavkulkarni)\
**Post date:** [June 20, 2023, 2:00pm UTC](https://discuss.huggingface.co/t/move-model-with-device-map-balanced-to-cpu/43932/1 "2023-06-20T14:00:15Z")

</div>

Hi,

I have a large model that I am unable to fit into GPU, so I am loading it as follows:

```auto
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

kwargs = {"device_map": "balanced", "torch_dtype": torch.float16}
max_memory = {0: "10GiB", "cpu": "99GiB"}

model_name = 'facebook/opt-6.7b'
config = AutoConfig.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, config=config, max_memory=max_memory, **kwargs)
tokenizer = AutoTokenizer.from_pretrained(model_name, fast=True)

```

Now, when I try to move the model back to CPU to free up GPU memory for other processing, I get an error:

```auto
model = model.to('cpu')
torch.cuda.empty_cache()

```

I get:

`NotImplementedError: Cannot copy out of meta tensor; no data!`

How can I free up GPU memory?

When I check `model.hf_device_map`, I see that some layers are on GPU while others are on CPU.

Thanks!

---

<div class="post-metadata">

**Author:** ![jhilgart22](https://avatars.discourse-cdn.com/v4/letter/j/f08c70/32.png) [@jhilgart22](https://discuss.huggingface.co/u/jhilgart22)\
**Post date:** [February 5, 2024, 8:38pm UTC](https://discuss.huggingface.co/t/move-model-with-device-map-balanced-to-cpu/43932/2 "2024-02-05T20:38:19Z")

</div>

I’m also running into this with device\_map=‘auto’