# Setting requires\_grad=False seems not saving GPU memory usage

**URL:** <https://discuss.huggingface.co/t/setting-requires-grad-false-seems-not-saving-gpu-memory-usage/69590>\
**Category:** 🤗Transformers\
**Created:** [January 18, 2024, 7:43am UTC](https://discuss.huggingface.co/t/setting-requires-grad-false-seems-not-saving-gpu-memory-usage/69590 "2024-01-18T07:43:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![sbchoi](https://avatars.discourse-cdn.com/v4/letter/s/ecae2f/32.png) [@sbchoi](https://discuss.huggingface.co/u/sbchoi)\
**Post date:** [January 18, 2024, 7:43am UTC](https://discuss.huggingface.co/t/setting-requires-grad-false-seems-not-saving-gpu-memory-usage/69590/1 "2024-01-18T07:43:46Z")

</div>

> [@Trainer() and required\_grad=false](https://discuss.huggingface.co/t/trainer-and-required-grad-false/41230):
>
> hey guys i just have a quick question, if i freeze 12 out of 16 layers, does huggingface trainer() load everything into memory anyway?

related to this topic,

from transformers import (  
AutoModelForObjectDetection,  
)  
model = AutoModelForObjectDetection.from\_pretrained(  
‘jozhang97/deta-swin-large’,  
)  
for n, i in model.model.backbone.named\_parameters():  
# i.eval()  
i.requires\_grad = False  
model.model.level\_embed.requires\_grad = False

However, training with transformers trainer is not reducing GPU memory. Both requires\_grad = False give both 23GB usage of VRAM.  
Any recommendation? Or is this related accelerate?