# Training memory footprint depends on instantiating method

**URL:** <https://discuss.huggingface.co/t/training-memory-footprint-depends-on-instantiating-method/113623>\
**Category:** Models\
**Created:** [October 23, 2024, 3:44pm UTC](https://discuss.huggingface.co/t/training-memory-footprint-depends-on-instantiating-method/113623 "2024-10-23T15:44:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vladkoz](https://avatars.discourse-cdn.com/v4/letter/v/8edcca/32.png) [@vladkoz](https://discuss.huggingface.co/u/vladkoz)\
**Post date:** [October 23, 2024, 3:44pm UTC](https://discuss.huggingface.co/t/training-memory-footprint-depends-on-instantiating-method/113623/1 "2024-10-23T15:44:51Z")

</div>

I’ve recently encountered a strange issue regarding the difference in GPU memory footprint during training.

I launched two training setups with identical parameters and random seeds. The only difference was the method used to instantiate the Hugging Face model:

**Option 1:**

`model = ModelClass.from_pretrained(path_to_hub)`

**Option 2:**

```auto
config = Config.from_pretrained(path_to_hub)
model = ModelClass(config)

```

Surprisingly, I was able to train the configuration initialized with Option 1 (from\_pretrained) using a larger batch size than with Option 2 (from config), which resulted in a “CUDA out of memory” error when I attempted a larger batch.

I compared the model configurations and the `dtype` of the weights, but I couldn’t find any differences between the two options.

Why am I observing this effect? Is this expected behavior? Could it be hardware-specific or related to CUDA memory allocation? Has anyone else experienced this? I haven’t found any relevant information online.

**P.S.**

All training experiments (including all GPU communication) were based on PyTorch Lightning, which might be a relevant detail.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [October 23, 2024, 4:04pm UTC](https://discuss.huggingface.co/t/training-memory-footprint-depends-on-instantiating-method/113623/2 "2024-10-23T16:04:18Z")

</div>

I have not noticed the same experience, but I guess I have.  
What I suspect in these cases would be `torch` and the `accelerate` library.  
It is possible that `accelerate` is doing a good job of allocating RAM and VRAM, and `torch` may be using a **meta tensor**.  
A bad pattern would be a bug in `from_pretrained()` or `load_from_config()` in the `transformers` library…