# Cache large models on GPU instances between reboots

**URL:** <https://discuss.huggingface.co/t/cache-large-models-on-gpu-instances-between-reboots/31415>\
**Category:** Spaces\
**Created:** [February 10, 2023, 2:44am UTC](https://discuss.huggingface.co/t/cache-large-models-on-gpu-instances-between-reboots/31415 "2023-02-10T02:44:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexedw](https://avatars.discourse-cdn.com/v4/letter/a/90ced4/32.png) [@alexedw](https://discuss.huggingface.co/u/alexedw)\
**Post date:** [February 10, 2023, 2:44am UTC](https://discuss.huggingface.co/t/cache-large-models-on-gpu-instances-between-reboots/31415/1 "2023-02-10T02:44:36Z")

</div>

Hey!

I’m using some A10G instances to run a 20GB model in a private Space, and I’ve got the Space set to shut down after 15 minutes of no use to save $.

Each cold-start though takes a few minutes to re-download all model weights from the hub, which is a bit of a pain to wait for and I’m sure an annoying amount of bandwidth for huggingface.

Is there any method of caching (or similar) these model weights, or a recommended way to load/store them that doesn’t require the space to reload them each time?

Cheers,  
Alex

---

<div class="post-metadata">

**Author:** ![chris-rannou](https://avatars.discourse-cdn.com/v4/letter/c/7feea3/32.png) [@chris-rannou](https://discuss.huggingface.co/u/chris-rannou)\
**Post date:** [February 10, 2023, 5:36pm UTC](https://discuss.huggingface.co/t/cache-large-models-on-gpu-instances-between-reboots/31415/2 "2023-02-10T17:36:00Z")

</div>

Hi @alexedw,

There is no solution as of now to cache anything in between reboots of a Space.

The only workaround currently would be to use a [Space with a Docker SDK](https://huggingface.co/docs/hub/spaces-sdks-docker) and download the weights as a step of the image build.

---

<div class="post-metadata">

**Author:** ![brianjking](https://avatars.discourse-cdn.com/v4/letter/b/db5fbb/32.png) [@brianjking](https://discuss.huggingface.co/u/brianjking)\
**Post date:** [February 13, 2023, 7:56pm UTC](https://discuss.huggingface.co/t/cache-large-models-on-gpu-instances-between-reboots/31415/3 "2023-02-13T19:56:46Z")

</div>

Do you have an example of what a Dockerfile would look like to cache the weights?

Thanks @chris-rannou

---

<div class="post-metadata">

**Author:** ![chris-rannou](https://avatars.discourse-cdn.com/v4/letter/c/7feea3/32.png) [@chris-rannou](https://discuss.huggingface.co/u/chris-rannou)\
**Post date:** [February 14, 2023, 9:27am UTC](https://discuss.huggingface.co/t/cache-large-models-on-gpu-instances-between-reboots/31415/4 "2023-02-14T09:27:10Z")

</div>

Hi @brianjking,

A minimal example could be, using the transformers cli:

```auto
FROM python:3.8-slim-buster

# Set up a new user named "user" with user ID 1000
RUN useradd -m -u 1000 user

# Switch to the "user" user
USER user

# Set home to the user's home directory
RUN mkdir -p $HOME/app

ENV HOME=/home/user \
	PATH=/home/user/.local/bin:$PATH

# Set the working directory to the user's home directory
WORKDIR $HOME/app

# install transformers cli
RUN pip install transformers

# Download model weights
RUN ["transformers-cli", "download", "distilbert-base-cased-distilled-squad"]

```