# Datasets not using the cache dir

**URL:** <https://discuss.huggingface.co/t/datasets-not-using-the-cache-dir/63469>\
**Category:** 🤗Datasets\
**Created:** [November 25, 2023, 5:39am UTC](https://discuss.huggingface.co/t/datasets-not-using-the-cache-dir/63469 "2023-11-25T05:39:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sebastienwood](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@sebastienwood](https://discuss.huggingface.co/u/sebastienwood)\
**Post date:** [November 25, 2023, 5:39am UTC](https://discuss.huggingface.co/t/datasets-not-using-the-cache-dir/63469/1 "2023-11-25T05:39:07Z")

</div>

Hi !  
I have a really simple script:

```auto
import datasets
import os

dset = datasets.load_dataset("imagenet-1k", cache_dir=os.environ["DATASET_STORE"])

```

which should use a cached version of the Imagenet1k dataset at $DATASET\_STORE, which exists:

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/4/4/44950196e3ffba19ba2afe0bc345dc999c3389e2.png)  
And as the screen shows, it always tries to download the dataset online rather than using the cache.  
Am I doing it wrong ? Thanks !

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\
**Post date:** [November 28, 2023, 3:48pm UTC](https://discuss.huggingface.co/t/datasets-not-using-the-cache-dir/63469/2 "2023-11-28T15:48:20Z")

</div>

The `datasets`’ cache structure is incompatible with manually downloaded files, so you should use `Dataset.from_generator` instead to parse those files.

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [November 29, 2023, 5:15pm UTC](https://discuss.huggingface.co/t/datasets-not-using-the-cache-dir/63469/3 "2023-11-29T17:15:59Z")

</div>

I think it should work no ? `imagenet-1k` doesn’t need manually downloaded files.

Maybe it fails to find the loading script in the cache, which is in a different cache directory `dynamic_modules_cache` ?