# \`load\_from\_cache\_file\` not working

**URL:** <https://discuss.huggingface.co/t/load-from-cache-file-not-working/5978>\
**Category:** 🤗Datasets\
**Created:** [May 4, 2021, 4:34pm UTC](https://discuss.huggingface.co/t/load-from-cache-file-not-working/5978 "2021-05-04T16:34:44Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sudarshan85](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@sudarshan85](https://discuss.huggingface.co/u/sudarshan85)\
**Post date:** [May 4, 2021, 4:34pm UTC](https://discuss.huggingface.co/t/load-from-cache-file-not-working/5978/1 "2021-05-04T16:34:44Z")

</div>

Hi,

I’m using the `datasets` library to load in the popular medical dataset MIMIC 3 (only the notes) and creating a huggingface `dataset` to get it ready for language modelling using BERT. I have a script that loads creates a custom dataset and tokenizes it and writes it to the cache file. I set `load_from_cache_file` in the `map` function of the `dataset` to `True`. The tokenization process takes a while because of the moderately large dataset size.

When I run the script again, I would expect the dataset to be loaded from the cache file quickly and move on to the next step in the process. However, even though I get a warning saying `Reusing dataset text`, the tokenization process is running again. This is not good because I intend of training multiple models and measuring the training time as part of a project.

However, I did notice that the time it takes to run when running the second time is less. For example, when running the first time without any cache file, the process took 25 minutes. When running the second time with the cache file available and `load_from_cache_file` set to `True`, it took 13 minutes. Is this expected behavior?

I’m using version `1.6.1` of the `datasets` library. I would like some help in resolving this problem. Please let me know if more info is needed.

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [May 10, 2021, 1:09pm UTC](https://discuss.huggingface.co/t/load-from-cache-file-not-working/5978/2 "2021-05-10T13:09:20Z")

</div>

Hi !

Have you tried using the latest release `1.6.2` ? Have you changed the parameters passed to `map` between the two runs ?  
If the parameters (both the tokenization function, the batch size, etc.) are the same, then it will reload the tokenized text from the cache.

Also if you share the code that you’re using it would be helpful 🙂