# HF Datasets best practices

**URL:** <https://discuss.huggingface.co/t/hf-datasets-best-practices/58566>\
**Category:** 🤗Datasets\
**Created:** [October 14, 2023, 6:04pm UTC](https://discuss.huggingface.co/t/hf-datasets-best-practices/58566 "2023-10-14T18:04:05Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![lucmos](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lucmos/32/20356_2.png) [@lucmos](https://discuss.huggingface.co/u/lucmos)\
**Post date:** [October 14, 2023, 6:04pm UTC](https://discuss.huggingface.co/t/hf-datasets-best-practices/58566/1 "2023-10-14T18:04:05Z")

</div>

Hello everyone!

We are building a [generic cookiecutter template to bootstrap PyTorch projects](https://github.com/grok-ai/nn-template) and to avoid writing boilerplate.

Among the other things, we integrated the HF Datasets library.

This means that any new project generated with this template already has a hugging face dataset configured and ready to be used to train a neural network 😄

I would really appreciate any feedback on how to improve this integration – are there any best practices that we are not using?

The interesting files are the following:

- `.../data/datamodule.py` where the (torchvision) transforms are applied with `set_transform`; and the DataLoaders are created.
- `.../utils/hf_io.py` that handle the loading of the train/val/test splits and some basic pre-processing.

I am particular interested in speed related improvements

Thanks!