# Loading community JSON based datasets without a script

**URL:** <https://discuss.huggingface.co/t/loading-community-json-based-datasets-without-a-script/10405>\
**Category:** 🤗Datasets\
**Created:** [September 30, 2021, 7:38am UTC](https://discuss.huggingface.co/t/loading-community-json-based-datasets-without-a-script/10405 "2021-09-30T07:38:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vblagoje](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vblagoje/32/20818_2.png) [@vblagoje](https://discuss.huggingface.co/u/vblagoje)\
**Post date:** [September 30, 2021, 7:38am UTC](https://discuss.huggingface.co/t/loading-community-json-based-datasets-without-a-script/10405/1 "2021-09-30T07:38:22Z")

</div>

Hi,

I can easily create and upload JSON community datasets to HF hub (without a script). However, when using the dataset I need to specify data\_files parameter, i.e. `dataset = load_dataset("vblagoje/dataset_abc", data_files={"train": "train.json", "validation": "validation.json", "test": "test.json"})` which is a bit of inconvenience.

Is there a way to avoid data\_files parameters short of providing a loading script as well? If not, why don’t we rely on naming conventions to resolve the data splits anyway?

Cheers,  
Vladimir

---

<div class="post-metadata">

**Author:** ![julien-c](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/julien-c/32/41937_2.png) [@julien-c](https://discuss.huggingface.co/u/julien-c)\
**Post date:** [September 30, 2021, 11:19am UTC](https://discuss.huggingface.co/t/loading-community-json-based-datasets-without-a-script/10405/2 "2021-09-30T11:19:01Z")

</div>

Tagging @lhoestq @albertvillanova in case you hadn’t seen this!

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [October 4, 2021, 9:17am UTC](https://discuss.huggingface.co/t/loading-community-json-based-datasets-without-a-script/10405/3 "2021-10-04T09:17:29Z")

</div>

Hi ! I like the idea of using a name convention. By default it could use something like

```auto
data_files = {
    "train": ["*train*"],
    "test": ["*test*"],
    "validation": ["*dev*", "*valid*"]
}

```

Another alternative would be to let the user specify the `data_files` patterns in an additional JSON file (config.json or something like this). Let me know what you think !

---

<div class="post-metadata">

**Author:** ![vblagoje](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vblagoje/32/20818_2.png) [@vblagoje](https://discuss.huggingface.co/u/vblagoje)\
**Post date:** [October 4, 2021, 10:45am UTC](https://discuss.huggingface.co/t/loading-community-json-based-datasets-without-a-script/10405/4 "2021-10-04T10:45:59Z")

</div>

Yes, that’s the perfect default @lhoestq Perhaps it could be expanded later, but for now, this solution suffices IMHO