# Dataset repo requires arbitrary Python code execution **URL:** \ **Category:** 🤗Datasets\ **Created:** [October 21, 2023, 12:44am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346 "2023-10-21T00:44:11Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![kargaranamir](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kargaranamir/32/25993_2.png) [@kargaranamir](https://discuss.huggingface.co/u/kargaranamir)\ **Post date:** [October 21, 2023, 12:44am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/1 "2023-10-21T00:44:11Z")
The viewer is disabled because this dataset repo requires arbitrary Python code execution. Please consider removing the loading script and relying on automated data support. If this is not possible, please open a discussion for direct help. Why? It worked well in before. By the way load\_datasets works! and viewer works yesterday as well! The dataset: 1. [cis-lmu/udhr-lid ¡ Datasets at Hugging Face](https://huggingface.co/datasets/cis-lmu/udhr-lid) 2. [cis-lmu/glotlid-corpus ¡ Datasets at Hugging Face](https://huggingface.co/datasets/cis-lmu/glotlid-corpus) ---
**Author:** ![unwilledset](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/unwilledset/32/17437_2.png) [@unwilledset](https://discuss.huggingface.co/u/unwilledset)\ **Post date:** [October 21, 2023, 5:12am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/2 "2023-10-21T05:12:57Z")
I have the exact same problem and cannot tell where the error is, given I have not changed anything and this used to work. Hope we can get some assistance. ---
**Author:** ![MajdTannous](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@MajdTannous](https://discuss.huggingface.co/u/MajdTannous)\ **Post date:** [October 21, 2023, 8:45am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/3 "2023-10-21T08:45:31Z")
> [@unwilledset](#): > > I have the exact same problem I have the exact same problem ---
**Author:** ![kargaranamir](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kargaranamir/32/25993_2.png) [@kargaranamir](https://discuss.huggingface.co/u/kargaranamir)\ **Post date:** [October 21, 2023, 8:58am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/4 "2023-10-21T08:58:44Z")
@lhoestq, care to join? I don’t know who to tag. ---
**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\ **Post date:** [October 21, 2023, 10:08am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/5 "2023-10-21T10:08:43Z")
We had to disable the viewer for datasets with a script for now, because some people were abusing it. Sorry for the inconvenience. We’re seeing if there is a viable long-term solution. In the meantime, if you want the dataset viewer to work you need to remove the dataset script and use a supported data format (csv, parquet, etc.) Personnally I’d recommend uploading the dataset using the `datasets` library and `push_to_hub()`. ---
**Author:** ![julien-c](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/julien-c/32/41937_2.png) [@julien-c](https://discuss.huggingface.co/u/julien-c)\ **Post date:** [October 23, 2023, 8:31am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/6 "2023-10-23T08:31:54Z")
If you have a popular datasets (\> 100 likes or downloads) that is affected, please let us know here – we can allowlist popular datasets. Thanks. ---
**Author:** ![kargaranamir](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kargaranamir/32/25993_2.png) [@kargaranamir](https://discuss.huggingface.co/u/kargaranamir)\ **Post date:** [October 26, 2023, 10:50am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/7 "2023-10-26T10:50:11Z")
I don’t have that much publicity. I was hoping to gain some :)) I [fixed](https://huggingface.co/datasets/cis-lmu/udhr-lid) one of my datasets, deleted the other [one](https://huggingface.co/datasets/cis-lmu/glotlid-corpus), and I’m still trying to figure out what’s wrong with the third [one](https://huggingface.co/datasets/cis-lmu/GlotSparse) since the viewer doesn’t work, even though I remove the load script and move to the automatic HuggingFace structure. ---
**Author:** ![julien-c](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/julien-c/32/41937_2.png) [@julien-c](https://discuss.huggingface.co/u/julien-c)\ **Post date:** [October 26, 2023, 12:18pm UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/8 "2023-10-26T12:18:29Z")
cc @severo maybe can help on that last one! ---
**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\ **Post date:** [October 26, 2023, 12:21pm UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/9 "2023-10-26T12:21:53Z")
The viewer is being created on your third dataset, it will be available soon 🙂 ---
**Author:** ![severo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/severo/32/27449_2.png) [@severo](https://discuss.huggingface.co/u/severo)\ **Post date:** [October 26, 2023, 12:44pm UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/10 "2023-10-26T12:44:26Z")
Nice ![Capture d’écran 2023-10-26 à 14.44.14](https://us1.discourse-cdn.com/hellohellohello/original/3X/1/c/1c92908dacb3226dcc84e5324f5cf2b9848111c5.png) ---
**Author:** ![lampent](https://avatars.discourse-cdn.com/v4/letter/l/fbc32d/32.png) [@lampent](https://discuss.huggingface.co/u/lampent)\ **Post date:** [November 26, 2023, 5:34pm UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/11 "2023-11-26T17:34:58Z")
Hi @julien-c, @lhoestq 👋 Is it possible to allow my dataset [“lampent/IRFL”](https://huggingface.co/datasets/lampent/IRFL) to use the dataset viewer with a script? In my previous work (“nlphuji/vasr”), it worked great, and I want to customize this dataset as well. Thank you. Ron. ---
**Author:** ![severo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/severo/32/27449_2.png) [@severo](https://discuss.huggingface.co/u/severo)\ **Post date:** [November 28, 2023, 11:29am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/12 "2023-11-28T11:29:03Z")
I see you made it work using data-only files, congrats! ![Capture d’écran 2023-11-28 à 12.27.07](https://us1.discourse-cdn.com/hellohellohello/original/3X/b/6/b65f848ebf6910812b209892e301e025edc1920b.png) ---
**Author:** ![severo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/severo/32/27449_2.png) [@severo](https://discuss.huggingface.co/u/severo)\ **Post date:** [November 28, 2023, 11:29am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/13 "2023-11-28T11:29:54Z")
We recently updated the docs to make it easier to structure your dataset without a dataset script: > **[Datasets](https://huggingface.co/docs/hub/datasets)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ---
**Author:** ![lampent](https://avatars.discourse-cdn.com/v4/letter/l/fbc32d/32.png) [@lampent](https://discuss.huggingface.co/u/lampent)\ **Post date:** [November 30, 2023, 11:23am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/14 "2023-11-30T11:23:54Z")
Yes, and it works great! However I would like to use the dataset viewer with a script to enable image display instead of strings. As you can see in the fields “distractors” and “answer” this are actually image identifiers and I would like to load them into the dataset viewer as images. The only methods I am aware of is the one we used in “nlphuji/vasr” (see image below). ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/2/1/2170a30ce17ece241eb06f65b76dd3476b02b127.png) ---
**Author:** ![severo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/severo/32/27449_2.png) [@severo](https://discuss.huggingface.co/u/severo)\ **Post date:** [November 30, 2023, 12:05pm UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/15 "2023-11-30T12:05:56Z")
Indeed. We have an issue to handle that case, feel free to chime in, or +1. > > > \### Feature request > > Extend the \`imagefolder\` dataloading script to support load…ing multiple images per dataset entry. > > This only really makes sense if a metadata file is present. > > Currently you can use the following format (example \`metadata.jsonl\`: > \`\`\` > {'file\_name': 'path\_to\_image.png', 'metadata': ...} > ... > \`\`\` > which will return a batch with key \`image\` and any other metadata. > > I would propose extending \`file\_name\` to also accept a list of files, which would return a batch with key \`images\` and any other metadata. > > \### Motivation > > This is useful for example in segmentation tasks in computer vision models, or in text-to-image models that also accept conditioning signals such as another image, feature map, or similar. Currently if I want to do this, I would need to write a custom dataset, rather than just use \`imagefolder\`. > > \### Your contribution > > Would be open to doing a PR, but also happy for someone else to take it as I am not familiar with the datasets library. ---
**Author:** ![leosocy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/leosocy/32/25540_2.png) [@leosocy](https://discuss.huggingface.co/u/leosocy)\ **Post date:** [April 7, 2024, 6:42am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/16 "2024-04-07T06:42:12Z")
Hi @severo @lhoestq Is it possible to allow my dataset [“leosocy/palmnet”](https://huggingface.co/datasets/leosocy/palmnet) to use the dataset viewer with a script? Thank you. Leosocy. ---
**Author:** ![Bakerbunker](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bakerbunker/32/29069_2.png) [@Bakerbunker](https://discuss.huggingface.co/u/Bakerbunker)\ **Post date:** [April 26, 2024, 9:19am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/17 "2024-04-26T09:19:53Z")
Hi, @severo @lhoestq We have a dataset [Wenetspeech4TTS/WenetSpeech4TTS](https://huggingface.co/datasets/Wenetspeech4TTS/WenetSpeech4TTS), is it possible to auto-convert this dataset to parquet and enable the dataset viewer? ---
**Author:** ![severo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/severo/32/27449_2.png) [@severo](https://discuss.huggingface.co/u/severo)\ **Post date:** [April 26, 2024, 9:38am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/18 "2024-04-26T09:38:16Z")
The last `datasets` release ([2.19.0](https://github.com/huggingface/datasets/releases/tag/2.19.0)) provides a CLI tool to convert to data-only (parquet): [Command Line Interface (CLI)](https://huggingface.co/docs/datasets/main/en/cli#convert-to-parquet) Please tell us if it works well for you! ---
**Author:** ![Bakerbunker](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bakerbunker/32/29069_2.png) [@Bakerbunker](https://discuss.huggingface.co/u/Bakerbunker)\ **Post date:** [July 3, 2024, 6:31am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/19 "2024-07-03T06:31:34Z")
Hi, we are the owner of [Wenetspeech4TTS/WenetSpeech4TTS](https://huggingface.co/datasets/Wenetspeech4TTS/WenetSpeech4TTS), we tried to use the CLI interface of `datasets` to convert the dataset, but in china we are faceing network connection issues. ---
**Author:** ![severo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/severo/32/27449_2.png) [@severo](https://discuss.huggingface.co/u/severo)\ **Post date:** [July 8, 2024, 8:40am UTC](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/20 "2024-07-08T08:40:46Z")
cc @Wauplin @lhoestq maybe [Next page](https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346.md?page=2)