# 🤗Datasets **URL:** https://discuss.huggingface.co/c/datasets/10.md [Latest](https://discuss.huggingface.co/latest.md) · [Categories](https://discuss.huggingface.co/categories.md) --- ## [Looking for one independent annotator: our LLM raters agree with each other at chance level and we can't tell why](https://discuss.huggingface.co/t/looking-for-one-independent-annotator-our-llm-raters-agree-with-each-other-at-chance-level-and-we-cant-tell-why/180645)
**Author:** [@leventbulut](https://discuss.huggingface.co/u/leventbulut)\ **Replies:** 8\ **Last updated:** [October 9, 2026, 8:34pm UTC](https://discuss.huggingface.co/t/looking-for-one-independent-annotator-our-llm-raters-agree-with-each-other-at-chance-level-and-we-cant-tell-why/180645 "2026-10-09T20:34:46Z")
Short version of the ask: I have 100 scenes, a locked set of human reference labels, and five machine raters that disagree with the human and with each other at roughly chance level. I cannot tell whether that is because… --- ## [Datasets \>= 5 ruins sharding when shuffle() is called](https://discuss.huggingface.co/t/datasets-5-ruins-sharding-when-shuffle-is-called/179493)
**Author:** [@joell1](https://discuss.huggingface.co/u/joell1)\ **Replies:** 4\ **Last updated:** [October 8, 2026, 8:28pm UTC](https://discuss.huggingface.co/t/datasets-5-ruins-sharding-when-shuffle-is-called/179493 "2026-10-08T20:28:19Z")
Hi HF community I am using a streaming dataset with a training dataset that is sharded into 8 different parquet files. Before datasets 5.0, when I would call shuffle() with the dataset, there would be 8 shards in the re… --- ## [Download a fraction of data from HuggingFace Datasets](https://discuss.huggingface.co/t/download-a-fraction-of-data-from-huggingface-datasets/126236)
**Author:** [@srinjoyMukherjee](https://discuss.huggingface.co/u/srinjoyMukherjee)\ **Replies:** 5\ **Last updated:** [October 3, 2026, 10:33pm UTC](https://discuss.huggingface.co/t/download-a-fraction-of-data-from-huggingface-datasets/126236 "2026-10-03T22:33:01Z")
hey all, I want to download about 15GBs of data from each subsection/language of the stack dataset. Is there any way to do this? I can download the entire subsection using following code snippet, but i want a fraction o… --- ## [Confirmation: Can I use HuggingFace datasets for academic ML training?](https://discuss.huggingface.co/t/confirmation-can-i-use-huggingface-datasets-for-academic-ml-training/182807)
**Author:** [@Fitzz007](https://discuss.huggingface.co/u/Fitzz007)\ **Replies:** 1\ **Last updated:** [October 3, 2026, 1:31am UTC](https://discuss.huggingface.co/t/confirmation-can-i-use-huggingface-datasets-for-academic-ml-training/182807 "2026-10-03T01:31:01Z")
Hello HuggingFace community, My name is Mohammed Talbani, a Computer Science student at ESTS in Safi, Morocco. I am completing an AI internship where I need to build an image dataset of traditional Arabic footwear (babo… --- ## [Dataset & LoRA Training Pipeline for Room Emptying / Object Removal (Image-to-Image)](https://discuss.huggingface.co/t/dataset-lora-training-pipeline-for-room-emptying-object-removal-image-to-image/180659)
**Author:** [@suruchiarora](https://discuss.huggingface.co/u/suruchiarora)\ **Replies:** 2\ **Last updated:** [October 1, 2026, 9:33am UTC](https://discuss.huggingface.co/t/dataset-lora-training-pipeline-for-room-emptying-object-removal-image-to-image/180659 "2026-10-01T09:33:59Z")
Hi Community, I am working on a project aimed at emptying indoor spaces (removing all furniture/decor) as well as removing specific individual objects from an image using provided object masks. I plan to fine-tune a mo… --- ## [Is selling datasets way harder than building them? Or is it just me?](https://discuss.huggingface.co/t/is-selling-datasets-way-harder-than-building-them-or-is-it-just-me/177440)
**Author:** [@Soloise](https://discuss.huggingface.co/u/Soloise)\ **Replies:** 3\ **Last updated:** [September 28, 2026, 6:08pm UTC](https://discuss.huggingface.co/t/is-selling-datasets-way-harder-than-building-them-or-is-it-just-me/177440 "2026-09-28T18:08:06Z")
Is selling datasets way harder than building them? Or is it just me? Maybe I’m thinking wrong, but I keep running into this. Let’s say you spent months building a really good dataset. Now you want to sell it. The buy… --- ## [Reliability check on my own dataset's annotation layer: five machine raters, one definition, answers from 0 to 78](https://discuss.huggingface.co/t/reliability-check-on-my-own-datasets-annotation-layer-five-machine-raters-one-definition-answers-from-0-to-78/178375)
**Author:** [@leventbulut](https://discuss.huggingface.co/u/leventbulut)\ **Replies:** 2\ **Last updated:** [September 28, 2026, 4:44pm UTC](https://discuss.huggingface.co/t/reliability-check-on-my-own-datasets-annotation-layer-five-machine-raters-one-definition-answers-from-0-to-78/178375 "2026-09-28T16:44:44Z")
I publish a small Turkish narrative corpus that ships a machine-generated annotation layer — six binary craft features per scene. Those flags were never validated against human judgement, so I ran three studies to check … --- ## [Can someone with a Chinese Baidu NetDisk account help me download this dataset?](https://discuss.huggingface.co/t/can-someone-with-a-chinese-baidu-netdisk-account-help-me-download-this-dataset/160296)
**Author:** [@tferhan](https://discuss.huggingface.co/u/tferhan)\ **Replies:** 7\ **Last updated:** [September 23, 2026, 11:39am UTC](https://discuss.huggingface.co/t/can-someone-with-a-chinese-baidu-netdisk-account-help-me-download-this-dataset/160296 "2026-09-23T11:39:14Z")
I’m working on a real-world satellite imagery segmentation project. The dataset is UBCv2 (vec), hosted on Baidu NetDisk, and I’m blocked from downloading it due to the Chinese phone number restriction. Link: https://pan… --- ## [Hausa Healthcare LLM Preference & Safety Evaluation Dataset](https://discuss.huggingface.co/t/hausa-healthcare-llm-preference-safety-evaluation-dataset/180613)
**Author:** [@abdulfatah740](https://discuss.huggingface.co/u/abdulfatah740)\ **Replies:** 3\ **Last updated:** [September 19, 2026, 6:04pm UTC](https://discuss.huggingface.co/t/hausa-healthcare-llm-preference-safety-evaluation-dataset/180613 "2026-09-19T18:04:00Z")
Hello Huggingface Community, I am a computational linguist and Hausa NLP/AI data specialist working on a Hausa LLM Healthcare & Safety Preference Evaluation Dataset. The dataset is developed collaboratively by computat… --- ## [FlyEye escape-neuron-v1: 3,376-neuron / 78,797-edge compact MaleCNS-derived graph](https://discuss.huggingface.co/t/flyeye-escape-neuron-v1-3-376-neuron-78-797-edge-compact-malecns-derived-graph/180617)
**Author:** [@wihaha](https://discuss.huggingface.co/u/wihaha)\ **Replies:** 3\ **Last updated:** [September 18, 2026, 6:40pm UTC](https://discuss.huggingface.co/t/flyeye-escape-neuron-v1-3-376-neuron-78-797-edge-compact-malecns-derived-graph/180617 "2026-09-18T18:40:43Z")
We have published the first FlyEye compact neuron-level dataset together with the browser/runtime developer kit. Dataset: Tubban/flyeye-escape-neuron-v1 · Datasets at Hugging Face It contains a 3,376-neuron / 78,797 si… --- ## [Access to the gated repository, StanfordAIMI/CheXpert-v1.0-512](https://discuss.huggingface.co/t/access-to-the-gated-repository-stanfordaimi-chexpert-v1-0-512/180567)
**Author:** [@Ephy3897](https://discuss.huggingface.co/u/Ephy3897)\ **Replies:** 3\ **Last updated:** [September 18, 2026, 4:18pm UTC](https://discuss.huggingface.co/t/access-to-the-gated-repository-stanfordaimi-chexpert-v1-0-512/180567 "2026-09-18T16:18:09Z")
I kindly request access to the gated repository, StanfordAIMI/CheXpert-v1.0-512 on Hugging Face. I intend to use the dataset for my ongoing research on artificial intelligence in medical imaging. I have made a request,… --- ## [Open call: test your agent memory layer on an adversarial coding benchmark](https://discuss.huggingface.co/t/open-call-test-your-agent-memory-layer-on-an-adversarial-coding-benchmark/179762)
**Author:** [@Gde05](https://discuss.huggingface.co/u/Gde05)\ **Replies:** 5\ **Last updated:** [September 17, 2026, 9:36am UTC](https://discuss.huggingface.co/t/open-call-test-your-agent-memory-layer-on-an-adversarial-coding-benchmark/179762 "2026-09-17T09:36:42Z")
I built AMB, an open and preregistered benchmark for memory layers in coding agents. Most memory benchmarks test whether a system retrieves a relevant chunk. AMB tests whether that memory actually helps an agent complet… --- ## [Finding a working mirror /copy of the PxSLU/PxCorpus dataset beacause Zenodo keeps returning a 504 Gateway Time-out](https://discuss.huggingface.co/t/finding-a-working-mirror-copy-of-the-pxslu-pxcorpus-dataset-beacause-zenodo-keeps-returning-a-504-gateway-time-out/180350)
**Author:** [@OGB2000](https://discuss.huggingface.co/u/OGB2000)\ **Replies:** 2\ **Last updated:** [September 13, 2026, 2:27pm UTC](https://discuss.huggingface.co/t/finding-a-working-mirror-copy-of-the-pxslu-pxcorpus-dataset-beacause-zenodo-keeps-returning-a-504-gateway-time-out/180350 "2026-09-13T14:27:45Z")
Looking for someone who has successfully downloaded PxSLU / PxCorpus (Zenodo 6524162). Zenodo currently returns 504 Gateway Time-out when I try to download pxslu.zip (375.5 MB). Has anyone already downloaded this dataset… --- ## [Open-source tool for detecting issues in robot-learning datasets](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602)
**Author:** [@omert27](https://discuss.huggingface.co/u/omert27)\ **Replies:** 5\ **Last updated:** [September 12, 2026, 4:50pm UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602 "2026-09-12T16:50:33Z")
Open-source tool for robot dataset issues I’ve been working on Calibra, an open-source toolkit for analyzing robot-learning datasets. While working with demo data, I noticed a common issue: we often waste compute train… --- ## [Trajlens: a validator for LeRobotDataset, audited 100 Hub datasets](https://discuss.huggingface.co/t/trajlens-a-validator-for-lerobotdataset-audited-100-hub-datasets/177276)
**Author:** [@Kunal-Somani](https://discuss.huggingface.co/u/Kunal-Somani)\ **Replies:** 1\ **Last updated:** [September 9, 2026, 4:06pm UTC](https://discuss.huggingface.co/t/trajlens-a-validator-for-lerobotdataset-audited-100-hub-datasets/177276 "2026-09-09T16:06:42Z")
Posting here because this is directly about Hub data quality. LeRobotDataset is now the largest dataset category on the Hub. I built trajlens, an open-source linter for it, and ran it against 100 random public lerobot-t… --- ## [Best Practices for Large-Scale Image Datasets? (between WebDataset and Parquet)](https://discuss.huggingface.co/t/best-practices-for-large-scale-image-datasets-between-webdataset-and-parquet/139293)
**Author:** [@ryan-minato](https://discuss.huggingface.co/u/ryan-minato)\ **Replies:** 4\ **Last updated:** [September 6, 2026, 2:50pm UTC](https://discuss.huggingface.co/t/best-practices-for-large-scale-image-datasets-between-webdataset-and-parquet/139293 "2026-09-06T14:50:08Z")
I am importing an image dataset from an external source that is several terabytes in size. Based on my research, WebDataset and Parquet seem to be the best choices. What are the advantages and disadvantages between WebD… --- ## [\[Commercial Dataset\] 11,000 Hours of Production-Ready Egyptian & Emirati Call Center Audio (Dual-Channel, 48kHz/16-bit)](https://discuss.huggingface.co/t/commercial-dataset-11-000-hours-of-production-ready-egyptian-emirati-call-center-audio-dual-channel-48khz-16-bit/179907)
**Author:** [@sahar-nomaddesk](https://discuss.huggingface.co/u/sahar-nomaddesk)\ **Replies:** 0\ **Last updated:** [September 4, 2026, 10:42pm UTC](https://discuss.huggingface.co/t/commercial-dataset-11-000-hours-of-production-ready-egyptian-emirati-call-center-audio-dual-channel-48khz-16-bit/179907 "2026-09-04T22:42:11Z")
Title: \[Commercial Dataset\] 11,000 Hours of Production-Ready Egyptian & Emirati Call Center Audio (Dual-Channel, 48kHz/16-bit) Hi community, For teams and labs working on Arabic Speech-to-Text (ASR) and Voice AI models… --- ## [Save a dataset while it's being created by from\_generator()](https://discuss.huggingface.co/t/save-a-dataset-while-its-being-created-by-from-generator/179254)
**Author:** [@maliced](https://discuss.huggingface.co/u/maliced)\ **Replies:** 2\ **Last updated:** [September 4, 2026, 9:12am UTC](https://discuss.huggingface.co/t/save-a-dataset-while-its-being-created-by-from-generator/179254 "2026-09-04T09:12:00Z")
I am creating a dataset by streaming another one and processing each sample one by one in a generator function used by Dataset.from\_generator(). It is very time consuming and I can’t wait until the end to save it, I alwa… --- ## [\[Quota Increase Request\] Academic Dataset Storage for \[TUTTI\]](https://discuss.huggingface.co/t/quota-increase-request-academic-dataset-storage-for-tutti/178570)
**Author:** [@pzzzzz](https://discuss.huggingface.co/u/pzzzzz)\ **Replies:** 3\ **Last updated:** [September 3, 2026, 7:21pm UTC](https://discuss.huggingface.co/t/quota-increase-request-academic-dataset-storage-for-tutti/178570 "2026-09-03T19:21:34Z")
Hi Hugging Face Team, We are preparing to release an open-source dataset and model weights for our upcoming academic paper which will be published in the ISMIR2026 conference. HF Username / Org: pzzzzz Estimated S… --- ## [JPEG XL images in the dataset](https://discuss.huggingface.co/t/jpeg-xl-images-in-the-dataset/179448)
**Author:** [@stablellama](https://discuss.huggingface.co/u/stablellama)\ **Replies:** 1\ **Last updated:** [August 30, 2026, 3:15pm UTC](https://discuss.huggingface.co/t/jpeg-xl-images-in-the-dataset/179448 "2026-08-30T15:15:36Z")
Due to the great compression (including lossless!) I want and do store my images as JPEG XL. The browser support is now also kicking in. So, please support JXL as a native format of the dataset as well. Or is this alre… --- ## [Dataset Intelligence for Robotics](https://discuss.huggingface.co/t/dataset-intelligence-for-robotics/179329)
**Author:** [@omert27](https://discuss.huggingface.co/u/omert27)\ **Replies:** 1\ **Last updated:** [August 28, 2026, 10:01pm UTC](https://discuss.huggingface.co/t/dataset-intelligence-for-robotics/179329 "2026-08-28T22:01:32Z")
Hello People, I built Calibra , it aims for catching bad demonstrations before they waste GPU time then selecting the episodes that actually matter. here is the github repo : GitHub - omertt27/Calibra: Dataset observa… --- ## [What are the best practices for detecting and fetching deltas from a dataset?](https://discuss.huggingface.co/t/what-are-the-best-practices-for-detecting-and-fetching-deltas-from-a-dataset/177360)
**Author:** [@merveguel](https://discuss.huggingface.co/u/merveguel)\ **Replies:** 4\ **Last updated:** [August 28, 2026, 10:52am UTC](https://discuss.huggingface.co/t/what-are-the-best-practices-for-detecting-and-fetching-deltas-from-a-dataset/177360 "2026-08-28T10:52:14Z")
Hi gurus, What are the best practices for detecting and fetching deltas from a dataset? We’re currently using a very large dataset from Hugging Face and would like to trigger our pipeline whenever new data is added or … --- ## [What are the biggest challenges you face when sourcing high-quality, domain-specific datasets for training and evaluating AI models?](https://discuss.huggingface.co/t/what-are-the-biggest-challenges-you-face-when-sourcing-high-quality-domain-specific-datasets-for-training-and-evaluating-ai-models/178544)
**Author:** [@viharshah85](https://discuss.huggingface.co/u/viharshah85)\ **Replies:** 1\ **Last updated:** [August 28, 2026, 8:53am UTC](https://discuss.huggingface.co/t/what-are-the-biggest-challenges-you-face-when-sourcing-high-quality-domain-specific-datasets-for-training-and-evaluating-ai-models/178544 "2026-08-28T08:53:25Z")
I’d be particularly interested in experiences with multimodal data (text, audio, image, video), multilingual datasets, and human-annotated data—and what criteria you use to determine whether a dataset is reliable enough … --- ## [Complete personal archive of Tsiolkovsky (51k sheets) — with recognition accuracy actually measured](https://discuss.huggingface.co/t/complete-personal-archive-of-tsiolkovsky-51k-sheets-with-recognition-accuracy-actually-measured/179228)
**Author:** [@vladimirbesk](https://discuss.huggingface.co/u/vladimirbesk)\ **Replies:** 0\ **Last updated:** [August 25, 2026, 4:30am UTC](https://discuss.huggingface.co/t/complete-personal-archive-of-tsiolkovsky-51k-sheets-with-recognition-accuracy-actually-measured/179228 "2026-08-25T04:30:08Z")
I’ve released the complete personal archive of Konstantin Tsiolkovsky as a dataset: 51,008 sheets, 2,019 archival files, CC0. :link: vladimirbesk/tsiolkovsky-papers · Datasets at Hugging Face … --- ## [New Dataset Preview - Celtic Constellation Reference Sessions — aligned ensemble stems for Celtic instruments](https://discuss.huggingface.co/t/new-dataset-preview-celtic-constellation-reference-sessions-aligned-ensemble-stems-for-celtic-instruments/179224)
**Author:** [@Admin-Harmonic-Front](https://discuss.huggingface.co/u/Admin-Harmonic-Front)\ **Replies:** 0\ **Last updated:** [August 25, 2026, 2:43am UTC](https://discuss.huggingface.co/t/new-dataset-preview-celtic-constellation-reference-sessions-aligned-ensemble-stems-for-celtic-instruments/179224 "2026-08-25T02:43:03Z")
We’ve published a preview of the Celtic Constellation Reference Sessions dataset: aligned multitrack recordings of traditional and contemporary Celtic ensemble arrangements. The architecture is aligned pairs. Each piece… --- ## [Open 78-card tarot meaning dataset for symbolic NLP and multilingual experiments](https://discuss.huggingface.co/t/open-78-card-tarot-meaning-dataset-for-symbolic-nlp-and-multilingual-experiments/179105)
**Author:** [@FoodSecuriry](https://discuss.huggingface.co/u/FoodSecuriry)\ **Replies:** 1\ **Last updated:** [August 23, 2026, 11:07am UTC](https://discuss.huggingface.co/t/open-78-card-tarot-meaning-dataset-for-symbolic-nlp-and-multilingual-experiments/179105 "2026-08-23T11:07:18Z")
The dataset contains one row per card, with fields for the card name, arcana, suit, element, upright meaning, reversed meaning, love context, career context, yes or no label, zodiac association, and an extended source UR… --- ## [A free, monthly-refreshed compliance matrix of all 4,893 Indic-language datasets on the Hub, showing that 65% declare no license — so teams can avoid licensing traps (and missing-tag repos) before they train on them](https://discuss.huggingface.co/t/a-free-monthly-refreshed-compliance-matrix-of-all-4-893-indic-language-datasets-on-the-hub-showing-that-65-declare-no-license-so-teams-can-avoid-licensing-traps-and-missing-tag-repos-before-they-train-on-them/178373)
**Author:** [@hardik90](https://discuss.huggingface.co/u/hardik90)\ **Replies:** 2\ **Last updated:** [August 23, 2026, 5:47am UTC](https://discuss.huggingface.co/t/a-free-monthly-refreshed-compliance-matrix-of-all-4-893-indic-language-datasets-on-the-hub-showing-that-65-declare-no-license-so-teams-can-avoid-licensing-traps-and-missing-tag-repos-before-they-train-on-them/178373 "2026-08-23T05:47:49Z")
Dataset: 65% of 4,893 Indic-language datasets declare no license — full compliance matrix I scanned every Indic-language dataset on the Hub (hindi, tamil, bengali, telugu, marathi, malayalam, kannada, urdu, +9 more) and … --- ## [Script-Generated Visual Datasets with Reproducible Metadata and Adobe Generator Tools](https://discuss.huggingface.co/t/script-generated-visual-datasets-with-reproducible-metadata-and-adobe-generator-tools/178857)
**Author:** [@JdwFoto](https://discuss.huggingface.co/u/JdwFoto)\ **Replies:** 1\ **Last updated:** [August 20, 2026, 10:55pm UTC](https://discuss.huggingface.co/t/script-generated-visual-datasets-with-reproducible-metadata-and-adobe-generator-tools/178857 "2026-08-20T22:55:14Z")
Hello everyone, My name is Jonathan Welch. I’m a multimedia visual data designer with more than 25 years of experience in media, photography, design, and stock advertising. I develop script-generated visual datasets us… --- ## [Why is there almost no manipulation data for agriculture?](https://discuss.huggingface.co/t/why-is-there-almost-no-manipulation-data-for-agriculture/178257)
**Author:** [@viharshah85](https://discuss.huggingface.co/u/viharshah85)\ **Replies:** 2\ **Last updated:** [August 20, 2026, 4:00am UTC](https://discuss.huggingface.co/t/why-is-there-almost-no-manipulation-data-for-agriculture/178257 "2026-08-20T04:00:18Z")
I farm, and I’ve been going through the robotics datasets on the Hub for a few months trying to find anything useful. Almost everything is indoor tabletop. Open X-Embodiment, DROID, most of what’s under LeRobot — pick up… --- ## [UncertaintyGym: A Benchmark for LLM Epistemic Calibration and Uncertainty Expression](https://discuss.huggingface.co/t/uncertaintygym-a-benchmark-for-llm-epistemic-calibration-and-uncertainty-expression/178647)
**Author:** [@Ill-Ness](https://discuss.huggingface.co/u/Ill-Ness)\ **Replies:** 4\ **Last updated:** [August 18, 2026, 2:32pm UTC](https://discuss.huggingface.co/t/uncertaintygym-a-benchmark-for-llm-epistemic-calibration-and-uncertainty-expression/178647 "2026-08-18T14:32:52Z")
Hi Hugging Face team, We would like to submit UncertaintyGym for official benchmark indexing and Evaluation Hub integration. Overview: UncertaintyGym evaluates an LLM’s meta-cognitive calibration—measuring whether mod… [Next page](https://discuss.huggingface.co/c/datasets/10.md?page=1)