# Error occurs when loading dataset with load\_dataset() **URL:** \ **Category:** Beginners\ **Created:** [December 7, 2023, 7:32am UTC](https://discuss.huggingface.co/t/error-occurs-when-loading-dataset-with-load-dataset/64903 "2023-12-07T07:32:43Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![eelearner](https://avatars.discourse-cdn.com/v4/letter/e/77aa72/32.png) [@eelearner](https://discuss.huggingface.co/u/eelearner)\ **Post date:** [December 7, 2023, 7:32am UTC](https://discuss.huggingface.co/t/error-occurs-when-loading-dataset-with-load-dataset/64903/1 "2023-12-07T07:32:43Z")
Hi all! I’m a beginner of hf and ai. When I tried to load dataset downloaded from hf, I met an error. Here is my code below. ```auto wiki_set = load_dataset("wikipedia", "20231101.en", split='train') ``` The error is ‘raise DatasetGenerationError(“An error occurred while generating the dataset”) from e datasets.builder.DatasetGenerationError: An error occurred while generating the dataset’. I searched online and found that load\_from\_disk() may work for me. However, the official document said load\_dataset() can be used to load dataset locally, I’d like to know how to modify my code can solve the error. Thanks ---
**Author:** ![mikehemberger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mikehemberger/32/28013_2.png) [@mikehemberger](https://discuss.huggingface.co/u/mikehemberger)\ **Post date:** [December 7, 2023, 11:48am UTC](https://discuss.huggingface.co/t/error-occurs-when-loading-dataset-with-load-dataset/64903/2 "2023-12-07T11:48:09Z")
Hi there @eelearner , Can you provide a link to the Wikipedia dataset that you’re referring to? The one I found: [wikipedia · Datasets at Hugging Face](https://huggingface.co/datasets/wikipedia) Seems to provide “ 20220301.en” subset only. > wiki\_set = load\_dataset(“wikipedia”, “20220301.en”, split=‘train’) Should work that case. Another case though would be if you have scraped Wikipedia more recently and want to load your scrape into hf dataset? For that you’d need to share more information. Best, M ---
**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\ **Post date:** [December 7, 2023, 5:52pm UTC](https://discuss.huggingface.co/t/error-occurs-when-loading-dataset-with-load-dataset/64903/3 "2023-12-07T17:52:57Z")
Hi! We plan to deprecate this dataset in the next few days. You should use ```python ds = load_dataset("wikimedia/wikipedia", "20231101.en", split="train") ``` instead. ---
**Author:** ![eelearner](https://avatars.discourse-cdn.com/v4/letter/e/77aa72/32.png) [@eelearner](https://discuss.huggingface.co/u/eelearner)\ **Post date:** [December 8, 2023, 3:08am UTC](https://discuss.huggingface.co/t/error-occurs-when-loading-dataset-with-load-dataset/64903/4 "2023-12-08T03:08:31Z")
Hi, this is the Wikipedia dataset I’m referring to: [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) Thx for help. I’ve found the problem that I downloaded the dataset using git but checkout failed. It means that I didn’t download the whole dataset properly. I solved this problem few minutes ago! Best, EElearner ---
**Author:** ![eelearner](https://avatars.discourse-cdn.com/v4/letter/e/77aa72/32.png) [@eelearner](https://discuss.huggingface.co/u/eelearner)\ **Post date:** [December 8, 2023, 3:10am UTC](https://discuss.huggingface.co/t/error-occurs-when-loading-dataset-with-load-dataset/64903/5 "2023-12-08T03:10:08Z")
Yeah. Thanks for reminding! ---
**Author:** ![Shawal777](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shawal777/32/29546_2.png) [@Shawal777](https://discuss.huggingface.co/u/Shawal777)\ **Post date:** [August 31, 2024, 4:22pm UTC](https://discuss.huggingface.co/t/error-occurs-when-loading-dataset-with-load-dataset/64903/6 "2024-08-31T16:22:56Z")
My issue was that my test split had null values. Filled these with a zero and it was solved