# Does huggingface support load raw text dataset from hdfs?

**URL:** <https://discuss.huggingface.co/t/does-huggingface-support-load-raw-text-dataset-from-hdfs/13233>\
**Category:** 🤗Datasets\
**Created:** [December 30, 2021, 5:57am UTC](https://discuss.huggingface.co/t/does-huggingface-support-load-raw-text-dataset-from-hdfs/13233 "2021-12-30T05:57:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pipi](https://avatars.discourse-cdn.com/v4/letter/p/e9bcb4/32.png) [@pipi](https://discuss.huggingface.co/u/pipi)\
**Post date:** [December 30, 2021, 5:57am UTC](https://discuss.huggingface.co/t/does-huggingface-support-load-raw-text-dataset-from-hdfs/13233/1 "2021-12-30T05:57:54Z")

</div>

I noticed that both **load\_from\_disk** and **save\_to\_disk** support the **‘fs’** parameters, but not **load\_dataset**.

I want to know if it is possible to load text data from **hdfs** via **load\_dataset**

Any suggestion will be appreciately.

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [January 5, 2022, 2:47pm UTC](https://discuss.huggingface.co/t/does-huggingface-support-load-raw-text-dataset-from-hdfs/13233/2 "2022-01-05T14:47:14Z")

</div>

Hi ! Currently `load_dataset` only supports reading files from local paths or HTTP. We may add support for other filesystems (hdfs, cloud storages…) later 🙂

---

<div class="post-metadata">

**Author:** ![pipi](https://avatars.discourse-cdn.com/v4/letter/p/e9bcb4/32.png) [@pipi](https://discuss.huggingface.co/u/pipi)\
**Post date:** [January 6, 2022, 5:39am UTC](https://discuss.huggingface.co/t/does-huggingface-support-load-raw-text-dataset-from-hdfs/13233/3 "2022-01-06T05:39:49Z")

</div>

Thanks for your reply, another question is Does **load\_from\_disk** support **stream** mode?

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [January 9, 2022, 9:16pm UTC](https://discuss.huggingface.co/t/does-huggingface-support-load-raw-text-dataset-from-hdfs/13233/4 "2022-01-09T21:16:18Z")

</div>

No it doesn’t, only lozd\_dataset supports streaming mode