# Compatibility for numpy arrays **URL:** \ **Category:** 🤗Datasets\ **Created:** [April 7, 2021, 4:18pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351 "2021-04-07T16:18:11Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Neel-Gupta](https://avatars.discourse-cdn.com/v4/letter/n/f1d935/32.png) [@Neel-Gupta](https://discuss.huggingface.co/u/Neel-Gupta)\ **Post date:** [April 7, 2021, 4:18pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/1 "2021-04-07T16:18:11Z")
Is there any native compatibility in `datasets` to construct it from NumPy arrays to be further used in `transformers` without writing it to a file and loading it that way? ---
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [April 8, 2021, 12:12pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/2 "2021-04-08T12:12:48Z")
As far as I know there isn’t a native `Dataset.from_numpy` method, but you could map your array to a Python dictionary and use the `from_dict` method: [Loading a Dataset — datasets 1.5.0 documentation](https://huggingface.co/docs/datasets/loading_datasets.html?highlight=numpy#from-a-python-dictionary) ---
**Author:** ![Neel-Gupta](https://avatars.discourse-cdn.com/v4/letter/n/f1d935/32.png) [@Neel-Gupta](https://discuss.huggingface.co/u/Neel-Gupta)\ **Post date:** [April 8, 2021, 1:10pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/3 "2021-04-08T13:10:12Z")
Native support is always preferable, but I guess making it a `dict` is a good workaround 😉 Thanks for your reply!! ---
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [April 8, 2021, 1:17pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/4 "2021-04-08T13:17:47Z")
you can always open an issue on the `datasets` repo and provide some details on your use case / workflow 🙂 my guess is that numpy arrays are bit tricky compared to pandas dataframes because you need some way of encoding the column name for the arrow format ---
**Author:** ![Neel-Gupta](https://avatars.discourse-cdn.com/v4/letter/n/f1d935/32.png) [@Neel-Gupta](https://discuss.huggingface.co/u/Neel-Gupta)\ **Post date:** [April 8, 2021, 2:19pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/5 "2021-04-08T14:19:51Z")
Hmmm…I didn’t think about the column names 🤔 Maybe automatic inference (like sometimes present in some AutoML tools) could be done, where it would prompt the user of what it feels are the most appropriate names and allows the user to change it as well? A separate argument that takes a list containing column names? While all this may just be less-used features, it is more on the side of a beginner who knows `numpy` and finds that simple arguments are enough to use them with `transformers` instead of getting him to read the docs on possible workarounds 😉 ---
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [April 8, 2021, 3:35pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/6 "2021-04-08T15:35:40Z")
i had a quick look at how the `from_pandas` method works and you can see here that it’s calling `InMemoryTable.from_pandas`: [datasets/arrow\_dataset.py at 3fc744abbef13468fa1f42cf4738e5267234549b · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/blob/3fc744abbef13468fa1f42cf4738e5267234549b/src/datasets/arrow_dataset.py#L370) interestingly, `InMemoryTable` has a `from_arrays` method ([link](https://github.com/huggingface/datasets/blob/3fc744abbef13468fa1f42cf4738e5267234549b/src/datasets/table.py#L320)) so it might be not be much work to expose this to the `Dataset` class - i’d definitely open up an issue for this as a feature request! ---
**Author:** ![Neel-Gupta](https://avatars.discourse-cdn.com/v4/letter/n/f1d935/32.png) [@Neel-Gupta](https://discuss.huggingface.co/u/Neel-Gupta)\ **Post date:** [April 8, 2021, 4:28pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/7 "2021-04-08T16:28:33Z")
that would definitely be cool if it can be implemented!! most people like using NumPy arrays for its simplicity. Integrating into `datasets` would be nice 🥳 ---
**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\ **Post date:** [April 27, 2021, 5:48pm UTC](https://discuss.huggingface.co/t/compatibility-for-numpy-arrays/5351/8 "2021-04-27T17:48:54Z")
Feel free to use Dataset.from\_dict and pass a dict containing your numpy arrays 😉