# Dataset map occasionally throws error

**URL:** <https://discuss.huggingface.co/t/dataset-map-occasionally-throws-error/67814>\
**Category:** 🤗Datasets\
**Created:** [January 3, 2024, 1:14pm UTC](https://discuss.huggingface.co/t/dataset-map-occasionally-throws-error/67814 "2024-01-03T13:14:46Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rlpatrao](https://avatars.discourse-cdn.com/v4/letter/r/b487fb/32.png) [@rlpatrao](https://discuss.huggingface.co/u/rlpatrao)\
**Post date:** [January 3, 2024, 1:14pm UTC](https://discuss.huggingface.co/t/dataset-map-occasionally-throws-error/67814/1 "2024-01-03T13:14:46Z")

</div>

Hello, I am getting this error occasionally while maping. Any ideas why and/or how to debug this ? Is this a issue ?

```auto
Map: 14%|█████████▏ | 999/7030 [6:06:58<36:55:23, 22.04s/ examples]
Traceback (most recent call last):
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3452, in _map_single
    writer.write(example)
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_writer.py", line 491, in write
    self.write_examples_on_file()
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_writer.py", line 445, in write_examples_on_file
    batch_examples[col] = [
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_writer.py", line 446, in <listcomp>
    row[0][col].to_pylist()[0] if isinstance(row[0][col], (pa.Array, pa.ChunkedArray)) else row[0][col]
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/user/workspace/genlaw/model/data_curation.py", line 189, in <module>
    DataCurator().create_training_data()
  File "/Users/user/workspace/genlaw/model/data_curation.py", line 113, in create_training_data
    ds_train = ds_train.map(self.summarize, with_indices=True)
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 591, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 556, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3089, in map
    for rank, done, content in Dataset._map_single(**dataset_kwargs):
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3497, in _map_single
    writer.finalize()
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_writer.py", line 587, in finalize
    self.write_examples_on_file()
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_writer.py", line 445, in write_examples_on_file
    batch_examples[col] = [
  File "/Users/user/miniconda3/envs/gen310/lib/python3.10/site-packages/datasets/arrow_writer.py", line 446, in <listcomp>
    row[0][col].to_pylist()[0] if isinstance(row[0][col], (pa.Array, pa.ChunkedArray)) else row[0][col]
TypeError: 'NoneType' object is not subscriptable

```

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\
**Post date:** [January 15, 2024, 7:52pm UTC](https://discuss.huggingface.co/t/dataset-map-occasionally-throws-error/67814/2 "2024-01-15T19:52:49Z")

</div>

Hi! Can you share the map transform’s code?