# How do I create a Image Segmentation Dataset **URL:** \ **Category:** 🤗Datasets\ **Created:** [September 14, 2022, 9:09am UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123 "2022-09-14T09:09:23Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 14, 2022, 9:09am UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/1 "2022-09-14T09:09:23Z")
Hi Guys, I have dataset that has the base images, their segmentation masks, and also the labels, how do I create an HF dataset from this, so that I can use segmentation transformers. Please help. ---
**Author:** ![NimaBoscarino](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nimaboscarino/32/4358_2.png) [@NimaBoscarino](https://discuss.huggingface.co/u/NimaBoscarino)\ **Post date:** [September 14, 2022, 9:59am UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/2 "2022-09-14T09:59:20Z")
Welcome, @Archan! So the SegFormer model in Transformers is going to expect the features to be like the ones found in this dataset: [segments/sidewalk-semantic · Datasets at Hugging Face](https://huggingface.co/datasets/segments/sidewalk-semantic) Namely, you need `pixel_values`, which is just the image, and `label`, which is the segmentation map, which is one label per pixel. This blog post might be helpful: [Fine-Tune a Semantic Segmentation Model with a Custom Dataset](https://huggingface.co/blog/fine-tune-segformer) If you have N masks per image, for N labels) then I think you have to merge it into one segmentation map first. The blog post shows how you can do that with a masks made with Segments.ai, but if you created the masks some other way you can write a similar script to do the same thing. Hope this helps! ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 14, 2022, 12:04pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/3 "2022-09-14T12:04:44Z")
Hey @NimaBoscarino, I currently have 2 sets of images, one the original and another masked and then a CSV file having the label of the mask. There are only one mask per image. I have gone through the blog, it has used segments.ai which I don’t have access to nor I can as the data is private. Also I don’t need to separately mask it, I already have the masking I just want to convert it to a HF dataset format that’s it. ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 14, 2022, 12:14pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/4 "2022-09-14T12:14:42Z")
Hey @NimaBoscarino, I was thinking about using this builder code: ```auto import datasets import glob IMAGES = glob.glob("path/to/archive/dataset/semantic_drone_dataset/original_images/*.jpg") SEG_MAPS = glob.glob("path/to/archive/dataset/semantic_drone_dataset/label_images_semantic/*.png") dataset = datasets.Dataset.from_dict({"image": IMAGES, "label": SEG_MAPS}, features=datasets.Features({"image": datasets.Image(), "label": datasets.Image()})) ``` ---
**Author:** ![NimaBoscarino](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nimaboscarino/32/4358_2.png) [@NimaBoscarino](https://discuss.huggingface.co/u/NimaBoscarino)\ **Post date:** [September 14, 2022, 2:08pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/5 "2022-09-14T14:08:25Z")
Yeah I think that builder code is in the right direction! Looks like this documentation gives you the exact code that you need for it: [transformers/examples/pytorch/semantic-segmentation at main · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/tree/main/examples/pytorch/semantic-segmentation#note-on-custom-data) ---
**Author:** ![nateraw](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nateraw/32/2556_2.png) [@nateraw](https://discuss.huggingface.co/u/nateraw)\ **Post date:** [September 14, 2022, 2:24pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/6 "2022-09-14T14:24:30Z")
You can also give [ImageFolder with Metadata](https://huggingface.co/docs/datasets/image_load#imagefolder-with-metadata) a shot 🙂 if you want to host this dataset on the Hub. This is also nice if you had other features you wanted to include. ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 14, 2022, 4:13pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/7 "2022-09-14T16:13:42Z")
@nateraw Thank you will sure take a look. ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 14, 2022, 4:16pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/8 "2022-09-14T16:16:06Z")
@NimaBoscarino and @nateraw a very naive question, so we have processed the images and now every image represent one single mask, like Img: |- Img 1 |- Img 2 Mask: |- Mask for Img 1 |- Mask for Img 2 and these mask are for single classes, so now I have to create a json file which will have the number of image and then their respective mask. This part I am not understanding at all, as how will the model understand which mask is for which class. ---
**Author:** ![nateraw](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nateraw/32/2556_2.png) [@nateraw](https://discuss.huggingface.co/u/nateraw)\ **Post date:** [September 14, 2022, 4:26pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/9 "2022-09-14T16:26:43Z")
Do you have more or less than 255 classes? ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 14, 2022, 4:28pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/10 "2022-09-14T16:28:14Z")
7 classes and total 4K points each nearly 550 times ---
**Author:** ![nateraw](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nateraw/32/2556_2.png) [@nateraw](https://discuss.huggingface.co/u/nateraw)\ **Post date:** [September 14, 2022, 4:46pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/11 "2022-09-14T16:46:44Z")
You can construct a greyscale image by combining the masks, then use that image as a new feature called “label” using the `datasets.Image` feature type 🙂. If you save this new mask as a file and use Imagefolder metadata, referring to the mask path, it should cast to `datasets.Image` for ya out of the box. * * * Now to do that…here’s a vvv quick explanation Let’s say you have an image that’s shape is (1080, 720, 3). You could construct a mask of shape (1080, 720) where each value therein is `[0, num_classes)` denoting which object is in that pixel location. So if you had two masks: Cat ```auto 0 1 0 0 0 0 0 1 1 ``` and Dog ```auto 1 0 1 0 0 1 1 0 0 ``` Let’s also assume you want a background class to make it clearer…you’d construct `labels = ['background', 'cat', 'dog']` and make following array you could save as greyscale image ```auto 2 1 2 0 0 2 2 1 1 ``` You can see a dataset like that in the [segments AI](https://huggingface.co/datasets/segments/sidewalk-semantic) dataset Nima mentioned above. ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 15, 2022, 8:09am UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/12 "2022-09-15T08:09:03Z")
@nateraw @NimaBoscarino Hey thank you for the help, I was able to mask the dataset and construct a datasetdict from it. But now there is another small issue. ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/e/e0392f9a089eb187ce8010fc6549d162703f1987.jpeg) I am getting this error, and not sure why this is coming. I am trying with the base code only. ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 15, 2022, 8:09am UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/13 "2022-09-15T08:09:41Z")
![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/7/7fcd39e1e853d6f2b3ea7b7aaf42c1a49c768662.png) This is the function where it is throwing the error. ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [September 15, 2022, 11:58am UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/14 "2022-09-15T11:58:15Z")
This could have to do with the fact that the images aren’t converted to RGB before passing them to the feature extractor. Can you do `inputs = feature_extractor(["image.convert("RGB") for image in images], labels)`? ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 15, 2022, 1:23pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/15 "2022-09-15T13:23:31Z")
I was able to solve that part by doing this: ```auto feature_extractor = SegformerFeatureExtractor(do_resize=False, do_normalize=False) ``` But I am getting another error ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/2/268ebff1b04396e9e6e49d1277b0d66206437e8a.png) Not sure why this is coming ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 15, 2022, 1:40pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/16 "2022-09-15T13:40:06Z")
Even after asserting the Image class same error is being thrown ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/6/632df0af22052a62e14db4f9f9a476f5cca47e8f.png) 😓 ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 15, 2022, 2:35pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/17 "2022-09-15T14:35:07Z")
@nielsr I even tried with the segments dataset that you had shown in the demo, I got the same error ```auto RuntimeError: Could not infer dtype of Image ``` ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [September 15, 2022, 2:49pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/18 "2022-09-15T14:49:33Z")
As always, [become one with the data](http://karpathy.github.io/2019/04/25/recipe/). Make sure the data is prepared in the right way. You can do the following things: - print examples of your dataset - create a PyTorch DataLoader and check whether the data is created appropriately ---
**Author:** ![Archan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/archan/32/28312_2.png) [@Archan](https://discuss.huggingface.co/u/Archan)\ **Post date:** [September 15, 2022, 3:01pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/19 "2022-09-15T15:01:13Z")
So when I tried with your dataset I did the same, checked if was missing any steps, but IDK why I got the same error. in case of the segments dataset: pixel\_values looks like this : \ label looks like: \ and my dataset looks like pixel\_values: \ label : \ I am genuinely lost at what is going wrong, I have tried both declaring as a dataset dict as well as using a datasetloader the dataset looks same hence I am getting the same error ---
**Author:** ![fredguth](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/fredguth/32/2843_2.png) [@fredguth](https://discuss.huggingface.co/u/fredguth)\ **Post date:** [September 29, 2022, 5:57pm UTC](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123/20 "2022-09-29T17:57:56Z")
I am also trying to create an image segmentation dataset and I am struggling with how to do that. The dataset I want to upload to HF is the [aisegmentcn-matting-human datasets](https://github.com/aisegmentcn/matting_human_datasets), which you also can find at [Kaggle](https://www.kaggle.com/datasets/laurentmih/aisegmentcom-matting-human-datasets). I also have the dataset in my machine in a folder named data: ```auto └── data/ ├── clip_img/ │ └── {group-id}/ │ └── clip_{subgroup-id}/ │ └── {group-id}-{img-id}.jpg └── matting/ └── {group-id}/ └── matting_{subgroup-id}/ └── {group-id}-{img-id}.png ``` all matting images are of the kind {group-id}-{img-id}.png are exactly equal to the input image {group-id}-{img-id}.jpg but with the background removed. So, if labels are `background` and `foreground`, if the color is 0 it is background, anything else is foreground. Just 2 classes of labels. The primary reason I want this dataset in HF is to use it in a HF space and notebook. It seems I have to process the raw data into an expected format. If I just try to: ```python >>> from datasets import load_from_disk >>> ds = load_from_disk('./data') ... FileNotFoundError: Directory data is neither a dataset directory nor a dataset dict directory. ``` So, I guess I have to create a DatasetDict with `image` and `label` columns, right? Is there a How To guide for this kind of image dataset? [Next page](https://discuss.huggingface.co/t/how-do-i-create-a-image-segmentation-dataset/23123.md?page=2)