# Uploading dataset compressing images

**URL:** <https://discuss.huggingface.co/t/uploading-dataset-compressing-images/70756>\
**Category:** 🤗Datasets\
**Created:** [January 26, 2024, 7:58pm UTC](https://discuss.huggingface.co/t/uploading-dataset-compressing-images/70756 "2024-01-26T19:58:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![exibings](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/exibings/32/23321_2.png) [@exibings](https://discuss.huggingface.co/u/exibings)\
**Post date:** [January 26, 2024, 7:58pm UTC](https://discuss.huggingface.co/t/uploading-dataset-compressing-images/70756/1 "2024-01-26T19:58:48Z")

</div>

Hey! I’m having some trouble uploading a VQA dataset to the hub via python’s DatasetDict.push\_to\_hub(). Everything seems to work perfectly except the images, they seem to be converted from png to jpg.

The original dataset can be downloaded from here: [Remote Sensing VQA - Low Resolution (RSVQA LR)](https://zenodo.org/records/6344334)

Here’s the dataset: [exibings/rsvqa-lr · Datasets at Hugging Face](https://huggingface.co/datasets/exibings/rsvqa-lr)

For comparision I was using the MMMU dataset: [MMMU/MMMU · Datasets at Hugging Face](https://huggingface.co/datasets/MMMU/MMMU) and here the images are displayed as pngs. (You can try to download an image directly from the dataset viewer and they will download as png, if you try the same thing with my dataset you’ll notice that the images have low-res and download as jpg)

I’m unsure if it’s something I’m doing incorrectly when creating the DatasetDict that I upload.  
This is the code snippet of out I build each Dataset of the DatasetDict:

```auto
def process_split(questions: list, answers: list, split: str, dataset: Literal["lr", "hr"]) -> Dataset:
    records = {
        "type": [],
        "question": [],
        "img_id": [],
        "img": [],
        "answer": [],
    }

    for question, answer in zip(questions, answers):
        if question["active"] and answer["active"]:
            if question["type"] == "count" and dataset == "lr":
                records["answer"].append(count_func(answer["answer"]))
            elif question["type"] == "area" and dataset == "hr":
                records["answer"].append(area_func(answer["answer"]))
            else:
                records["answer"].append(answer["answer"])

            records["type"].append(question["type"])
            records["question"].append(question["question"])
            records["img_id"].append(question["img_id"])
            records["img"].append(os.path.join("data", f"rsvqa-{dataset}", "images", f"{question['img_id']}.tif"))
    return Dataset.from_dict(records, split=split).cast_column("img", Image())

```

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [January 29, 2024, 4:44pm UTC](https://discuss.huggingface.co/t/uploading-dataset-compressing-images/70756/2 "2024-01-29T16:44:24Z")

</div>

Hi ! The Hugging Face Hub may convert the images to another format to show them in your browser.

Under the hood the images in your dataset are still in their original format. Indeed `datasets` doesn’t convert images when pushing to the Hub.

---

<div class="post-metadata">

**Author:** ![exibings](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/exibings/32/23321_2.png) [@exibings](https://discuss.huggingface.co/u/exibings)\
**Post date:** [January 29, 2024, 5:00pm UTC](https://discuss.huggingface.co/t/uploading-dataset-compressing-images/70756/3 "2024-01-29T17:00:46Z")

</div>

Oh ok, maybe I should have tried to actually download it and check 😅😅

Anyhow, just for the sake of trying to learn. How come the MMMU dataset images are presented in .png format in the browser ? My guess was that it might be cause of the auto-parquet conversion (or lack of thereof in their case), not sure if that’s the case or not but would love to know more!

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [January 29, 2024, 6:26pm UTC](https://discuss.huggingface.co/t/uploading-dataset-compressing-images/70756/4 "2024-01-29T18:26:05Z")

</div>

Surely because the dataset was already in PNG, and in the browser we try to use either JPEG or PNG