# Use tf.data.Data with HuggingFace datasets

**URL:** <https://discuss.huggingface.co/t/use-tf-data-data-with-huggingface-datasets/4885>\
**Category:** 🤗Transformers\
**Created:** [March 22, 2021, 12:37pm UTC](https://discuss.huggingface.co/t/use-tf-data-data-with-huggingface-datasets/4885 "2021-03-22T12:37:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jominmathew](https://avatars.discourse-cdn.com/v4/letter/j/76d3ee/32.png) [@jominmathew](https://discuss.huggingface.co/u/jominmathew)\
**Post date:** [March 22, 2021, 12:37pm UTC](https://discuss.huggingface.co/t/use-tf-data-data-with-huggingface-datasets/4885/1 "2021-03-22T12:37:27Z")

</div>

I was going through this tutorial [Using a Dataset with PyTorch/Tensorflow — datasets 1.5.0 documentation](https://huggingface.co/docs/datasets/torch_tensorflow.html) .  
The example s for PyTorch.  
Do we have the same for Tensorflow?

---

<div class="post-metadata">

**Author:** ![eddie96](https://avatars.discourse-cdn.com/v4/letter/e/db5fbb/32.png) [@eddie96](https://discuss.huggingface.co/u/eddie96)\
**Post date:** [March 22, 2021, 1:45pm UTC](https://discuss.huggingface.co/t/use-tf-data-data-with-huggingface-datasets/4885/2 "2021-03-22T13:45:13Z")

</div>

Well there’s a section for tensorflow, on the top right corner there’s a split for tensorflow or pytorch, default is in pytorch

This is was took from the official documentation, this is for tensorflow btw

```python
>>> import tensorflow as tf
>>> from datasets import load_dataset
>>> from transformers import AutoTokenizer
>>> dataset = load_dataset('glue', 'mrpc', split='train')
>>> tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
>>> dataset = dataset.map(lambda e: tokenizer(e['sentence1'], truncation=True, padding='max_length'), batched=True)
>>>
>>> dataset.set_format(type='tensorflow', columns=['input_ids', 'token_type_ids', 'attention_mask', 'label'])
>>> features = {x: dataset[x].to_tensor(default_value=0, shape=[None, tokenizer.model_max_length]) for x in ['input_ids', 'token_type_ids', 'attention_mask']}
>>> tfdataset = tf.data.Dataset.from_tensor_slices((features, dataset["label"])).batch(32)
>>> next(iter(tfdataset))
({'input_ids': <tf.Tensor: shape=(32, 512), dtype=int32, numpy=
array([[ 101, 7277, 2180, ...,   

```

---

<div class="post-metadata">

**Author:** ![jominmathew](https://avatars.discourse-cdn.com/v4/letter/j/76d3ee/32.png) [@jominmathew](https://discuss.huggingface.co/u/jominmathew)\
**Post date:** [March 22, 2021, 2:08pm UTC](https://discuss.huggingface.co/t/use-tf-data-data-with-huggingface-datasets/4885/3 "2021-03-22T14:08:36Z")

</div>

Thanks alot 🙂