# Error in Model.prepare\_tf\_dataset()

**URL:** <https://discuss.huggingface.co/t/error-in-model-prepare-tf-dataset/45406>\
**Category:** 🤗Transformers\
**Created:** [July 3, 2023, 2:05am UTC](https://discuss.huggingface.co/t/error-in-model-prepare-tf-dataset/45406 "2023-07-03T02:05:17Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![raygx](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/raygx/32/17328_2.png) [@raygx](https://discuss.huggingface.co/u/raygx)\
**Post date:** [July 3, 2023, 2:05am UTC](https://discuss.huggingface.co/t/error-in-model-prepare-tf-dataset/45406/1 "2023-07-03T02:05:18Z")

</div>

##Data used is " lotusacharya/nepalinewsdataset " from kaggle.  
##Code:  
%%time

def preprocess\_function(rows):  
return tokenizer(rows[‘text’],truncation=True,max\_length=128,padding=True)

print(“Tokenizing the data”)  
tokenized\_inputs = data.map(  
preprocess\_function,  
batched=True,  
num\_proc=2,  
remove\_columns=data[“train”].column\_names,  
)

tokenized\_inputs[‘train’] = tokenized\_inputs[‘train’].add\_column(  
name=“labels”,column=data[‘train’][‘label’]  
)  
tokenized\_inputs[‘test’] = tokenized\_inputs[‘test’].add\_column(  
name=“labels”,column=data[‘test’][‘label’]  
)

tokenized\_inputs

from transformers import DataCollatorWithPadding

print(“Initializing Data Collator”)  
data\_collator = DataCollatorWithPadding(tokenizer=tokenizer,  
max\_length=256,  
return\_tensors=“tf”)

print(“Preparing Training and Testing sets to TRAIN the MODEL”)  
tf\_train\_set = model.prepare\_tf\_dataset(  
tokenized\_inputs[“train”],  
shuffle=True,  
batch\_size=16,  
collate\_fn=data\_collator,  
)

tf\_test\_set = model.prepare\_tf\_dataset(  
tokenized\_inputs[“test”],  
shuffle=False,  
batch\_size=16,  
collate\_fn=data\_collator,  
)

##Error:  
Preparing Training and Testing sets to TRAIN the MODEL

/usr/local/lib/python3.10/dist-packages/transformers/tokenization\_utils\_base.py:2395: UserWarning: `max_length` is ignored when `padding`=`True` and there is no truncation strategy. To pad to max length, use `padding='max_length'`.  
warnings.warn(

* * *

RuntimeError Traceback (most recent call last)

in \<cell line: 2\>()  
1 print(“Preparing Training and Testing sets to TRAIN the MODEL”)  
----\> 2 tf\_train\_set = model.prepare\_tf\_dataset(  
3 tokenized\_inputs[“train”],  
4 shuffle=True,  
5 batch\_size=16,

1 frames

/usr/local/lib/python3.10/dist-packages/datasets/arrow\_dataset.py in \_get\_output\_signature(dataset, collate\_fn, collate\_fn\_args, cols\_to\_retain, batch\_size, num\_test\_batches)  
289 tf\_dtype = tf.string  
290 else:  
 → 291 raise RuntimeError(  
292 f"Unrecognized array dtype {np\_arrays[0].dtype}. \n"  
293 “Nested types and image/audio types are not supported yet.”

RuntimeError: Unrecognized array dtype object.  
Nested types and image/audio types are not supported yet.

---

<div class="post-metadata">

**Author:** ![raygx](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/raygx/32/17328_2.png) [@raygx](https://discuss.huggingface.co/u/raygx)\
**Post date:** [July 5, 2023, 12:29am UTC](https://discuss.huggingface.co/t/error-in-model-prepare-tf-dataset/45406/2 "2023-07-05T00:29:22Z")

</div>

I realized I forgot to encode labels and was passing in the text labels…  
Noob mistake… all good now  
can be closed