# NLP chapter 3 question

**URL:** <https://discuss.huggingface.co/t/nlp-chapter-3-question/148420>\
**Category:** Beginners\
**Created:** [April 1, 2025, 2:28pm UTC](https://discuss.huggingface.co/t/nlp-chapter-3-question/148420 "2025-04-01T14:28:15Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 1, 2025, 3:19pm UTC](https://discuss.huggingface.co/t/nlp-chapter-3-question/148420/2 "2025-04-01T15:19:41Z")

</div>

Apparently, there is a version incompatibility issue between Keras and TensorFlow that has been around for a long time. The solution differs for each version…

For more information, search for the version you want to use…

> <https://github.com/keras-team/keras/issues/19262>
>
> \`\`\`python
> import tensorflow as tf
> from datasets import load\_dataset
> from tran…sformers import AutoTokenizer, TFAutoModelForSequenceClassification, DataCollatorWithPadding
> from tensorflow.keras.optimizers import Adam
> from tensorflow.keras.optimizers.schedules import PolynomialDecay
> from tensorflow.keras.losses import SparseCategoricalCrossentropy
> 
> def prepare\_imdb\_dataset(tokenizer):
> """
> Prepares the IMDB dataset for training and validation.
> 
> Args:
> tokenizer: The tokenizer to use for text tokenization.
> 
> Returns:
> A tuple containing the tokenized training and validation datasets.
> """
> imdb = load\_dataset("imdb")
> train\_set = imdb\['train'\].map(lambda x: tokenizer(x\['text'\], truncation=True), batched=True)
> test\_set = imdb\['test'\].map(lambda x: tokenizer(x\['text'\], truncation=True), batched=True)
> return train\_set, test\_set
> 
> tokenizer = AutoTokenizer.from\_pretrained("distilbert-base-uncased")
> model = TFAutoModelForSequenceClassification.from\_pretrained("distilbert-base-uncased", num\_labels=2)
> 
> train\_set, test\_set = prepare\_imdb\_dataset(tokenizer)
> 
> data\_collator = DataCollatorWithPadding(tokenizer=tokenizer, return\_tensors="tf")
> 
> tf\_train\_dataset = train\_set.to\_tf\_dataset(
> columns=\["attention\_mask", "input\_ids"\],
> label\_cols=\["label"\],
> shuffle=True,
> collate\_fn=data\_collator,
> batch\_size=8,
> )
> 
> tf\_validation\_dataset = test\_set.to\_tf\_dataset(
> columns=\["attention\_mask", "input\_ids"\],
> label\_cols=\["label"\],
> shuffle=False,
> collate\_fn=data\_collator,
> batch\_size=8,
> )
> 
> batch\_size = 16
> num\_epochs = 1
> num\_train\_steps = len(tf\_train\_dataset) \* num\_epochs
> lr\_scheduler = PolynomialDecay(
> initial\_learning\_rate=5e-5, end\_learning\_rate=0.0, decay\_steps=num\_train\_steps
> )
> 
> optimizer = Adam(learning\_rate=lr\_scheduler)
> loss = SparseCategoricalCrossentropy(from\_logits=True)
> model.compile(optimizer=optimizer, loss=loss, metrics=\["accuracy"\])
> 
> model.fit(tf\_train\_dataset, validation\_data=tf\_validation\_dataset, epochs=5)
> 
> \`\`\`
> 
> \`\`\`
> Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFDistilBertForSequenceClassification: \['vocab\_layer\_norm.weight', 'vocab\_transform.weight', 'vocab\_projector.bias', 'vocab\_transform.bias', 'vocab\_layer\_norm.bias'\]
> \- This IS expected if you are initializing TFDistilBertForSequenceClassification from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
> \- This IS NOT expected if you are initializing TFDistilBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
> Some weights or buffers of the TF 2.0 model TFDistilBertForSequenceClassification were not initialized from the PyTorch model and are newly initialized: \['pre\_classifier.weight', 'pre\_classifier.bias', 'classifier.weight', 'classifier.bias'\]
> You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
> Map: 100%
>  25000/25000 \[00:23\<00:00, 1086.84 examples/s\]
> Map: 100%
>  25000/25000 \[00:20\<00:00, 1304.86 examples/s\]
> \---------------------------------------------------------------------------
> ValueError Traceback (most recent call last)
> \[\<ipython-input-17-ac80246ded67\>\](https://localhost:8080/#) in \<cell line: 55\>()
> 53 optimizer = Adam(learning\_rate=lr\_scheduler)
> 54 loss = SparseCategoricalCrossentropy(from\_logits=True)
> \---\> 55 model.compile(optimizer=optimizer, loss=loss, metrics=\["accuracy"\])
> 56 
> 57 model.fit(tf\_train\_dataset, validation\_data=tf\_validation\_dataset, epochs=5)
> 
> 2 frames
> \[/usr/local/lib/python3.10/dist-packages/tf\_keras/src/optimizers/\_\_init\_\_.py\](https://localhost:8080/#) in get(identifier, \*\*kwargs)
> 332 )
> 333 else:
> \--\> 334 raise ValueError(
> 335 f"Could not interpret optimizer identifier: {identifier}"
> 336 )
> 
> ValueError: Could not interpret optimizer identifier: \<keras.src.optimizers.adam.Adam object at 0x79d9071160e0\>
> \`\`\`

> [@Pretrain model not accepting optimizer](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209):
>
> For this code, model = TFAutoModelForSequenceClassification.from\_pretrained(“bert-base-cased”, num\_labels=3) model.compile( optimizer = tf.keras.optimizers.Adam(learning\_rate=5e-5) ) This gives me this error ValueError: Could not interpret optimizer identifier: \<keras.src.optimizers.adam.Adam object at 0x7e0d28e55fc0\> what to do? I am using google colab

> <https://stackoverflow.com/questions/50056356/could-not-interpret-optimizer-identifier-error-in-keras>

---

_[View the full topic](https://discuss.huggingface.co/t/nlp-chapter-3-question/148420)._