# Pretrain model not accepting optimizer **URL:** \ **Category:** 🤗Transformers\ **Created:** [March 6, 2024, 1:42pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209 "2024-03-06T13:42:56Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![GunsRuth](https://avatars.discourse-cdn.com/v4/letter/g/77aa72/32.png) [@GunsRuth](https://discuss.huggingface.co/u/GunsRuth)\ **Post date:** [March 6, 2024, 1:42pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/1 "2024-03-06T13:42:56Z")
For this code, model = TFAutoModelForSequenceClassification.from\_pretrained(“bert-base-cased”, num\_labels=3) model.compile( optimizer = tf.keras.optimizers.Adam(learning\_rate=5e-5) ) This gives me this error ValueError: Could not interpret optimizer identifier: \ what to do? I am using google colab ---
**Author:** ![fabifabine](https://avatars.discourse-cdn.com/v4/letter/f/0ea827/32.png) [@fabifabine](https://discuss.huggingface.co/u/fabifabine)\ **Post date:** [March 8, 2024, 1:21am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/2 "2024-03-08T01:21:35Z")
I’m having the same issue, but using this code: model = TFBertForSequenceClassification.from\_pretrained(‘neuralmind/bert-base-portuguese-cased’, num\_labels=2) optimizer = tf.keras.optimizers.Adam(learning\_rate=5e-5) loss = tf.keras.losses.SparseCategoricalCrossentropy(from\_logits=True) model.compile(optimizer=optimizer, loss=loss, metrics=[‘accuracy’]) ---
**Author:** ![linhco](https://avatars.discourse-cdn.com/v4/letter/l/c0e974/32.png) [@linhco](https://discuss.huggingface.co/u/linhco)\ **Post date:** [March 10, 2024, 1:47am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/3 "2024-03-10T01:47:26Z")
Same problem. Tried switching optimizers but not working for any (SGD or Adam) ---
**Author:** ![CanerCoban](https://avatars.discourse-cdn.com/v4/letter/c/85e7bf/32.png) [@CanerCoban](https://discuss.huggingface.co/u/CanerCoban)\ **Post date:** [March 11, 2024, 1:26am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/4 "2024-03-11T01:26:17Z")
This is also a case for me when I’m using tf on 2.15, and when I change it to 2.16 versions and try to “from transformers import TFBertForSequenceClassification” I’m having the issue below: RuntimeError: Failed to import transformers.models.bert.modeling\_tf\_bert because of the following error (look up to see its traceback): module ‘tensorflow.\_api.v2.compat.v2. **internal** ’ has no attribute ‘register\_load\_context\_function’ ---
**Author:** ![ansegura](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ansegura/32/24718_2.png) [@ansegura](https://discuss.huggingface.co/u/ansegura)\ **Post date:** [March 11, 2024, 10:08am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/5 "2024-03-11T10:08:09Z")
Same error here. I am using keras 2.15 and this code is failing now (two months ago it was working fine): ```python import tensorflow as tf from transformers import TFBertForSequenceClassification hf_model_name = "dccuchile/bert-base-spanish-wwm-cased" model = TFBertForSequenceClassification.from_pretrained(hf_model_name) optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0) loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy') model.compile(optimizer=optimizer, loss=loss, metrics=[metric]) ``` Error: ```auto ValueError: Could not interpret optimizer identifier: ``` ---
**Author:** ![Rocketknight1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rocketknight1/32/21055_2.png) [@Rocketknight1](https://discuss.huggingface.co/u/Rocketknight1)\ **Post date:** [March 11, 2024, 6:23pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/6 "2024-03-11T18:23:31Z")
Hi all, Matt from Hugging Face here! The cause is that TensorFlow has switched to Keras 3 as the ‘default’ Keras as of TF 2.16, and Keras 3 is often installed alongside TF 2.15 as well. The errors in this thread are because Keras 3 objects are being passed to Keras 2 model objects and code. The quickest solution is to `pip install tf-keras` and then set the environment variable `TF_USE_LEGACY_KERAS=1`. This will make `tf.keras` point to Keras 2, and your code should work as before. We’re also pushing a fix to `transformers` to do this by default [here](https://github.com/huggingface/transformers/pull/29598). If you want to try the fixed `transformers` branch instead, you can install it with `pip install git+https://github.com/huggingface/transformers.git@keras_3_compat_fix` This is an area of active work for us - please let us know if these approaches fixed your problem, and if you’re encountering any other issues with the Keras 3 transition! ---
**Author:** ![ansegura](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ansegura/32/24718_2.png) [@ansegura](https://discuss.huggingface.co/u/ansegura)\ **Post date:** [March 12, 2024, 9:42am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/7 "2024-03-12T09:42:53Z")
Thanks for the solution, @Rocketknight1 It works correctly for me. Looking forward to seeing this fix by default in the `transformer` library. Regards ---
**Author:** ![leon-hecht](https://avatars.discourse-cdn.com/v4/letter/l/96bed5/32.png) [@leon-hecht](https://discuss.huggingface.co/u/leon-hecht)\ **Post date:** [March 21, 2024, 2:05am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/8 "2024-03-21T02:05:50Z")
Hi Matt, both solutions did not work for me in Google Colab. - Solution 1 produced the same error for me ```auto !pip install tf-keras import os os.environ['TF_USE_LEGACY_KERAS'] = '1' ``` - Solution 2 `!pip install git+https://github.com/huggingface/transformers.git@keras_3_compat_fix` resulted in the following message: ```auto Collecting git+https://github.com/huggingface/transformers.git@keras_3_compat_fix Cloning https://github.com/huggingface/transformers.git (to revision keras_3_compat_fix) to /tmp/pip-req-build-u6t3ei27 Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-req-build-u6t3ei27 WARNING: Did not find branch or tag 'keras_3_compat_fix', assuming revision or ref. Running command git checkout -q keras_3_compat_fix error: pathspec 'keras_3_compat_fix' did not match any file(s) known to git error: subprocess-exited-with-error × git checkout -q keras_3_compat_fix did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × git checkout -q keras_3_compat_fix did not run successfully. │ exit code: 1 ╰─> See above for output. ``` What can I do? ---
**Author:** ![linhco](https://avatars.discourse-cdn.com/v4/letter/l/c0e974/32.png) [@linhco](https://discuss.huggingface.co/u/linhco)\ **Post date:** [March 21, 2024, 9:06am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/9 "2024-03-21T09:06:29Z")
try !pip install tf-keras==2.15.0 🙂 ---
**Author:** ![Rocketknight1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rocketknight1/32/21055_2.png) [@Rocketknight1](https://discuss.huggingface.co/u/Rocketknight1)\ **Post date:** [March 21, 2024, 12:01pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/10 "2024-03-21T12:01:44Z")
Hi @leon-hecht, the `keras_3_compat_fix` branch has been deleted since it has now been merged into transformers, and included in version 4.39 which released today. You can now just run `pip install --upgrade transformers` to get the fix instead of `pip install git+https://github.com/huggingface/transformers.git@keras_3_compat_fix`. Let me know if you’re still encountering problems! ---
**Author:** ![mahreenfatima](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahreenfatima/32/25112_2.png) [@mahreenfatima](https://discuss.huggingface.co/u/mahreenfatima)\ **Post date:** [March 21, 2024, 6:25pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/11 "2024-03-21T18:25:11Z")
I am facing the same issue, and I followed along the entire thread and still not able to resolve it. ---
**Author:** ![ansegura](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ansegura/32/24718_2.png) [@ansegura](https://discuss.huggingface.co/u/ansegura)\ **Post date:** [March 21, 2024, 8:56pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/12 "2024-03-21T20:56:29Z")
I just installed the latest version of the Transformers library (`v4.39.0`) on my Google Colab, and it worked for me. Thanks @Rocketknight1 Code: ```auto !pip install --upgrade transformers import transformers print(transformers. __version__ ) ``` ---
**Author:** ![Rocketknight1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rocketknight1/32/21055_2.png) [@Rocketknight1](https://discuss.huggingface.co/u/Rocketknight1)\ **Post date:** [March 22, 2024, 3:49pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/13 "2024-03-22T15:49:51Z")
Hi @mahreenfatima, can you confirm that you’re using transformers 4.39, and if so, can you paste me some sample code that shows your issue? ---
**Author:** ![mahreenfatima](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahreenfatima/32/25112_2.png) [@mahreenfatima](https://discuss.huggingface.co/u/mahreenfatima)\ **Post date:** [March 22, 2024, 7:41pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/14 "2024-03-22T19:41:09Z")
This is the code that is raising a ValueError, I am using transformers version 4.39.0 and tensorflow 2.15.0, I tried restarting the session on google colab to see the changes and stil the same error. opt\_new = Adam(learning\_rate = learning\_rate\_scheduler) model = TFAutoModelForSequenceClassification.from\_pretrained(checkpoint,num\_labels=2) loss = SparseCategoricalCrossentropy(from\_logits = True) model.compile(optimizer=opt\_new, loss=loss, metrics=[“accuracy”]) * * * ValueError Traceback (most recent call last) in \() ----\> 1 model.compile(optimizer=opt\_new, loss=loss, metrics=[“accuracy”]) * * * ## 2 frames [/usr/local/lib/python3.10/dist-packages/transformers/modeling\_tf\_utils.py](https://localhost:8080/#) in compile(self, optimizer, loss, metrics, loss\_weights, weighted\_metrics, run\_eagerly, steps\_per\_execution, \*\*kwargs) 1502 # This argument got renamed, we need to support both versions 1503 if “steps\_per\_execution” in parent\_args: → 1504 super().compile( 1505 optimizer=optimizer, 1506 loss=loss, /usr/local/lib/python3.10/dist-packages/tf\_keras/src/utils/traceback\_utils.py in error\_handler(\*args, \*\*kwargs) 68 # To get the full stack trace, call: 69 # `tf.debugging.disable_traceback_filtering()` —\> 70 raise e.with\_traceback(filtered\_tb) from None 71 finally: 72 del filtered\_tb /usr/local/lib/python3.10/dist-packages/tf\_keras/src/optimizers/ **init**.py in get(identifier, \*\*kwargs) 332 ) 333 else: → 334 raise ValueError( 335 f"Could not interpret optimizer identifier: {identifier}" 336 ) ## ValueError: Could not interpret optimizer identifier: \ Also while importing, from tensorflow.keras.losses import SparseCategoricalCrossentropy from tensorflow.keras.optimizers.schedules import PolynomialDecay, as I hover over the tensorflow.keras it is underlined and shows the following message --------Import “tensorflow.keras.optimizers.schedules” could not be resolved(reportMissingImports)------- ---
**Author:** ![Umair2002](https://avatars.discourse-cdn.com/v4/letter/u/ee7513/32.png) [@Umair2002](https://discuss.huggingface.co/u/Umair2002)\ **Post date:** [March 28, 2024, 1:16pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/15 "2024-03-28T13:16:12Z")
I tried all the solutions suggested in the discussion but still facing the same issue. ---
**Author:** ![Icarozea](https://avatars.discourse-cdn.com/v4/letter/i/4491bb/32.png) [@Icarozea](https://discuss.huggingface.co/u/Icarozea)\ **Post date:** [April 3, 2024, 6:57am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/16 "2024-04-03T06:57:00Z")
I am using Colab, I did all that this thread says: - !pip install --upgrade transformers → Upgraded to 4.39.2 version - !pip install tf-keras - os.environ[‘TF\_USE\_LEGACY\_KERAS’] = ‘1’ → I’ve set up the legacy version After all, I restarted the colab session and it worked, thanks ---
**Author:** ![Fan-Of-Drizzt](https://avatars.discourse-cdn.com/v4/letter/f/b5a626/32.png) [@Fan-Of-Drizzt](https://discuss.huggingface.co/u/Fan-Of-Drizzt)\ **Post date:** [April 4, 2024, 6:18pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/18 "2024-04-04T18:18:26Z")
I have also tried everything mentioned above. My transformer version is up to date and I have done what others have done but to no avail. What is the current recommendation that works? ---
**Author:** ![Aligorjis](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aligorjis/32/25575_2.png) [@Aligorjis](https://discuss.huggingface.co/u/Aligorjis)\ **Post date:** [April 8, 2024, 3:32am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/19 "2024-04-08T03:32:58Z")
I have the same problem and still getting same error. I tried everything, but it doesn’t work. I am working on a project and I am short on time. Please help. ---
**Author:** ![himjoshi](https://avatars.discourse-cdn.com/v4/letter/h/7ba0ec/32.png) [@himjoshi](https://discuss.huggingface.co/u/himjoshi)\ **Post date:** [April 9, 2024, 7:16am UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/20 "2024-04-09T07:16:49Z")
I’ve also tried all the steps as mentioned above but nothing is working. Attached is the SS showing the error I’m getting ![Screenshot 2024-04-09 at 12.46.38 PM](https://us1.discourse-cdn.com/hellohellohello/original/3X/b/3/b399724abe1212876de649f409a6e9607b0b159e.png) EDIT: The problem solved when I restarted the kernel. Do not refresh the google colab (after doing all the steps), rather restart it ---
**Author:** ![tligorio](https://avatars.discourse-cdn.com/v4/letter/t/8e8cbc/32.png) [@tligorio](https://discuss.huggingface.co/u/tligorio)\ **Post date:** [April 10, 2024, 8:57pm UTC](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209/21 "2024-04-10T20:57:21Z")
Hello @Rocketknight1 , I have followed all of the above recommendations: ```auto !pip install --upgrade transformers !pip install tf-keras import os os.environ['TF_USE_LEGACY_KERAS'] = '1' ``` AND restarted the colab session after installation, but I still get the error ```auto ValueError: Could not interpret optimizer identifier: ``` I am using ```auto checkpoint = "bert-base-uncased" model = TFAutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2) ``` Any recommendations why this may still not be working for some? [Next page](https://discuss.huggingface.co/t/pretrain-model-not-accepting-optimizer/76209.md?page=2)