# Error while saving and loading a Bert model

**URL:** <https://discuss.huggingface.co/t/error-while-saving-and-loading-a-bert-model/26462>\
**Category:** 🤗Transformers\
**Created:** [November 21, 2022, 6:08pm UTC](https://discuss.huggingface.co/t/error-while-saving-and-loading-a-bert-model/26462 "2022-11-21T18:08:27Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Himalay](https://avatars.discourse-cdn.com/v4/letter/h/22d042/32.png) [@Himalay](https://discuss.huggingface.co/u/Himalay)\
**Post date:** [November 21, 2022, 6:08pm UTC](https://discuss.huggingface.co/t/error-while-saving-and-loading-a-bert-model/26462/1 "2022-11-21T18:08:27Z")

</div>

Hello ,

I am very new to transformers and this is a model I built using the transformers library for text classification.

```auto
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased-finetuned-sst-2-english')

model = TFDistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased-finetuned-sst-2-english')

optimizerr = tf.keras.optimizers.Adam(learning_rate=5e-5)
losss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=optimizerr,loss=losss,metrics=['accuracy'])  

model.fit(train_dataset.shuffle(train.count()).batch(16), epochs=1,batch_size=16)

```

Now My goal is to save this trained model and load it at a later point of time to make prediction.

To save the model I tried :

```auto
path = '/Shared/saved_models'
model.save_pretrained(path, saved_model=True)

```

I am getting the below warnings when I do this :

```auto
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.core.dropout.Dropout object at 0x7f51b81941c0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.core.dropout.Dropout object at 0x7f51b828cf10>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.core.dropout.Dropout object at 0x7f51b81086a0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.core.dropout.Dropout object at 0x7f51b830ed90>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.core.dropout.Dropout object at 0x7f51b83194f0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.core.dropout.Dropout object at 0x7f51b8322c10>, because it is not built.
WARNING:absl:Found untraced functions such as embeddings_layer_call_fn, embeddings_layer_call_and_return_conditional_losses, transformer_layer_call_fn, transformer_layer_call_and_return_conditional_losses, LayerNorm_layer_call_fn while saving (showing 5 of 164). These functions will not be directly callable after loading.

```

Now to load the model to make my predictions , I am doing :

```auto
new_model = AutoModelForSequenceClassification.from_pretrained('/Shared/saved_models',from_tf=True)

```

This gives me another warning :

```auto
All TF 2.0 model weights were used when initializing DistilBertForSequenceClassification.
All the weights of DistilBertForSequenceClassification were initialized from the TF 2.0 model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use DistilBertForSequenceClassification for predictions without further training.

```

But I am not able to make predictions using this ‘new\_model’ .  
I am not able to fully understand what is happening and how I can sucesfully save and load my model. Any help is **highly** appreciated. Thank you.