# How to save bert or distilbert model? **URL:** \ **Category:** 🤗Transformers\ **Created:** [November 3, 2022, 8:53am UTC](https://discuss.huggingface.co/t/how-to-save-bert-or-distilbert-model/25465 "2022-11-03T08:53:02Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![Pranavsatyal](https://avatars.discourse-cdn.com/v4/letter/p/ce7236/32.png) [@Pranavsatyal](https://discuss.huggingface.co/u/Pranavsatyal)\ **Post date:** [November 3, 2022, 8:53am UTC](https://discuss.huggingface.co/t/how-to-save-bert-or-distilbert-model/25465/1 "2022-11-03T08:53:02Z")
Hi ALL! I’m having issues loading my trained distilbert model, cannot figure out a way to resolve the issue, when i try to load the model i get value error that im not able to resolve, my codes are below, you can find the notebook, data and model in my google drive > **[issue\_distilbert - Google Drive](https://drive.google.com/drive/folders/1HV60_4D2JZ_Yru5tVRDblhUdgKOIQCS7?usp=sharing)** “”" ValueError: Unknown layer: TFDistilBertModel. Please ensure this object is passed to the `custom_objects` argument. See [https://www.tensorflow.org/guide/keras/save\_and\_serialize#registering\_the\_custom\_object](https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object) for details.“”“” #config = DistilBertConfig.from\_pretrained(MODEL\_NAME, output\_hidden\_states=True, output\_attentions=True) # DistilBERT = TFDistilBertModel.from\_pretrained(MODEL\_NAME, config=config) # input\_ids\_in = tf.keras.layers.Input(shape=(MAX\_LENGTH,), name=‘input\_token’, dtype=‘int32’) # input\_masks\_in = tf.keras.layers.Input(shape=(MAX\_LENGTH,), name=‘masked\_token’, dtype=‘int32’) # embedding\_layer = DistilBERT(input\_ids = input\_ids\_in, attention\_mask = input\_masks\_in)[0] # X = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(128, return\_sequences=True))(embedding\_layer) # X = tf.keras.layers.GlobalMaxPool1D()(X) # X = tf.keras.layers.Dense(64, activation=‘relu’)(X) # X = tf.keras.layers.Dropout(0.2)(X) # X = tf.keras.layers.Dense(4, activation=‘softmax’)(X) # model = tf.keras.Model(inputs=[input\_ids\_in, input\_masks\_in], outputs = X) # for layer in model.layers[:3]: # layer.trainable = False # model.summary() # optim = tf.keras.optimizers.Adam() # loss\_func = tf.keras.losses.CategoricalCrossentropy() # acc = tf.keras.metrics.CategoricalAccuracy(‘accuracy’) # model.compile(loss=loss\_func, # optimizer=optim, # metrics=[tf.keras.metrics.categorical\_accuracy]) #Y\_train\_cat = keras.utils.to\_categorical(y\_train) #history = model.fit(X\_train, # Y\_train\_cat, # epochs = 4, # batch\_size=256, # shuffle= True, # validation\_split=.2, # callbacks=[model\_checkpoint, early\_stopping, reduce\_lr]) ## model.save(“test\_distilbert.h5”) #from tensorflow.keras.models import load\_model #load\_model(“test\_distilbert.h5”)