# Error loading model via from\_pretrained

**URL:** <https://discuss.huggingface.co/t/error-loading-model-via-from-pretrained/17575>\
**Category:** 🤗Transformers\
**Created:** [May 5, 2022, 7:14pm UTC](https://discuss.huggingface.co/t/error-loading-model-via-from-pretrained/17575 "2022-05-05T19:14:21Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![samvdp](https://avatars.discourse-cdn.com/v4/letter/s/f1d935/32.png) [@samvdp](https://discuss.huggingface.co/u/samvdp)\
**Post date:** [May 5, 2022, 7:14pm UTC](https://discuss.huggingface.co/t/error-loading-model-via-from-pretrained/17575/1 "2022-05-05T19:14:21Z")

</div>

I have trained a TFDistilBertForSequenceClassification model and successfully saved it to disk using save\_pretrained. The expected files (tf\_model.h5 and config.json) are in my model directory. However, when I try to load this model via TFDistilBertForSequenceClassification.from\_pretrained, I get the following error concerning embeddings.

> * * *
> 
> InvalidArgumentError Traceback (most recent call last)  
> in   
> 1 model = TFDistilBertForSequenceClassification.from\_pretrained(  
> 2 ckpt\_dir,  
> ----\> 3 config=distilbertconfig  
> 4 )
> 
> /opt/anaconda3/envs/subreg/lib/python3.7/site-packages/transformers/modeling\_tf\_utils.py in from\_pretrained(cls, pretrained\_model\_name\_or\_path, \*model\_args, \*\*kwargs)  
> 1323 model(model.dummy\_inputs) # build the network with dummy inputs  
> 1324 else:  
> → 1325 model(model.dummy\_inputs) # build the network with dummy inputs  
> 1326  
> 1327 assert os.path.isfile(resolved\_archive\_file), f"Error retrieving file {resolved\_archive\_file}"
> 
> /opt/anaconda3/envs/subreg/lib/python3.7/site-packages/keras/utils/traceback\_utils.py in error\_handler(\*args, \*\*kwargs)  
> 65 except Exception as e: # pylint: disable=broad-except  
> 66 filtered\_tb = \_process\_traceback\_frames(e. **traceback** )  
> —\> 67 raise e.with\_traceback(filtered\_tb) from None  
> 68 finally:  
> 69 del filtered\_tb
> 
> /opt/anaconda3/envs/subreg/lib/python3.7/site-packages/transformers/models/distilbert/modeling\_tf\_distilbert.py in call(self, input\_ids, attention\_mask, head\_mask, inputs\_embeds, output\_attentions, output\_hidden\_states, return\_dict, labels, training, \*\*kwargs)  
> 806 output\_hidden\_states=inputs[“output\_hidden\_states”],  
> 807 return\_dict=inputs[“return\_dict”],  
> → 808 training=inputs[“training”],  
> 809 )  
> 810 hidden\_state = distilbert\_output[0] # (bs, seq\_len, dim)
> 
> /opt/anaconda3/envs/subreg/lib/python3.7/site-packages/transformers/models/distilbert/modeling\_tf\_distilbert.py in call(self, input\_ids, attention\_mask, head\_mask, inputs\_embeds, output\_attentions, output\_hidden\_states, return\_dict, training, \*\*kwargs)  
> 414  
> 415 embedding\_output = self.embeddings(  
> → 416 inputs[“input\_ids”], inputs\_embeds=inputs[“inputs\_embeds”]  
> 417 ) # (bs, seq\_length, dim)  
> 418 tfmr\_output = self.transformer(
> 
> /opt/anaconda3/envs/subreg/lib/python3.7/site-packages/transformers/models/distilbert/modeling\_tf\_distilbert.py in call(self, input\_ids, position\_ids, inputs\_embeds, training)  
> 110  
> 111 if input\_ids is not None:  
> → 112 inputs\_embeds = tf.gather(params=self.weight, indices=input\_ids)  
> 113  
> 114 input\_shape = shape\_list(inputs\_embeds)[:-1]
> 
> InvalidArgumentError: Exception encountered when calling layer “embeddings” (type TFEmbeddings).
> 
> indices[0,0] = 7 is not in [0, 5) [Op:ResourceGather]
> 
> Call arguments received:  
> • input\_ids=tf.Tensor(shape=(3, 5), dtype=int32)  
> • position\_ids=None  
> • inputs\_embeds=None  
> • training=False

Potentially noteworthy is that I got the following warning when saving my model using save\_pretrained:

> WARNING:absl:Found untraced functions such as embeddings\_layer\_call\_fn, embeddings\_layer\_call\_and\_return\_conditional\_losses, transformer\_layer\_call\_fn, transformer\_layer\_call\_and\_return\_conditional\_losses, add\_layer\_call\_fn while saving (showing 5 of 62). These functions will not be directly callable after loading.

I read online that this warning is harmless, but the warning mentions embeddings, so I figured it may be related to my error.

I would very much appreciate any references or guidance on how to properly load my TFDistilBertForSequenceClassification model after it is saved. Thank you!