# Convert transformer to SavedModel

**URL:** <https://discuss.huggingface.co/t/convert-transformer-to-savedmodel/353>\
**Category:** Beginners\
**Created:** [July 17, 2020, 9:26am UTC](https://discuss.huggingface.co/t/convert-transformer-to-savedmodel/353 "2020-07-17T09:26:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![aishutin](https://avatars.discourse-cdn.com/v4/letter/a/e19adc/32.png) [@aishutin](https://discuss.huggingface.co/u/aishutin)\
**Post date:** [July 17, 2020, 9:26am UTC](https://discuss.huggingface.co/t/convert-transformer-to-savedmodel/353/1 "2020-07-17T09:26:19Z")

</div>

Hi! I found out that this is common unresolved problem.

So, I need to convert transformers’ DistilBERT to TensorFlow`s SavedModel format. I've converted it, but I can`t inference it.

### Conversion code

```auto
import tensorflow as tf
from transformers import TFAutoModel, AutoTokenizer
dir = "distilbert_savedmodel"

model = TFAutoModel.from_pretrained('distilbert-base-uncased')
model.save(dir)

```

### Inference code

```auto
tokenizer = AutoTokenizer.from_pretrained('distilbert-base-uncased')
encoded = tokenizer.encode('Hello, world!', add_special_tokens=True, return_tensors="tf")
model = tf.keras.models.load_model(dir)
model(encoded)

```

### Error

```auto

ValueError: Could not find matching function to call loaded from the SavedModel. Got:
  Positional arguments (1 total):
    * Tensor("inputs:0", shape=(1, 6), dtype=int32)
  Keyword arguments: {'training': False}

Expected these arguments to match one of the following 4 option(s):

Option 1:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='input_ids')}
  Keyword arguments: {'training': False}

Option 2:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='input_ids')}
  Keyword arguments: {'training': True}

Option 3:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='inputs/input_ids')}
  Keyword arguments: {'training': True}

Option 4:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='inputs/input_ids')}
  Keyword arguments: {'training': False}

```

### Related issues

[huggingface/transformers#4004](https://github.com/huggingface/transformers/issues/4004)  
[huggingface/transformers#2135](https://github.com/huggingface/transformers/issues/2135)  
[huggingface/transformers#2021](https://github.com/huggingface/transformers/issues/2021)

Please, help me!

---

<div class="post-metadata">

**Author:** ![rgwatwormhill](https://avatars.discourse-cdn.com/v4/letter/r/a9adbd/32.png) [@rgwatwormhill](https://discuss.huggingface.co/u/rgwatwormhill)\
**Post date:** [November 18, 2020, 12:25pm UTC](https://discuss.huggingface.co/t/convert-transformer-to-savedmodel/353/2 "2020-11-18T12:25:31Z")

</div>

In pytorch, you could save the model with something like

> torch.save(model.state\_dict(),‘/content/drive/My Drive/ftmodelname’ )

Then you could create a model using the pre-trained weights

> tuned\_model = BertForSequenceClassification.from\_pretrained(‘bert-base-uncased’,  
> num\_labels=NCLASSES,  
> output\_attentions=True)

and then overwrite its weights from the saved state\_dict with

> tuned\_model.load\_state\_dict(torch.load(‘/content/drive/My Drive/ftmodelname’,  
> map\_location=torch.device(“cpu”)),  
> strict=False)

I expect you could do a similar save of the model state\_dict using Tensorflow.

---

<div class="post-metadata">

**Author:** ![alchencjt](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@alchencjt](https://discuss.huggingface.co/u/alchencjt)\
**Post date:** [September 8, 2021, 7:30am UTC](https://discuss.huggingface.co/t/convert-transformer-to-savedmodel/353/3 "2021-09-08T07:30:25Z")

</div>

How did you end up solving the problem. I came across the same one.

---

<div class="post-metadata">

**Author:** ![OverFlow7](https://avatars.discourse-cdn.com/v4/letter/o/ecb155/32.png) [@OverFlow7](https://discuss.huggingface.co/u/OverFlow7)\
**Post date:** [November 27, 2021, 10:06pm UTC](https://discuss.huggingface.co/t/convert-transformer-to-savedmodel/353/4 "2021-11-27T22:06:09Z")

</div>

I have the same problem.

I can train a model, evaluate and predict if it’s within the same script, but if I save the model and then load it I get the same kind of error, is it currently impossible to load a saved model with tensorflow/keras?

---

<div class="post-metadata">

**Author:** ![Kforcode](https://avatars.discourse-cdn.com/v4/letter/k/57b2e6/32.png) [@Kforcode](https://discuss.huggingface.co/u/Kforcode)\
**Post date:** [November 30, 2021, 3:34pm UTC](https://discuss.huggingface.co/t/convert-transformer-to-savedmodel/353/5 "2021-11-30T15:34:39Z")

</div>

hugging face has model.save\_pretrained() method