# Loading a trained model gives an error that weights are randomly initialized

**URL:** <https://discuss.huggingface.co/t/loading-a-trained-model-gives-an-error-that-weights-are-randomly-initialized/42314>\
**Category:** 🤗Transformers\
**Created:** [June 6, 2023, 8:21pm UTC](https://discuss.huggingface.co/t/loading-a-trained-model-gives-an-error-that-weights-are-randomly-initialized/42314 "2023-06-06T20:21:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![devengqc](https://avatars.discourse-cdn.com/v4/letter/d/ea666f/32.png) [@devengqc](https://discuss.huggingface.co/u/devengqc)\
**Post date:** [June 6, 2023, 8:21pm UTC](https://discuss.huggingface.co/t/loading-a-trained-model-gives-an-error-that-weights-are-randomly-initialized/42314/1 "2023-06-06T20:21:49Z")

</div>

Hello,

I have trained `OPTCausalLM` model finetuned and saved to disk in the following fashion.

```py
trainer.save_model("peft-finetuned-10e")
model.config.to_json_file("peft-finetuned-10e/config.json")

```

When I try to load the model again, using the `from_pretrained` argument, I see this error.

```py
config = AutoConfig.from_pretrained('./peft-finetuned-10e/config.json')
model = AutoModelForCausalLM.from_pretrained('./peft-finetuned-10e/', config=config, from_tf=False)

```

Here is the entire stacktrace

```py
Some weights of the model checkpoint at ./peft-finetuned-10e/ were not used when initializing OPTForCausalLM: ['base_model.model.model.decoder.layers.14.fc2.bias', 
...
...
...
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

```