# Some Roberta weights are not initializing from the checkpoint

**URL:** <https://discuss.huggingface.co/t/some-roberta-weights-are-not-initializing-from-the-checkpoint/73510>\
**Category:** 🤗Transformers\
**Created:** [February 16, 2024, 5:54pm UTC](https://discuss.huggingface.co/t/some-roberta-weights-are-not-initializing-from-the-checkpoint/73510 "2024-02-16T17:54:04Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![divyachandana](https://avatars.discourse-cdn.com/v4/letter/d/73ab20/32.png) [@divyachandana](https://discuss.huggingface.co/u/divyachandana)\
**Post date:** [February 16, 2024, 5:54pm UTC](https://discuss.huggingface.co/t/some-roberta-weights-are-not-initializing-from-the-checkpoint/73510/1 "2024-02-16T17:54:04Z")

</div>

I got this warning after loading Roberta model from the checkpoint  
Loading the trained model this way

```auto
model = RobertaModel.load_from_checkpoint(
            checkpoint_path=<path>
        )
        model.freeze()

```

Warning

```auto
Some weights of RobertaModel were not initialized from the model checkpoint at roberta-base and are newly initialized: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

```