# Problems when loading checkpoints

**URL:** <https://discuss.huggingface.co/t/problems-when-loading-checkpoints/125867>\
**Category:** 🤗Transformers\
**Created:** [November 19, 2024, 4:33am UTC](https://discuss.huggingface.co/t/problems-when-loading-checkpoints/125867 "2024-11-19T04:33:23Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![erv1n](https://avatars.discourse-cdn.com/v4/letter/e/8dc957/32.png) [@erv1n](https://discuss.huggingface.co/u/erv1n)\
**Post date:** [November 19, 2024, 4:33am UTC](https://discuss.huggingface.co/t/problems-when-loading-checkpoints/125867/1 "2024-11-19T04:33:23Z")

</div>

I was distilling my student model (base model t5-small) based on a fine-tuned T5-xxl. Here is the config  
`student_model = AutoModelForSeq2SeqLM.from_pretrained( args.student_model_name_or_path, torch_dtype=torch.float32, device_map="auto", cache_dir=args.cache_dir, quantization_config=quantization_config, )`  
I saved the trained model using

```auto
output_dir = f"checkpoint"
student_model.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)

```

But when I trying to load the ckp using

```auto
tokenizer = AutoTokenizer.from_pretrained(args.model_path)
model = AutoModelForSeq2SeqLM.from_pretrained(
        "checkpoint"
        torch_dtype=torch.float32,
        device_map="auto",
        cache_dir=args.cache_dir,
        quantization_config=quantization_config
)

```

it said that "Some weights of the model checkpoint at checkpoints were not used when initializing T5ForConditionalGeneration: " and the output of the model is really a mess. I was trying to figure it out but have no clues now.

---

<div class="post-metadata">

**Author:** ![not-lain](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/not-lain/32/59092_2.png) [@not-lain](https://discuss.huggingface.co/u/not-lain)\
**Post date:** [November 19, 2024, 3:26pm UTC](https://discuss.huggingface.co/t/problems-when-loading-checkpoints/125867/2 "2024-11-19T15:26:49Z")

</div>

you might have forgotten to change the parameters in the config or did something wrong in the distillation process.  
the error that you’re having means that there is a mismatch between the parameters of the model and the checkpoint.  
try checking the model state\_dict and the checkpoint file and figure out where is the mismatch happening.  
this link might also help you with the distillation process :

> **[transformers/examples/research\_projects/distillation at main ·...](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation)**
>
> 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX. - huggingface/transformers

---

<div class="post-metadata">

**Author:** ![erv1n](https://avatars.discourse-cdn.com/v4/letter/e/8dc957/32.png) [@erv1n](https://discuss.huggingface.co/u/erv1n)\
**Post date:** [November 20, 2024, 9:23pm UTC](https://discuss.huggingface.co/t/problems-when-loading-checkpoints/125867/3 "2024-11-20T21:23:42Z")

</div>

The problem was `model.save_pretrained()` can’t save weights properly if you set the quantization `load_in_8bit`.  
Check this link if you need to save weights loaded in 8 bits.

> **[Model quantization](https://huggingface.co/docs/accelerate/usage_guides/quantization)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.