# Load\_best\_model\_at\_end doesn't work?

**URL:** <https://discuss.huggingface.co/t/load-best-model-at-end-doesnt-work/99822>\
**Category:** 🤗Transformers\
**Created:** [July 28, 2024, 9:42pm UTC](https://discuss.huggingface.co/t/load-best-model-at-end-doesnt-work/99822 "2024-07-28T21:42:24Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ice90](https://avatars.discourse-cdn.com/v4/letter/i/51bf81/32.png) [@Ice90](https://discuss.huggingface.co/u/Ice90)\
**Post date:** [July 28, 2024, 9:42pm UTC](https://discuss.huggingface.co/t/load-best-model-at-end-doesnt-work/99822/1 "2024-07-28T21:42:24Z")

</div>

I am using trainer to finetune llava model. I want to save the best model during training based on the specific metric, however I found it loads the model at the end of training instead of the best one during the training. Here’s my relevant hyperparameters:

```auto
trainer.train()
trainer.evaluate(eval_dataset)

```

```auto
deepspeed llava/train/train_xformers.py \
    --bf16 False \
    --fp16 True \
    --num_train_epochs 10 \
    --per_device_train_batch_size 16 \
    --per_device_eval_batch_size 4 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "steps" \
    --eval_steps 10 \
    --save_steps 10 \
    --save_strategy "steps" \
    --greater_is_better True \
    --load_best_model_at_end True \
    --metric_for_best_model eval_roc_auc \
    --learning_rate 0.000005 \
    --weight_decay 0.0000 \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --dataloader_num_workers 4 \
    --save_total_limit 1 \
    --lazy_preprocess True \
    --report_to wandb

```

Thanks for your help in advance.

---

<div class="post-metadata">

**Author:** ![fabikru](https://avatars.discourse-cdn.com/v4/letter/f/e0b2c6/32.png) [@fabikru](https://discuss.huggingface.co/u/fabikru)\
**Post date:** [March 25, 2025, 12:46pm UTC](https://discuss.huggingface.co/t/load-best-model-at-end-doesnt-work/99822/2 "2025-03-25T12:46:26Z")

</div>

I have the same issue. Did anybody find a solution for this yet?