# Testing best model after hyperparameter\_search

**URL:** <https://discuss.huggingface.co/t/testing-best-model-after-hyperparameter-search/19819>\
**Category:** 🤗Transformers\
**Created:** [June 29, 2022, 10:43pm UTC](https://discuss.huggingface.co/t/testing-best-model-after-hyperparameter-search/19819 "2022-06-29T22:43:33Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](https://discuss.huggingface.co/u/BramVanroy)\
**Post date:** [June 29, 2022, 10:43pm UTC](https://discuss.huggingface.co/t/testing-best-model-after-hyperparameter-search/19819/1 "2022-06-29T22:43:33Z")

</div>

I am trying to get my head around how to use Ray Tune’s PB2 scheduler alongside the Trainer. Specifically, how can you load the best/final model of a PBT-based scheduler and then continue to test/predict?

With grid search, we could do something like this:

```python
best_params = trainer.hyperparameter_search(...)

# Set the trainer to the best hyperparameters found
for hparam, v in best_params.hyperparameters.items():
    setattr(trainer.args, hparam, v)

# Save optimal hparams
with output_dir.joinpath("opt_hparams.json").open("w", encoding="utf-8") as hp_out:
    dump(best_params, hp_out, indent=4, sort_keys=True)

# Now train the model from-scratch with the best hparams
train_result = trainer.train()
# ... and then get predictions from the model 
predictions = trainer.predict()

```

but because PB2 changes its hyperparameters on-the-fly, this won’t work because you cannot just do one training run with a single set of parameters.

So how do we typically go about this scenario? How do we load the final, best model after a PB-based hyperparameter-search, and then predict on the test set?

PS I asked this on [the Ray forums](https://discuss.ray.io/t/ray-tune-pbt-pbt2-with-transformers/6466/2?u=bramvanroy) but did not get a response unfortunately.