# Accessing model after training with hyper-parameter search

**URL:** <https://discuss.huggingface.co/t/accessing-model-after-training-with-hyper-parameter-search/20081>\
**Category:** 🤗Transformers\
**Created:** [July 7, 2022, 4:12am UTC](https://discuss.huggingface.co/t/accessing-model-after-training-with-hyper-parameter-search/20081 "2022-07-07T04:12:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dqrx](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@dqrx](https://discuss.huggingface.co/u/dqrx)\
**Post date:** [July 7, 2022, 4:12am UTC](https://discuss.huggingface.co/t/accessing-model-after-training-with-hyper-parameter-search/20081/1 "2022-07-07T04:12:31Z")

</div>

I’m trying to do a hyper-parameter search using the huggingface Trainer and ray[tune].

I’m more specifically following [this example from RAY](https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/pbt_transformers/pbt_transformers.py).

However, when I try to use the trainer (after all of the code from the example is executed) to predict I get the following error:

```auto
(Pdb) trainer.predict(train_dataset)
*** AttributeError: 'NoneType' object has no attribute 'forward'

```

Am I supposed to retrain the trainer after a hyper-parameter search?

Here is the exact steps I took for clarification:

1. Downloaded example file i.e. `pbt_transformers.py`
2. Added a `import pdb; pdb.set_trace()` on line 147 (after the last call in the function)
3. Ran this with `python pbt_transformers.py --smoke-test`
4. Once the debugger started I entered `trainer.predict(train_dataset)`

I apologize if the answer is somewhere but I looked at these examples (below) and didn’t see anything that is done after calling `trainer.hyperparameter_search`:

> **[Hyperparameter Search with Transformers and Ray Tune](https://huggingface.co/blog/ray-tune)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [July 7, 2022, 12:59pm UTC](https://discuss.huggingface.co/t/accessing-model-after-training-with-hyper-parameter-search/20081/2 "2022-07-07T12:59:17Z")

</div>

Currently, you have to retrain your model with the best set of hyperparameters to use it afterwards, unless you can find its checkpoint in the runs saved.

---

<div class="post-metadata">

**Author:** ![dqrx](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@dqrx](https://discuss.huggingface.co/u/dqrx)\
**Post date:** [July 7, 2022, 4:43pm UTC](https://discuss.huggingface.co/t/accessing-model-after-training-with-hyper-parameter-search/20081/3 "2022-07-07T16:43:15Z")

</div>

Ok great, thank you!