# About Hyperparameter Search with Ray Tune

**URL:** <https://discuss.huggingface.co/t/about-hyperparameter-search-with-ray-tune/144536>\
**Category:** 🤗Transformers\
**Created:** [March 6, 2025, 11:58pm UTC](https://discuss.huggingface.co/t/about-hyperparameter-search-with-ray-tune/144536 "2025-03-06T23:58:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![josephleyva](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/josephleyva/32/42048_2.png) [@josephleyva](https://discuss.huggingface.co/u/josephleyva)\
**Post date:** [March 6, 2025, 11:58pm UTC](https://discuss.huggingface.co/t/about-hyperparameter-search-with-ray-tune/144536/1 "2025-03-06T23:58:18Z")

</div>

When using `hyperparameter_search()` we have different choices like Ray, Optuna or HyperOpt.

With Ray, the hyperparameter search uses `tune.run()`.  
However, there is a better way that Ray recommends using that is called `Tuner.fit()` (Added in Ray 2.0).

[Tune Experiment Results (tune.ResultGrid) — Ray 2.43.0](https://docs.ray.io/en/latest/tune/api/result_grid.html)

> ℹ Note  
> An `ExperimentAnalysis` is the output of the `tune.run` API. It’s now recommended to use [`Tuner.fit`](https://docs.ray.io/en/latest/tune/api/doc/ray.tune.Tuner.fit.html#ray.tune.Tuner.fit), which outputs a `ResultGrid` object.

Are there any plans in the future to implement the Tuner for the Ray backend?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [March 7, 2025, 9:44am UTC](https://discuss.huggingface.co/t/about-hyperparameter-search-with-ray-tune/144536/2 "2025-03-07T09:44:27Z")

</div>

I think you’ll have to ask Ray’s developer on github.

> **[ray-project/ray](https://github.com/ray-project/ray/issues)**
>
> Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. - ray-project/ray

---

<div class="post-metadata">

**Author:** ![josephleyva](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/josephleyva/32/42048_2.png) [@josephleyva](https://discuss.huggingface.co/u/josephleyva)\
**Post date:** [March 7, 2025, 4:04pm UTC](https://discuss.huggingface.co/t/about-hyperparameter-search-with-ray-tune/144536/3 "2025-03-07T16:04:30Z")

</div>

I think it should be a new feature.

Looking at the source code, it still calling the `tune.run()`

[transformers/src/transformers/integrations/integration\_utils.py at main · huggingface/transformers](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/integration_utils.py)

```python
analysis = ray.tune.run(
        dynamic_modules_import_trainable,
        config=trainer.hp_space(None),
        num_samples=n_trials,
        **kwargs,
    )
    best_trial = analysis.get_best_trial(metric="objective", mode=direction[:3], scope=trainer.args.ray_scope)
    best_run = BestRun(best_trial.trial_id, best_trial.last_result["objective"], best_trial.config, analysis)
    if _tb_writer is not None:
        trainer.add_callback(_tb_writer)
    return best_run

```