# \[Possibly\] Forgotten TODO Comment for \`TrainingArguments.default\_optim\`

**URL:** <https://discuss.huggingface.co/t/possibly-forgotten-todo-comment-for-trainingarguments-default-optim/145788>\
**Category:** 🤗Transformers\
**Created:** [March 14, 2025, 3:30pm UTC](https://discuss.huggingface.co/t/possibly-forgotten-todo-comment-for-trainingarguments-default-optim/145788 "2025-03-14T15:30:06Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![pkuderov](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pkuderov/32/43300_2.png) [@pkuderov](https://discuss.huggingface.co/u/pkuderov)\
**Post date:** [March 14, 2025, 3:30pm UTC](https://discuss.huggingface.co/t/possibly-forgotten-todo-comment-for-trainingarguments-default-optim/145788/1 "2025-03-14T15:30:06Z")

</div>

While reading the metadata and default parameters for the `TrainingArguments` attributes, I noticed the following comment regarding `default_optim` ([permalink](https://github.com/huggingface/transformers/blob/42ebb6c23e61119f769d7c7c067d5b4ae10e4a7f/src/transformers/training_args.py#L1302)):

```auto
   default_optim = "adamw_torch"
    # XXX: enable when pytorch==2.0.1 comes out - we want to give it time to get all the bugs sorted out
    # if is_torch_available() and version.parse(version.parse(torch. __version__ ).base_version) >= version.parse("2.1.0"):
    # default_optim = "adamw_torch_fused"
    # and update the doc above to:
    # optim (`str` or [`training_args.OptimizerNames`], *optional*, defaults to `"adamw_torch_fused"` (for torch<2.1.0 `"adamw_torch"`):
    optim: Union[OptimizerNames, str] = field(
        default=default_optim,
        metadata={"help": "The optimizer to use."},
    )

```

My question is whether this comment is outdated and should be removed, or if it is still relevant and should be applied, since it seems that the time has come.

**NB:** I figured it was worth drawing attention to this TODO.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [March 14, 2025, 11:45pm UTC](https://discuss.huggingface.co/t/possibly-forgotten-todo-comment-for-trainingarguments-default-optim/145788/2 "2025-03-14T23:45:40Z")

</div>

It’s almost time for the release of PyTorch 2.7…😅  
You might want to raise an issue on the Transformers github.

> **[huggingface/transformers](https://github.com/huggingface/transformers/issues)**
>
> 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX. - huggingface/transformers