# Hyperparameters for lr\_scheduler\_type in Trainer Arguments

**URL:** <https://discuss.huggingface.co/t/hyperparameters-for-lr-scheduler-type-in-trainer-arguments/38469>\
**Category:** Beginners\
**Created:** [May 2, 2023, 9:24pm UTC](https://discuss.huggingface.co/t/hyperparameters-for-lr-scheduler-type-in-trainer-arguments/38469 "2023-05-02T21:24:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![harpreetmann](https://avatars.discourse-cdn.com/v4/letter/h/eada6e/32.png) [@harpreetmann](https://discuss.huggingface.co/u/harpreetmann)\
**Post date:** [May 2, 2023, 9:24pm UTC](https://discuss.huggingface.co/t/hyperparameters-for-lr-scheduler-type-in-trainer-arguments/38469/1 "2023-05-02T21:24:54Z")

</div>

My understanding is that I can pass the following string values to use the corresponding learning rate schedulers in Training Arguments:

> - “linear” → transformers.get\_linear\_schedule\_with\_warmup
> - “cosine” → transformers.get\_cosine\_schedule\_with\_warmup
> - “cosine\_with\_restarts” --\>transformers.get\_cosine\_with\_hard\_restarts\_schedule\_with\_warmup
> - “polynomial” → transformers.get\_polynomial\_decay\_schedule\_with\_warmup
> - “constant” → transformers.get\_constant\_schedule
> - “constant\_with\_warmup” → transformers.get\_constant\_schedule\_with\_warmup
> - “inverse\_sqrt” → transformers.get\_inverse\_sqrt\_schedule

However, some of the schedulers have specific hyperparameters. For example, transformers.get\_cosine\_schedule\_with\_warmup has num\_cycels as a parameter. How can we change the default values?

---

<div class="post-metadata">

**Author:** ![wbtxhqt](https://avatars.discourse-cdn.com/v4/letter/w/a88e57/32.png) [@wbtxhqt](https://discuss.huggingface.co/u/wbtxhqt)\
**Post date:** [January 29, 2024, 4:22pm UTC](https://discuss.huggingface.co/t/hyperparameters-for-lr-scheduler-type-in-trainer-arguments/38469/2 "2024-01-29T16:22:32Z")

</div>

Hi, there is a another argument called **lr\_scheduler\_kwargs** , you can change the hyperparameters in this with dict.

---

<div class="post-metadata">

**Author:** ![HandsomeWu666](https://avatars.discourse-cdn.com/v4/letter/h/74df32/32.png) [@HandsomeWu666](https://discuss.huggingface.co/u/HandsomeWu666)\
**Post date:** [March 5, 2024, 12:35am UTC](https://discuss.huggingface.co/t/hyperparameters-for-lr-scheduler-type-in-trainer-arguments/38469/3 "2024-03-05T00:35:35Z")

</div>

which version have the argument called **lr\_scheduler\_kwargs**. i used transformers 4.28.0