# Optimizer returned by create\_optimizer function

**URL:** <https://discuss.huggingface.co/t/optimizer-returned-by-create-optimizer-function/21885>\
**Category:** Beginners\
**Created:** [August 21, 2022, 5:59pm UTC](https://discuss.huggingface.co/t/optimizer-returned-by-create-optimizer-function/21885 "2022-08-21T17:59:29Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gozdi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/gozdi/32/4221_2.png) [@Gozdi](https://discuss.huggingface.co/u/Gozdi)\
**Post date:** [August 21, 2022, 5:59pm UTC](https://discuss.huggingface.co/t/optimizer-returned-by-create-optimizer-function/21885/1 "2022-08-21T17:59:29Z")

</div>

Hi  
I have a question regarding create\_optimizer function.

```auto
num_epochs = 5
num_train_steps = len(tf_train_dataset) * num_epochs

optimizer, schedule = create_optimizer(
    init_lr = 3e-5,
    num_warmup_steps=int(num_train_steps*0.06),
    num_train_steps=num_train_steps,
    weight_decay_rate=0.01
)
model_0.compile(optimizer=optimizer)

```

If I use it like that will learning rate be decaying over time or do i need to do it like that???

```auto
model_0.compile(optimizer=optimizer(learning_rate=schedule))

```