# Huggingface transformers longformer optimizer warning AdamW

**URL:** <https://discuss.huggingface.co/t/huggingface-transformers-longformer-optimizer-warning-adamw/14711>\
**Category:** Models\
**Created:** [February 14, 2022, 3:21pm UTC](https://discuss.huggingface.co/t/huggingface-transformers-longformer-optimizer-warning-adamw/14711 "2022-02-14T15:21:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nitempe](https://avatars.discourse-cdn.com/v4/letter/n/54ee81/32.png) [@nitempe](https://discuss.huggingface.co/u/nitempe)\
**Post date:** [February 14, 2022, 3:21pm UTC](https://discuss.huggingface.co/t/huggingface-transformers-longformer-optimizer-warning-adamw/14711/1 "2022-02-14T15:21:25Z")

</div>

I get below warning when I try to run the code from this [page](https://github.com/jlealtru/website_tutorials/blob/main/notebooks/Longformer%20with%20IMDB.ipynb).

```
/usr/local/lib/python3.7/dist-packages/transformers/optimization.py:309: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use thePyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
  FutureWarning,

```

I am super confused because the code doesn’t seem to set the optimizer at all. The most probable places where the optimizer was set could be below but I dont know how to change the optimizer then

```
# define the training arguments
training_args = TrainingArguments(
    output_dir = '/media/data_files/github/website_tutorials/results',
    num_train_epochs = 5,
    per_device_train_batch_size = 8,
    gradient_accumulation_steps = 8,    
    per_device_eval_batch_size= 16,
    evaluation_strategy = "epoch",
    disable_tqdm = False, 
    load_best_model_at_end=True,
    warmup_steps=200,
    weight_decay=0.01,
    logging_steps = 4,
    fp16 = True,
    logging_dir='/media/data_files/github/website_tutorials/logs',
    dataloader_num_workers = 0,
    run_name = 'longformer-classification-updated-rtx3090_paper_replication_2_warm'
)

# instantiate the trainer class and check for available devices
trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_data,
    eval_dataset=test_data
)
device = 'cuda' if torch.cuda.is_available() else 'cpu'
device

```

I tried another transformer such as `distilbert-base-uncased` using the identical code but it seems to run without any warnings.

1. Is this warning more specific to `longformer`?
2. How should I change the optimizer?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [February 14, 2022, 4:13pm UTC](https://discuss.huggingface.co/t/huggingface-transformers-longformer-optimizer-warning-adamw/14711/2 "2022-02-14T16:13:27Z")

</div>

It’s a deprecation warning, so you will only get it once (that’s why you don’t see it for DistilBERT). To switch optimizer, put `optim="adamw_torch"` in your `TrainingArguments` (the default is `"adamw_hf"`)

---

<div class="post-metadata">

**Author:** ![Gerwin](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/gerwin/32/4548_2.png) [@Gerwin](https://discuss.huggingface.co/u/Gerwin)\
**Post date:** [April 25, 2022, 8:30am UTC](https://discuss.huggingface.co/t/huggingface-transformers-longformer-optimizer-warning-adamw/14711/3 "2022-04-25T08:30:18Z")

</div>

Under the hood, if you do not specify an optimizer/scheduler in the Trainer class, it will create an instance of AdamW with a linear scheduler. To overcome this problem, you can add an optimizer yourself by adding the argument ‘optimizers=(your\_optimizer, your\_scheduler)’ to the Trainer.