# Trainer class optimization for transformer models

**URL:** <https://discuss.huggingface.co/t/trainer-class-optimization-for-transformer-models/13484>\
**Category:** Models\
**Created:** [January 8, 2022, 1:01am UTC](https://discuss.huggingface.co/t/trainer-class-optimization-for-transformer-models/13484 "2022-01-08T01:01:06Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jeffreyyu0602](https://avatars.discourse-cdn.com/v4/letter/j/eb8c5e/32.png) [@jeffreyyu0602](https://discuss.huggingface.co/u/jeffreyyu0602)\
**Post date:** [January 8, 2022, 1:01am UTC](https://discuss.huggingface.co/t/trainer-class-optimization-for-transformer-models/13484/1 "2022-01-08T01:01:06Z")

</div>

I am following the “fine-tuning a pretrained model” on the huggingface transformers tutorial. I replaced the bert-base-uncased model with google/mobilebert-uncased. I can get decent result using a huggingface Trainer, but when I trained using native PyTorch, the accuracy stucks around 50%. I am wondering is the huggingface Trainer doing some optimization that I must explicitly add in the native PyTorch code in order to work? Thank you!