# Weird losses while fine tuning

**URL:** <https://discuss.huggingface.co/t/weird-losses-while-fine-tuning/10073>\
**Category:** Beginners\
**Created:** [September 17, 2021, 9:09am UTC](https://discuss.huggingface.co/t/weird-losses-while-fine-tuning/10073 "2021-09-17T09:09:56Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![marlon89](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@marlon89](https://discuss.huggingface.co/u/marlon89)\
**Post date:** [September 17, 2021, 9:09am UTC](https://discuss.huggingface.co/t/weird-losses-while-fine-tuning/10073/1 "2021-09-17T09:09:56Z")

</div>

Hello,

I got a question regarding fine tuning a BERT Sentiment model. I am working with Customer Feedback data and want to fine tune “nlptown/bert-base-multilingual-uncased-sentiment” for my specific usecase as there are always patterns for wrong classified sentiments no matter which pretrained model I use. However, when I fine tune it I get weird losses like this:  
 ![Screenshot (43)](https://us1.discourse-cdn.com/hellohellohello/original/2X/0/0c810ae31fdd495715ad2c35907d22af74fd2748.png)

These loss jumparounds appear in the same pattern no matter if I reduce the learning rate, choose different batch sizes or update my weight decay. I am working with a Trainer() insance. Nevertheless, if I run the new trained model with test data the results are looking fine. Now finally my question: Can I still use this model? The losses indicate overfitting, don’t they?