emotion-distilbert

distilbert-base-uncased fine-tuned to classify short English text into six emotions: sadness, joy, love, anger, fear, surprise.

Part of the project feedback-intelligence, where every number below was measured by code in that repository (see its development log for the full record).

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="BHARATHKUMAR0409/emotion-distilbert", top_k=None)
print(clf("i feel so helpless right now"))

Training data

dair-ai/emotion, English tweets with six emotion labels. Train rows with conflicting labels, exact duplicates, or texts also present in validation/test were removed (77 of 16,000 rows), leaving 15,923 for training. Validation (2,000) and test (2,000) were left unfiltered.

Training procedure

Setting	Value
Base model	distilbert-base-uncased (66,958,086 parameters)
Epochs	3 (best epoch by validation macro-F1: 3)
Batch size / learning rate	32 / 2e-5
Weight decay / warmup	0.01 / 10% linear warmup, linear decay
Max length	64 tokens (truncates 0.157% of train)
Precision	fp16 mixed precision
Seed	42
Hardware	one RTX 4050 Laptop GPU (6 GB), 103.65 s
Loss	standard cross-entropy (no class weights)

Evaluation (test split, 2,000 examples)

Model	Accuracy	Macro-F1
This model	0.9265	0.8873
Class-weighted variant (not published)	0.9225	0.8903
TF-IDF + Linear SVM, balanced (best classic baseline)	0.891	0.8441

Class	Precision	Recall	F1	Support
sadness	0.9606	0.9656	0.9631	581
joy	0.9559	0.9353	0.9455	695
love	0.7955	0.8805	0.8358	159
anger	0.9097	0.9164	0.913	275
fear	0.8982	0.9062	0.9022	224
surprise	0.8246	0.7121	0.7642	66

Validation: accuracy 0.9425, macro-F1 0.9146.

License and intended use

This model was fine-tuned on dair-ai/emotion, whose card states: "The dataset should be used for educational and research purposes only." Please treat this model the same way: use it for educational and research purposes. It is not licensed for commercial use. I am not a lawyer; if you need certainty, check the dataset terms yourself. The base model, distilbert-base-uncased, is released under Apache-2.0.

Citation

If you use this model, please also cite the dataset: Saravia, E., Liu, H.-C. T., Huang, Y.-H., Wu, J., Chen, Y.-S. (2018). CARER: Contextualized Affect Representations for Emotion Recognition. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3687-3697. https://www.aclweb.org/anthology/D18-1404

Limitations

• Rare classes are weaker. Surprise (3.5% of training data) has F1 0.7642 and recall 0.7121; love has F1 0.8358. The most common error is joy predicted as love.
• Label noise. Many misclassified examples are ambiguous or arguably mislabeled. I believe the dataset labels come from automatic keyword-based labeling, but I did not verify this.
• Single run. One seed per configuration; differences of about 0.01 macro-F1 between configurations are within noise. The validation score was still rising at epoch 3.
• Narrow domain. English tweets in a specific first-person style ("i feel ..."). Performance on other text types (reviews, emails, other languages) was not measured.
• Not for sensitive decisions. Do not use it to infer a person's mental state or for anything with real consequences for individuals.

Alternative

A class-weighted variant reached surprise recall 0.9697 but surprise precision 0.6737. It is not published here.