# Trainer do not log validation loss and metrics

**URL:** <https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850>\
**Category:** Beginners\
**Created:** [November 10, 2023, 8:01pm UTC](https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850 "2023-11-10T20:01:11Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![lanhf](https://avatars.discourse-cdn.com/v4/letter/l/f4b2a3/32.png) [@lanhf](https://discuss.huggingface.co/u/lanhf)\
**Post date:** [November 10, 2023, 8:01pm UTC](https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850/1 "2023-11-10T20:01:11Z")

</div>

Hello, today I use Trainer to train a Lora model, but there is no log for validation loss and metrics in the results of trainer.train(). The codes are as follows:

```auto
accuracy = evaluate.load("accuracy")

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return accuracy.compute(predictions=predictions, references=labels)

tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-base-v2")
dataset = load_from_disk("data")

def tokenize(batch):
    return tokenizer(batch["sentence"], truncation=True, max_length=150)

dataset = dataset.map(tokenize, batched=True).remove_columns(["sentence"])
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dataset.set_format(
    "torch", device=device, columns=["input_ids", "attention_mask", "labels"]
)
### input_ids must be the first column
dataset = dataset.map(lambda batch: {"new_labels": batch["labels"]}, batched=True)
dataset = dataset.remove_columns("labels")
dataset = dataset.rename_column("new_labels", "labels")

from peft import LoraConfig, TaskType, get_peft_model
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.allow_tf32 = True

peft_config = LoraConfig(
    TaskType.SEQ_CLS, "vinai/phobert-base-v2", r=8, lora_alpha=8, lora_dropout=0.1
)
model = AutoModelForSequenceClassification.from_pretrained(
    "vinai/phobert-base-v2", num_labels=2
)
model = get_peft_model(model, peft_config)

args = TrainingArguments(
    output_dir="./checkpoints",
    overwrite_output_dir=True,
    evaluation_strategy="epoch",
    per_device_eval_batch_size=64,
    per_device_train_batch_size=64,
    gradient_accumulation_steps=4,
    optim="adamw_torch_fused",
    tf32=True,
    learning_rate=5e-5,
    weight_decay=0.01,
    num_train_epochs=10,
    logging_strategy="epoch",
    save_strategy="epoch",
    dataloader_num_workers=10,
    remove_unused_columns=False
)

args.set_dataloader(auto_find_batch_size=True)

trainer = Trainer(
    model=model,
    args=args,
    tokenizer=tokenizer,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
    compute_metrics=compute_metrics,
)

trainer.train()

```

The output are:  
 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/f/a/fa28afa55cbdf22cebef4c3211173b3063fddf3f.png)

---

<div class="post-metadata">

**Author:** ![surya-narayanan](https://avatars.discourse-cdn.com/v4/letter/s/d2c977/32.png) [@surya-narayanan](https://discuss.huggingface.co/u/surya-narayanan)\
**Post date:** [March 1, 2024, 4:24am UTC](https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850/2 "2024-03-01T04:24:12Z")

</div>

Ive been dealing with this same issue, and it looks like you have to provide a name of a column that contains your labels, for example pass this as an argument to your trainer.

`label_names = ["labels"],`

---

<div class="post-metadata">

**Author:** ![waxef](https://avatars.discourse-cdn.com/v4/letter/w/b77776/32.png) [@waxef](https://discuss.huggingface.co/u/waxef)\
**Post date:** [March 2, 2024, 10:23am UTC](https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850/3 "2024-03-02T10:23:05Z")

</div>

Thanks for this reply, I was facing a [similar issue](https://discuss.huggingface.co/t/following-a-tutorial-on-hf-results-into-precision-is-ill-defined-and-being-set-to-0-0/75491).  
I haven’t tried it yet to see if it solves my solution as well.  
If you don’t mind me asking a clarifying question, is the argument `label_names` to the `trainer` dependent on what model one uses or is it a generic argument regardless of the underlying model that is being used?

As beginner with hf library on one hand we are all grateful that it exists on the other hand IMHO it’s a mess, it didn’t copy the good practices from other libraries like scikit or pytorch.

e.g. `from transformers import X` where `X` can be anything under the sun, instead of having a more structured approach `from transformers.models import X`, `from transformers.tokenizers import Y`, `from transformers.datasets import Z`, etc.

Aso , the whole `AutoModelXYZ` is utterly confusing, it would have been much more clear if we only had `from transformers.models import Model, ModelConfig` and then in the `ModelConfig` one defines whatever task they are interested in.

---

<div class="post-metadata">

**Author:** ![waxef](https://avatars.discourse-cdn.com/v4/letter/w/b77776/32.png) [@waxef](https://discuss.huggingface.co/u/waxef)\
**Post date:** [March 4, 2024, 9:11am UTC](https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850/4 "2024-03-04T09:11:04Z")

</div>

This didn’t work in my case 😢 😭

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/3/8/38e29c9fa0873f78178bd742dece1e9160a0bb35.png)

---

<div class="post-metadata">

**Author:** ![surya-narayanan](https://avatars.discourse-cdn.com/v4/letter/s/d2c977/32.png) [@surya-narayanan](https://discuss.huggingface.co/u/surya-narayanan)\
**Post date:** [March 5, 2024, 7:19pm UTC](https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850/5 "2024-03-05T19:19:47Z")

</div>

the keyword goes in TrainingArguments, not in Trainer

---

<div class="post-metadata">

**Author:** ![madog](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/madog/32/30849_2.png) [@madog](https://discuss.huggingface.co/u/madog)\
**Post date:** [September 1, 2024, 10:38am UTC](https://discuss.huggingface.co/t/trainer-do-not-log-validation-loss-and-metrics/61850/6 "2024-09-01T10:38:40Z")

</div>

I confirm that this fixes the error, thank you so much, it took some time to found the solution 🙂