# Problems with trainer.compute\_metrics

**URL:** <https://discuss.huggingface.co/t/problems-with-trainer-compute-metrics/79771>\
**Category:** 🤗Transformers\
**Created:** [April 2, 2024, 6:41am UTC](https://discuss.huggingface.co/t/problems-with-trainer-compute-metrics/79771 "2024-04-02T06:41:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![virt99](https://avatars.discourse-cdn.com/v4/letter/v/898d66/32.png) [@virt99](https://discuss.huggingface.co/u/virt99)\
**Post date:** [April 2, 2024, 6:41am UTC](https://discuss.huggingface.co/t/problems-with-trainer-compute-metrics/79771/1 "2024-04-02T06:41:38Z")

</div>

hello, i use transformers.trainer to fine-tune the codellama-7b-python. i want using bleu to eval model during training, but when i rewrite the compute\_metric function, i decode the lables and input\_ids,and i find that the decode result is not the primary text i encode. here is my encode function

def tokenize2( prompt, tokenizer, cutoff\_len, padding=False):  
result = tokenizer(  
prompt,  
truncation=True,  
max\_length=cutoff\_len,  
padding=‘max\_length’ if padding else False,  
return\_tensors=None  
)  
return {  
“input\_ids”: result[“input\_ids”],  
“labels”: copy.deepcopy(result[“input\_ids”])  
}  
def tokenize\_prompt(data\_point:Dict, tokenizer=tokenizer, raw\_source\_len=256, cutoff\_len=512):  
# tokenized\_result = self.src\_tokenize(src, tokenizer, cutoff\_len)

```
    src=data_point['text']
    tgt=data_point['target']
    tokenized_result = tokenize2(src, tokenizer, raw_source_len, padding=False)

    source_len = len(tokenized_result['input_ids'])

    assert source_len<=raw_source_len
    assert len(tgt)>0

    src = tokenizer.decode(tokenized_result['input_ids'], skip_special_tokens=True, clean_up_tokenization_spaces=True)
    prompt_with_response = src + tgt + " " + tokenizer.eos_token

    tokenized_with_response = tokenize2(prompt_with_response, tokenizer, cutoff_len, padding=False)

    tokenized_with_response["labels"] = [-100] * source_len + tokenized_with_response["labels"][source_len:]
    # print(tokenizer.decode(tokenized_with_response["labels"], skip_special_tokens=True,
    # clean_up_tokenization_spaces=True))

    assert len(tokenized_with_response["input_ids"]) == len(tokenized_with_response["labels"])

    return {'input_ids':tokenized_with_response["input_ids"],'labels': tokenized_with_response["labels"]}

```

---

<div class="post-metadata">

**Author:** ![newUser69w](https://avatars.discourse-cdn.com/v4/letter/n/ea5d25/32.png) [@newUser69w](https://discuss.huggingface.co/u/newUser69w)\
**Post date:** [September 15, 2024, 1:09pm UTC](https://discuss.huggingface.co/t/problems-with-trainer-compute-metrics/79771/2 "2024-09-15T13:09:03Z")

</div>

Hey there, have you found a solution for this?

Best,  
Joe