# T5 - model.generate() issue

**URL:** <https://discuss.huggingface.co/t/t5-model-generate-issue/15644>\
**Category:** Beginners\
**Created:** [March 13, 2022, 8:28am UTC](https://discuss.huggingface.co/t/t5-model-generate-issue/15644 "2022-03-13T08:28:50Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rakshith291](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Rakshith291](https://discuss.huggingface.co/u/Rakshith291)\
**Post date:** [March 13, 2022, 8:28am UTC](https://discuss.huggingface.co/t/t5-model-generate-issue/15644/1 "2022-03-13T08:28:50Z")

</div>

I have finetuned a T5-model on custom dataset, so  
if I do  
model(input\_ids=inputs.input\_ids, attention\_mask=inputs.attention\_mask, decoder\_input\_ids=batch[“decoder\_input\_ids”],labels=batch[“labels”],decoder\_attention\_mask=batch[“decoder\_attention\_mask”]).loss

I’m getting around 0.003 loss, so I assume that’s an indication model has been trained well.

But if I use model.generate()-\> I’m getting a random or trash as output, I was wondering where i’m going wrong?

Please can anyone help me with this

---

<div class="post-metadata">

**Author:** ![Rakshith291](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Rakshith291](https://discuss.huggingface.co/u/Rakshith291)\
**Post date:** [March 14, 2022, 10:05am UTC](https://discuss.huggingface.co/t/t5-model-generate-issue/15644/2 "2022-03-14T10:05:09Z")

</div>

It’s resolved now,

I specifically added decoded input ids in the argument, I assumed it will be right shifted while training but that’s not the case.  
In order to right shift the target sequence, only labels should be provided in the argument

---

<div class="post-metadata">

**Author:** ![waxef](https://avatars.discourse-cdn.com/v4/letter/w/b77776/32.png) [@waxef](https://discuss.huggingface.co/u/waxef)\
**Post date:** [March 18, 2024, 12:26pm UTC](https://discuss.huggingface.co/t/t5-model-generate-issue/15644/3 "2024-03-18T12:26:48Z")

</div>

Hi, I’m having the same issue.

I assume the solution here was to:

either remove the `decoder_input_ids` from the model forward call

```auto
model(input_ids=batch['input_ids'], attention_mask=batch['attention_mask'], labels=batch['labels'])

```

or manually `shift_right` the `decoder_input_ids`.

Do we also have to `shift_right` the `decoder_attention_mask` as well in this case if we were to pass `decoder_input_ids` and `decoder_attention_mask` to model?