# Problem generating with T5ForConditionalGeneration on a custom task

**URL:** <https://discuss.huggingface.co/t/problem-generating-with-t5forconditionalgeneration-on-a-custom-task/137408>\
**Category:** 🤗Transformers\
**Created:** [January 26, 2025, 1:32pm UTC](https://discuss.huggingface.co/t/problem-generating-with-t5forconditionalgeneration-on-a-custom-task/137408 "2025-01-26T13:32:23Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [January 26, 2025, 1:41pm UTC](https://discuss.huggingface.co/t/problem-generating-with-t5forconditionalgeneration-on-a-custom-task/137408/2 "2025-01-26T13:41:27Z")

</div>

There seems to have been a problem with the output of T5 in a newer version of Transformers a while ago, so if it’s related to that, it’s easy…🤔

> <https://github.com/huggingface/transformers/issues/30892>
>
> \### System Info
> 
> \- \`transformers\` version: 4.41.0
> \- Platform: Linux-5.15.0-1033…-aws-x86\_64-with-glibc2.31
> \- Python version: 3.10.9
> \- Huggingface\_hub version: 0.23.0
> \- Safetensors version: 0.4.3
> \- Accelerate version: 0.30.0
> \- Accelerate config: not found
> \- PyTorch version (GPU?): 2.3.0+cu121 (True)
> \- Tensorflow version (GPU?): not installed (NA)
> \- Flax version (CPU?/GPU?/TPU?): not installed (NA)
> \- Jax version: not installed
> \- JaxLib version: not installed
> \- Using GPU in script?: Yes
> \- Using distributed or parallel set-up in script?: No
> 
> \### Who can help?
> 
> @ArthurZucker and @younesbelkada
> 
> \### Information
> 
> \- \[\] The official example scripts
> \- \[X\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> The following code breaks in \`v4.41.0\` (it works on earlier versions).
> 
> \`\`\`py
> import torch
> from transformers import GenerationConfig
> from transformers import T5ForConditionalGeneration
> 
> model = T5ForConditionalGeneration.from\_pretrained(
> "google/t5-efficient-tiny", device\_map="cuda"
> )
> input\_ids = torch.tensor(\[\[4, 5, 6, 6, 7\]\], device="cuda")
> model.generate(
> input\_ids=input\_ids,
> generation\_config=GenerationConfig(do\_sample=True),
> )
> \`\`\`
> 
> Error: 
> \`\`\`
> ValueError: \`decoder\_start\_token\_id\` or \`bos\_token\_id\` has to be defined for encoder-decoder generation.
> \`\`\`
> 
> \### Expected behavior
> 
> Expected generate to work like before without manually specifying \`decoder\_start\_token\_id\` or \`bos\_token\_id\` in the \`GenerationConfig\`.

> [@Unable to use model.generate for Vision encoder decoder model](https://discuss.huggingface.co/t/unable-to-use-model-generate-for-vision-encoder-decoder-model/75801):
>
> Here is my model configuration from transformers import ViTImageProcessor, BertTokenizer, VisionEncoderDecoderModel from datasets import load\_dataset model = VisionEncoderDecoderModel.from\_encoder\_decoder\_pretrained( "google/vit-base-patch16-224-in21k", "google-bert/bert-base-uncased" ) image\_processor = ViTImageProcessor.from\_pretrained("google/vit-base-patch16-224-in21k") tokenizer = BertTokenizer.from\_pretrained("google-bert/bert-base-uncased") model.config.decoder\_start\_token\_id = to…

---

_[View the full topic](https://discuss.huggingface.co/t/problem-generating-with-t5forconditionalgeneration-on-a-custom-task/137408)._