# Rewriting generate function for manual decoder input

**URL:** <https://discuss.huggingface.co/t/rewriting-generate-function-for-manual-decoder-input/3034>\
**Category:** 🤗Transformers\
**Created:** [January 3, 2021, 4:03pm UTC](https://discuss.huggingface.co/t/rewriting-generate-function-for-manual-decoder-input/3034 "2021-01-03T16:03:23Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![patrickvonplaten](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/patrickvonplaten/32/2171_2.png) [@patrickvonplaten](https://discuss.huggingface.co/u/patrickvonplaten)\
**Post date:** [January 3, 2021, 10:10pm UTC](https://discuss.huggingface.co/t/rewriting-generate-function-for-manual-decoder-input/3034/2 "2021-01-03T22:10:50Z")

</div>

Hey @marcoabrate,

The current version of `generate` (and also the one of v4.1.1.) already includes the possibility to provide user specific `decoder_input_ids`. You just have to add it to `generate()`.

E.g. the following code works as expected

```
from transformers import T5ForConditionalGeneration, T5TokenizerFast

model = T5ForConditionalGeneration.from_pretrained("t5-small")
tokenizer = T5TokenizerFast.from_pretrained("t5-small")

input_ids = tokenizer("translate English to German: How are you?", return_tensors="pt").input_ids
decoder_input_ids = tokenizer("<pad> Wie geht", return_tensors="pt", add_special_tokens=False).input_ids

output = model.generate(input_ids, decoder_input_ids=decoder_input_ids, num_beams=4, num_return_sequences=4)

print("With decoder_input_ids num_beams=4", tokenizer.batch_decode(output, skip_special_tokens=True))

output = model.generate(input_ids, num_beams=4, num_return_sequences=4)

print("Without decoder_input_ids num_beams=4", tokenizer.batch_decode(output, skip_special_tokens=True)) 

```

Also see this notebook for the answers of this specific use case: [https://colab.research.google.com/drive/11js9We6ZtjN15hb3-PoFZBXJrcSOo\_Qa?usp=sharing](https://colab.research.google.com/drive/11js9We6ZtjN15hb3-PoFZBXJrcSOo_Qa?usp=sharing)

This feature was enabled by the generate refactor: [Big `generate()` refactor](https://discuss.huggingface.co/t/big-generate-refactor/1857/6)

Does this correspond to what you were trying to achieve or are you looking for some other behavior?

---

_[View the full topic](https://discuss.huggingface.co/t/rewriting-generate-function-for-manual-decoder-input/3034)._