# Encoder-decoder \`generate()\` with forced start for decoder?

**URL:** <https://discuss.huggingface.co/t/encoder-decoder-generate-with-forced-start-for-decoder/26285>\
**Category:** 🤗Transformers\
**Created:** [November 18, 2022, 3:06am UTC](https://discuss.huggingface.co/t/encoder-decoder-generate-with-forced-start-for-decoder/26285 "2022-11-18T03:06:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![astariul](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@astariul](https://discuss.huggingface.co/u/astariul)\
**Post date:** [November 18, 2022, 3:06am UTC](https://discuss.huggingface.co/t/encoder-decoder-generate-with-forced-start-for-decoder/26285/1 "2022-11-18T03:06:47Z")

</div>

I’m training an encoder-decoder model (BART) on my task. My task is spell checking, I framed it as a seq2seq task, so for example given the sentence `I liike thus framewok much.`, the model is trained to predict the corrected sentence : `I like this framework much.`.

Now at inference time, I would like to force the beginning of the decoder sentence. For example for the partial sentence `I like this framewok` :

- I want to input `I like this framewok` in the encoder
- I want to give `I like this` in the decoder, and let the decoder predict the next word (in this case if the model is well-trained, it should predict `framework`).

How can I achieve this goal with the `generate()` method ?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [November 18, 2022, 7:59am UTC](https://discuss.huggingface.co/t/encoder-decoder-generate-with-forced-start-for-decoder/26285/2 "2022-11-18T07:59:12Z")

</div>

cc’ing @joaogante here

---

<div class="post-metadata">

**Author:** ![astariul](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@astariul](https://discuss.huggingface.co/u/astariul)\
**Post date:** [November 18, 2022, 10:21am UTC](https://discuss.huggingface.co/t/encoder-decoder-generate-with-forced-start-for-decoder/26285/3 "2022-11-18T10:21:03Z")

</div>

I could get something to work by using the keyword argument `decoder_input_ids` :

```auto
# Create the input for the encoder : the sentence with typo
encoder_inp = tokenizer(["I like this framewok"], max_length=model.config.max_position_embeddings, padding=True, truncation=True, return_tensors="pt")

# Create the input for the decoder : first part of the sentence, without typo
decoder_inp = tokenizer(["I like this"], max_length=model.config.max_position_embeddings, padding=True, truncation=True, return_tensors="pt")

# Remove the EOS token generated by the tokenizer
decoder_inp["input_ids"] = decoder_inp["input_ids"][:, :-1]

# Then call the generate method with the encoder's input AND decoder's initial input
out = model.generate(encoder_inp["input_ids"], num_beams=2, min_length=1, max_length=model.config.max_position_embeddings, decoder_input_ids=decoder_inp["input_ids"])

```

This seems to work as I expect it to, but I’d love to get a second opinion from someone more knowledgeable !

---

<div class="post-metadata">

**Author:** ![joaogante](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/joaogante/32/20106_2.png) [@joaogante](https://discuss.huggingface.co/u/joaogante)\
**Post date:** [November 18, 2022, 10:44am UTC](https://discuss.huggingface.co/t/encoder-decoder-generate-with-forced-start-for-decoder/26285/4 "2022-11-18T10:44:29Z")

</div>

Hi @astariul – yup, using `decoder_input_ids` is precisely what you should do 👍