# Is there a way to return the "decoder\_input\_ids" from "tokenizer.prepare\_seq2seq\_batch"?

**URL:** <https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929>\
**Category:** 🤗Transformers\
**Created:** [December 23, 2020, 12:52am UTC](https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929 "2020-12-23T00:52:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![wgpubs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/wgpubs/32/303_2.png) [@wgpubs](https://discuss.huggingface.co/u/wgpubs)\
**Post date:** [December 23, 2020, 12:52am UTC](https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929/1 "2020-12-23T00:52:16Z")

</div>

Using BART as an example … this:

```auto
tokenizer.prepare_seq2seq_batch(
    src_texts=['This is a very short text', 'This is shorter'],
    tgt_texts=['very short', 'much shorter than very short'])

```

returns …

```auto
{'input_ids': [[100, 19, 3, 9, 182, 710, 1499, 1], [947, 19, 10951, 1, 0, 0, 0, 0]], 
'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 0, 0, 0, 0]], 
'labels': [[182, 710, 1, 0, 0, 0], [231, 10951, 145, 182, 710, 1]]}

```

For fine-tuning, how should we build the `decoder_input_ids`? And do we also need to shift the `labels` to the right so that they look like this?

```auto
[[710, 1, 0, 0, 0, 0], [10951, 145, 182, 710, 1, 0]]

```

---

<div class="post-metadata">

**Author:** ![wgpubs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/wgpubs/32/303_2.png) [@wgpubs](https://discuss.huggingface.co/u/wgpubs)\
**Post date:** [December 23, 2020, 1:57am UTC](https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929/2 "2020-12-23T01:57:17Z")

</div>

Or do we even have to pass `decoder_input_ids` anymore???

Looking at this example for MT5, it looks like hte answer is “no” …

```auto
from transformers import MT5ForConditionalGeneration, T5Tokenizer
model = MT5ForConditionalGeneration.from_pretrained("google/mt5-small")
tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
summary = "Weiter Verhandlung in Syrien."
batch = tokenizer.prepare_seq2seq_batch(src_texts=[article], tgt_texts=[summary], return_tensors="pt")
outputs = model(**batch)
loss = outputs.loss

```

This sure would make it easier if all we have to pass in are the “labels” and not have to deal with the `decoder_input_ids` ourselves when working within ConditionalGeneration models. Please lmk either way.

Thanks

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [December 23, 2020, 6:11am UTC](https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929/3 "2020-12-23T06:11:16Z")

</div>

Hi @wgpubs,

if you just pass `labels` the `decoder_input_ids ` are prepared inside the model by shifting the `labels`. See

> <https://github.com/huggingface/transformers/blob/cbe63949d76efd153a1f389f38fe9ce1287e06b0/src/transformers/models/bart/modeling_bart.py#L1229-L1231>

---

<div class="post-metadata">

**Author:** ![wgpubs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/wgpubs/32/303_2.png) [@wgpubs](https://discuss.huggingface.co/u/wgpubs)\
**Post date:** [December 23, 2020, 11:08pm UTC](https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929/4 "2020-12-23T23:08:55Z")

</div>

Ah cool … thanks for confirming.

---

<div class="post-metadata">

**Author:** ![wgpubs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/wgpubs/32/303_2.png) [@wgpubs](https://discuss.huggingface.co/u/wgpubs)\
**Post date:** [December 26, 2020, 7:42pm UTC](https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929/5 "2020-12-26T19:42:19Z")

</div>

Are the target tokens (the labels) replaced with the ignore token id somewhere as well? Doesn’t look like it from what I can see … so I’m assuming we need to do that ourselves, and pass the label ids with padding tokens set to -100.

Also, the `decoder_input_ids` come back in the form of `<eos> <bos> X ...`, but my understanding was always that it should start with `<bos>` and the `labels` shifted so that `<bos>` attempts to predict `X[0]` and so forth.

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [December 29, 2020, 6:47am UTC](https://discuss.huggingface.co/t/is-there-a-way-to-return-the-decoder-input-ids-from-tokenizer-prepare-seq2seq-batch/2929/6 "2020-12-29T06:47:44Z")

</div>

Yes, we should manually replace pad token with -100 in `labels`.

Ideally yes, it should start with `bos` token, but in the original `fairseq` implementation the models are trained with `<eos> <bos> X ....` , so we have kept it like that for reproducibility.