# The way to get Seq2SeqLM's \`decoder\_input\_ids\` to obtain \`past\_key\_values\`

**URL:** <https://discuss.huggingface.co/t/the-way-to-get-seq2seqlms-decoder-input-ids-to-obtain-past-key-values/1733>\
**Category:** Beginners\
**Created:** [October 25, 2020, 5:48am UTC](https://discuss.huggingface.co/t/the-way-to-get-seq2seqlms-decoder-input-ids-to-obtain-past-key-values/1733 "2020-10-25T05:48:30Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![yusukemori](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/yusukemori/32/23302_2.png) [@yusukemori](https://discuss.huggingface.co/u/yusukemori)\
**Post date:** [October 25, 2020, 5:48am UTC](https://discuss.huggingface.co/t/the-way-to-get-seq2seqlms-decoder-input-ids-to-obtain-past-key-values/1733/1 "2020-10-25T05:48:30Z")

</div>

Sorry for the daily questions.

I’m working on `Seq2SeqLM` and thanks to the advice I’ve gotten on this forum, I’m able to move forward with understanding and implementation. Thank you so much!

Now, I would like to get `past` (or, `past_key_values`) in `Seq2SeqLM` as well as `CausalLM`.

> **[Model outputs](https://huggingface.co/docs/transformers/main_classes/output)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

> past\_key\_values (List[torch.FloatTensor], optional, returned when use\_cache=True is passed or when config.use\_cache=True) – List of torch.FloatTensor of length config.n\_layers, with each tensor of shape (2, batch\_size, num\_heads, sequence\_length, embed\_size\_per\_head)). Contains pre-computed hidden-states (key and values in the attention blocks) of the decoder that can be used (see past\_key\_values input) to speed up sequential decoding.

At first, I was having trouble getting `past_key_values` even though I confirm `use_cache = true`, but then I read the `class BartModel`’s implementation of `forward` and realized I need `decoder_input_ids` for inputs.

```python
        if decoder_input_ids is None:
            use_cache = False

```

Now I am wondering how to get the `decoder_input_ids` in `Seq2seqLM`.

I interpret as follows: `CausalLM` obtains `past_key_values` using the given context (or, prompt) and the output that comes from it in turn.

So how does `Seq2SeqLM` get the `decoder_input_ids` when it sequentially generates tokens with the given context after processing it with encoder?

Thank you in advance.

yusukemori