# T5 models: About the decoder\_input\_ids argument

**URL:** <https://discuss.huggingface.co/t/t5-models-about-the-decoder-input-ids-argument/27270>\
**Category:** Models\
**Created:** [December 5, 2022, 2:39pm UTC](https://discuss.huggingface.co/t/t5-models-about-the-decoder-input-ids-argument/27270 "2022-12-05T14:39:11Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![tom192180](https://avatars.discourse-cdn.com/v4/letter/t/f0a364/32.png) [@tom192180](https://discuss.huggingface.co/u/tom192180)\
**Post date:** [December 5, 2022, 2:39pm UTC](https://discuss.huggingface.co/t/t5-models-about-the-decoder-input-ids-argument/27270/1 "2022-12-05T14:39:11Z")

</div>

I read online tutorial about implementation fine-tuning from the website:

> **[Fine Tuning T5 Transformer Model with PyTorch](https://shivanandroy.com/fine-tune-t5-transformer-with-pytorch/)**
>
> In this article, you will learn how to fine tune a T5 model with PyTorch and transformers

I do not know why for the decoder\_input\_ids and labels, the author removed last and first token ID, respectively.

See the following codes

```auto
for _, data in enumerate(loader, 0):
        y = data["target_ids"].to(device, dtype=torch.long)
       y_ids = y[:, :-1].contiguous()

        lm_labels = y[:, 1:].clone().detach()
        lm_labels[y[:, 1:] == tokenizer.pad_token_id] = -100
        ids = data["source_ids"].to(device, dtype=torch.long)
        mask = data["source_mask"].to(device, dtype=torch.long)
        outputs = model(
            input_ids=ids,
            attention_mask=mask,
            decoder_input_ids=y_ids,
            labels=lm_labels,
        )

```

I read the original paper and hugging-face documents but I still do not understand. Could anyone tell me?