# GPT2 Conditional Text Generation

**URL:** <https://discuss.huggingface.co/t/gpt2-conditional-text-generation/1278>\
**Category:** Models\
**Created:** [September 28, 2020, 3:13pm UTC](https://discuss.huggingface.co/t/gpt2-conditional-text-generation/1278 "2020-09-28T15:13:25Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![manzar](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@manzar](https://discuss.huggingface.co/u/manzar)\
**Post date:** [September 28, 2020, 3:13pm UTC](https://discuss.huggingface.co/t/gpt2-conditional-text-generation/1278/1 "2020-09-28T15:13:25Z")

</div>

Hello,  
I would like to fine-tune the GPT2 model on EmpatheticDialogues doing kind of conditional generation as like in this paper: [https://arxiv.org/pdf/1911.11161.pdf](https://arxiv.org/pdf/1911.11161.pdf)  
What concerns me is the format of the input\_ids and labels in the forward function.  
I think that concatenating the input with the target is a good solution separating them with a special token  
(e.g. "hi! how are you? I am fine!)  
However I am not sure what to do with the labels. Shall I mask all the input part and the padded tokens with -100 index and leave only the target part as is? or shall I mask with -100 only the padded tokens?

Thank you in advance 🙂