# Decoder vs Encoder-decoder clarification

**URL:** <https://discuss.huggingface.co/t/decoder-vs-encoder-decoder-clarification/44330>\
**Category:** Beginners\
**Created:** [June 23, 2023, 4:13pm UTC](https://discuss.huggingface.co/t/decoder-vs-encoder-decoder-clarification/44330 "2023-06-23T16:13:28Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![patrick-dench](https://avatars.discourse-cdn.com/v4/letter/p/58f4c7/32.png) [@patrick-dench](https://discuss.huggingface.co/u/patrick-dench)\
**Post date:** [June 23, 2023, 4:13pm UTC](https://discuss.huggingface.co/t/decoder-vs-encoder-decoder-clarification/44330/1 "2023-06-23T16:13:28Z")

</div>

I’m following the HF nlp course and I’m coming across a hurdle as I try to establish my ‘mental model’.

Specifically, the tutorial calls out that ‘Decoder models are suited for text generation from a prompt’ and that ‘Encoder-decoder models are suited for input summarization’. In my mind, I’m imagining working with ChatGPT where “everything is a prompt” and “everything is input”.

How does the decoder/encoder-decoder relationship compare? Isn’t ‘everything a prompt’ with regards to input into either a decoder or encoder-decoder model?

---

<div class="post-metadata">

**Author:** ![dblakely](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@dblakely](https://discuss.huggingface.co/u/dblakely)\
**Post date:** [June 26, 2023, 2:56pm UTC](https://discuss.huggingface.co/t/decoder-vs-encoder-decoder-clarification/44330/2 "2023-06-26T14:56:54Z")

</div>

Could you clarify what you’re asking in this part of your post?

> [@patrick-dench](#):
>
> Isn’t ‘everything a prompt’ with regards to input into either a decoder or encoder-decoder model?

Are you asking what’s the difference between a “prompt” (the input to a decoder-only) and the input to an encoder-decoder model?

---

<div class="post-metadata">

**Author:** ![patrick-dench](https://avatars.discourse-cdn.com/v4/letter/p/58f4c7/32.png) [@patrick-dench](https://discuss.huggingface.co/u/patrick-dench)\
**Post date:** [June 27, 2023, 8:08pm UTC](https://discuss.huggingface.co/t/decoder-vs-encoder-decoder-clarification/44330/3 "2023-06-27T20:08:14Z")

</div>

Essentially, yes. I understand the concept of decoder-only vs encoder-decoder (sort of). Is there any ‘conceptual’ difference between “prompt” and “input” or is it simply semantic in order to draw a distinction between decoder-only and encoder/decoder?

---

<div class="post-metadata">

**Author:** ![dblakely](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@dblakely](https://discuss.huggingface.co/u/dblakely)\
**Post date:** [August 1, 2023, 6:31pm UTC](https://discuss.huggingface.co/t/decoder-vs-encoder-decoder-clarification/44330/4 "2023-08-01T18:31:16Z")

</div>

Sorry for the late response.

> Is there any ‘conceptual’ difference between “prompt” and “input” or is it simply semantic in order to draw a distinction between decoder-only and encoder/decoder?

I think there are both some semantic and technical differences. When people say “prompt”, what they usually mean is something like “an input to a decoder-only model that contains instructions or examples of the task to be completed (aka a ‘few-shot prompt’).” On the other hand, the “input” to an encoder-decoder model is generally not really a “prompt”, it’s just an input that the model is supposed to do something to [1] (e.g., it’s a sentence in French and we want the model to ingest it and translate it into English). So in this sense, “prompt” vs “encoder-decoder input” is basically a semantic thing.

But there are some technical differences between the input to a decoder-only model and the input to an enc-dec model, just given the fact that they are different architectures. E.g., when you feed in a input (regardless of what you call it) to a decoder-only model, it’ll process the input with unidirectional attention (every token attends to the previous tokens), while the encoder of an enc-dec uses bidirectional attention (every input token attends to all the other tokens).

You can also think of decoder-only models as just having one big set of parameters that are used to process both the input (or prompt) and generate the output. Meanwhile, enc-dec models effectively have one set of parameters for processing the input (the encoder) and another set for generating the output (the decoder).

[1] This is a bit of a simplification though - there are instruction-tuned T5 models (flan-t5, flan-t5-ul2) which are trained to have instructions fed into the encoder. In which case, you’d probably also refer to the inputs as a “prompt”