Encoder, Decoder, and Encoder-Decoder Architectures

Three ways to arrange a transformer stack — encoder-only, decoder-only, and encoder-decoder — differ in one thing: which positions can see which. That choice determines what each is good for.

The same transformer block can be arranged three ways, and the arrangements are not interchangeable. Encoder-only, decoder-only, and encoder-decoder each suit a different kind of task. The difference between them reduces to one design decision: which positions are allowed to attend to which. Get that single idea and the three families fall into place — along with why “an LLM” almost always means one specific arrangement.

The one variable that matters

Everything below is really about the attention mask — the rule for what each position can look at.

• Bidirectional attention: every position can attend to every other position, both earlier and later. Each token sees the full context on both sides.
• Causal attention: every position can attend only to itself and earlier positions. The future is masked out.

That single switch — can a token see what comes after it, or not — is what separates the three architectures. The self-attention mechanism is identical in all of them; only the mask changes. Hold that, and the rest is consequences.

Encoder-only: built to understand

An encoder-only model uses bidirectional attention throughout. Every token sees the entire input at once, both directions. This is ideal for understanding a fixed piece of text: classification, sentiment, extracting spans, or producing a rich embedding of a whole sentence. BERT-style models are the well-known representatives of this family.

The catch is that bidirectional attention makes text generation incoherent. If every position can see every other, there is no meaningful “predict the next token” task — the answer is already visible on the right. So encoder-only models are trained differently: typically by masking out random tokens and asking the model to fill them in, using context from both sides. That objective produces excellent representations of existing text but no ability to generate a continuation. An encoder-only model reads; it does not write.

Decoder-only: built to generate

A decoder-only model uses causal attention throughout. Every token sees only itself and what came before, never the future. This is precisely what next-token prediction requires: the model must not peek at the token it is trying to predict. When people say “a language model,” this is nearly always the architecture they mean.

The causal mask is what makes the autoregressive generation loop coherent — the model produces one token, appends it, and predicts again, never having seen an answer it was not entitled to. It also enables the KV cache: because past positions can never attend to future ones, their computed keys and values never change as generation proceeds and can be stored and reused.

The trade-off runs opposite to the encoder. A decoder-only model, restricted to looking backward, builds a slightly weaker representation of a fixed input than a bidirectional model would — but it can generate, which is the capability that turned out to matter most. That is why the field converged on this family for general-purpose models.

Encoder-decoder: built to transform

The original transformer was neither of the above — it was an encoder-decoder, designed for translation, where the task is to transform one sequence into another. It combines both stacks. This is the transformer form of the classic sequence-to-sequence design.

• The encoder reads the entire input bidirectionally and produces a set of context representations.
• The decoder generates the output causally, one token at a time — and adds a third ingredient: cross-attention, where the decoder’s queries attend to the encoder’s keys and values.

Cross-attention is the bridge. At each output step, the decoder looks back at the fully-understood input to decide what to produce next. This suits tasks with a clear input-to-output mapping where both sides are substantial: translation, summarization, speech transcription. The encoder gets full bidirectional understanding of the source; the decoder gets proper causal generation of the target; cross-attention connects them.

Choosing among the three

The pattern is clean once you see it through the mask:

Why did decoder-only win as the default for large models? Partly because it does one thing that scales beautifully — next-token prediction on unlabeled text — and partly because a large enough decoder-only model handles “understanding” and “transformation” tasks well enough by framing them as generation. You can ask a decoder-only model to classify or translate simply by prompting it, whereas an encoder-only model cannot be prompted to generate at all. Generality beat specialization.

What to remember

• The three architectures differ in one thing: the attention mask — bidirectional (see everything) versus causal (see only the past).
• Encoder-only (bidirectional): best for understanding fixed text — classification, embeddings — but cannot generate.
• Decoder-only (causal): best for generation, makes next-token prediction coherent and the KV cache possible; this is what “LLM” usually means.
• Encoder-decoder: a bidirectional encoder plus a causal decoder joined by cross-attention, suited to transforming one sequence into another.
• Decoder-only became the default because next-token prediction scales on unlabeled data and a large model can handle understanding and transformation through prompting.

Next: Decoding Strategies — once a decoder produces a distribution, how do you turn it into a sequence?