Encoder, Decoder & Encoder-Decoder Models

TL;DR

Intuition

The Transformer block from [1] is one architectural unit, but it can be used three ways depending on what you mask during training and inference. Think of the difference as what direction information flows:

• Bidirectional. Every token sees every other token. Good for understanding a fixed input. Cannot generate by default because there is no left-to-right ordering to exploit.
• Left-to-right (causal). Each token sees only the past. Naturally generative. Slightly worse at pure understanding tasks at small scale because right-context is unavailable, but the gap closes (and reverses) as models get bigger.
• Encoder-decoder. The input runs through a bidirectional encoder, then a causal decoder generates the output one token at a time, with cross-attention to the encoder output. Best when the input and output are different shapes (translate English to French, summarize a document, transcribe audio).

A short visual taxonomy:

The three inner regions are disjoint at the architecture level but overlap heavily in their building blocks: every one of them is a stack of pre-norm or post-norm Transformer blocks with multi-head attention, a position-wise FFN, and a residual stream. What differs is whether the attention mask is full (encoder), upper-triangular (decoder), or both at once (encoder-decoder), and what objective the gradient comes from.

BERT and the encoder-only family

BERT [2] took the encoder half of the original Transformer and asked: what if we pretrain it on raw text with a self-supervised objective, then fine-tune for downstream tasks? The result was the model that, in 2018, briefly held SOTA on essentially every natural-language understanding benchmark.

RoBERTa: the same model, trained right

RoBERTa [3] is BERT-large with a re-tuned recipe: 10x more pretraining data (160 GB vs 16 GB), longer training, larger batches, dynamic masking (re-mask every epoch instead of once), no NSP, no shorter-sequence ramp-up. Same architecture, no new ideas, just a more careful experimental sweep. It outperforms the original BERT by 5-10 points across GLUE and SQuAD, and the lesson stuck: BERT was undertrained. The same lesson would re-emerge for LLMs four years later (see Scaling Laws & Training).

DeBERTa: disentangled attention

DeBERTa [4] introduces two refinements:

1. Disentangled attention. Each token is represented by two vectors: a content vector $H$ and a position vector $P$. The attention score is computed as a sum of four cross-product terms, $H_i^\top H_j + H_i^\top P_j + P_i^\top H_j + P_i^\top P_j$, instead of one. The intuition: content-content interaction is fundamentally different from position-content interaction, so let the model learn them separately. In practice the diagonal terms ($P_i^\top P_j$) are dropped or simplified.
2. Enhanced mask decoder. At pretraining, the absolute positions of masked tokens are injected only at the very last decoding layer (not as input embeddings). This prevents the model from “cheating” by exploiting absolute-position artefacts during MLM.

The wins are modest but consistent: DeBERTa-v3 sits on top of the SuperGLUE leaderboard for several years.

The GPT family and decoder-only models

Where BERT keeps the encoder half of the original Transformer, GPT keeps the decoder half (without the cross-attention sublayer). Pretraining is the simplest possible language-model objective: maximize the probability of the next token given the past.

The causal mask in self-attention enforces that position $t$ can only attend to positions $1, \dots, t-1$, so the same forward pass produces $n$ independent loss terms (one per position) that the optimizer sums.

1. 2018GPT-1: 117M params, BookCorpus pretrain + supervised fine-tunepaper

   Radford et al. show that pretraining a Transformer decoder on raw text and then fine-tuning the same model on downstream tasks beats every earlier non-pretrained baseline. The recipe: pretrain on next-token prediction, fine-tune the whole model with a small classification head on the final hidden state of the last token.

2. 2019GPT-2: 1.5B params, WebText pretrain, zero-shot framingpaper

   Radford et al. push the scale 13x and show the model can do many tasks (translation, QA, summarization) with zero gradient updates by phrasing the task in natural language at the prompt. The phrase "language models are unsupervised multitask learners" enters the field.

3. 2020GPT-3: 175B params, in-context learning emergespaper

   Brown et al. scale 100x further. The headline result: with a few worked examples in the prompt (few-shot prompting), GPT-3 matches or beats fine-tuned smaller models on many benchmarks, with no parameter updates. In-context learning becomes the new transfer-learning interface.

4. 2022InstructGPT and RLHFsystem

   Ouyang et al. fine-tune GPT-3 on human-written instructions and align it with reinforcement learning from human feedback. This produces ChatGPT, which makes prompted instruction-following the dominant interaction pattern.

5. 2023Llama 1, Llama 2, Mistral 7Brelease

   Open-weight decoder-only models match closed GPT-3.5 quality at 7B-70B scale, on the same recipe (causal LM, RoPE positions, pre-norm, RMSNorm, GQA at 70B). The decoder-only template is now industry default for general-purpose LLMs.

What changes from GPT-1 to GPT-3 architecturally

Almost nothing. All three are pre-norm (GPT-1 used post-norm; GPT-2 onward switched to pre-norm) decoder-only Transformers with causal self-attention, learned absolute positional embeddings up to $n_{\max}$ (1024 for GPT-2, 2048 for GPT-3), and a 50K BPE tokenizer. What changes is depth, width, and pretraining tokens:

• GPT-1: 12 layers, $d = 768$, 117M params, 5 GB pretraining (BookCorpus).
• GPT-2: 48 layers, $d = 1600$, 1.5B params, 40 GB pretraining (WebText).
• GPT-3: 96 layers, $d = 12288$, 175B params, 300B-token pretraining (filtered Common Crawl + WebText2 + books + Wikipedia).

The 1500x compute increase from GPT-1 to GPT-3 is the entire story. New capabilities (zero-shot prompting, then few-shot in-context learning) appear at scale without any architecture change. Brown et al. [7] demonstrate this directly: a 13B GPT-3 still does few-shot poorly; a 175B GPT-3 does it well; the only thing different is parameter count and training compute.

Why decoder-only won for general-purpose LLMs

Three reasons, in roughly increasing order of importance:

1. Generation is the universal interface. A decoder can do classification by generating a label, span extraction by generating the span, summarization by generating the summary, translation by generating the target language. An encoder cannot generate without surgery. Once you commit to “the user asks a natural-language question and the model writes a natural-language answer”, decoder-only is the natural shape.
2. Causal LM gives you $n$ training signals per sequence. A length-$n$ sequence produces $n$ independent next-token loss terms in one forward pass. MLM produces 0.15$n$ loss terms (15% of positions are masked). At equal compute, causal LM trains roughly 6x more efficiently per parameter, which compounds at scale.
3. In-context learning emerges from scale. GPT-3 [7] showed that a sufficiently large autoregressive language model develops the ability to learn new tasks from worked examples in the prompt, with no gradient updates. This is the property that turned LLMs from “fine-tunable embeddings” into a reasoning interface, and it appears in causal LMs, not in MLMs. The reason is mechanical: the prompt is the training distribution for the predicted continuation, so a sufficiently capable next-token predictor must implicitly run a Bayesian inference over the prompt to do well.

A useful heuristic: if your task is “given an input, label or extract” and your latency budget is tight, encoder-only beats decoder-only on quality per dollar. If your task is “given a description, produce open-ended text” or “answer arbitrary questions”, decoder-only is the only sane choice.

T5 and the encoder-decoder family

T5 [8] takes the third architectural slot: the full Vaswani encoder-decoder. Its big idea is text-to-text framing: every NLP task is reformulated as “input text in, output text out”, with a task-specific prefix.

"translate English to German: That is good."   ->  "Das ist gut."
"cola sentence: He are at home."                ->  "not acceptable"
"summarize: <long article>"                     ->  "<short summary>"
"squad question: ... context: ..."              ->  "<answer span>"

A single model, a single training objective, and a uniform interface. No task-specific heads.

Pretraining: span corruption

T5 does not use MLM. Instead it uses span corruption: select random spans of consecutive tokens, replace each span with a sentinel token (<X>, <Y>, …), and train the decoder to produce <X> <span_x_tokens> <Y> <span_y_tokens> ... as the target. Roughly: the encoder sees the corrupted input, the decoder produces the missing pieces in order.

input  : Thank you <X> me to your party <Y> week.
output : <X> for inviting <Y> last <Z>

Span corruption sits between BERT MLM (mask single tokens, predict in place) and BART denoising (corrupt the input in many ways, reconstruct the original). It naturally produces variable-length outputs, which matches the encoder-decoder shape.

Architecture details

T5 keeps a few choices that distinguish it from BERT and GPT:

• Relative positional bias. No absolute embeddings; instead, the attention score gets a learned bias indexed by a bucketed relative offset. This is a precursor to RoPE and ALiBi [1].
• Pre-norm with RMSNorm. Pre-norm placement (LayerNorm inside the residual), with a simplified RMSNorm variant that skips the mean-centering. RMSNorm is now the modern default in LLMs.
• No bias terms in attention or FFN linears. A small parameter saving that compounds at scale and slightly improves training stability.
• Encoder-decoder split. The encoder is bidirectional over the input; the decoder is causal over the output and has cross-attention to the encoder output.

T5 is trained at sizes from T5-Small (60M) up to T5-XXL (11B). The 11B variant is the model that fine-tunes well on most NLP benchmarks and is still a credible choice in 2024 for translation, summarization, and structured-output tasks.

BART: denoising sequence-to-sequence

BART [9] is the same encoder-decoder architecture trained on a richer denoising objective. The input is corrupted by an arbitrary noising function (token masking, token deletion, sentence permutation, document rotation, text infilling), and the decoder produces the original document. Token infilling alone is comparable to T5 span corruption; the full set of noising operations gives BART slightly stronger performance on summarization and translation than T5 at matched scale.

The mental model: BART is to T5 what RoBERTa is to BERT. Same architecture, more careful pretraining, slightly better numbers.

The four-way comparison

How to pick the right model

The selection rule is small enough to memorize:

• Classification, span extraction, retrieval (does the embedding live in a vector database?). Use an encoder-only model. BERT-base or RoBERTa-base for cheap; DeBERTa-v3 if you can pay; specialized retrieval models like E5 or BGE for embedding tasks.
• Open-ended generation, chat, instruction following, code, agents. Use a decoder-only model. The 7B-13B class (Llama-3-8B, Mistral-7B, Qwen2-7B) covers most application work; 70B+ for hard reasoning; closed APIs (GPT-4, Claude) for state of the art.
• Translation, summarization, audio-to-text, OCR-to-text, any task where input and output are different shapes. Use an encoder-decoder. T5-XXL or BART for text; Whisper for audio; specialized T5 derivatives (mT5 for multilingual, ByT5 for byte-level) where they apply.
• Structured extraction with hard format constraints. Decoder-only with constrained decoding (JSON schema, grammar-guided sampling) usually beats fine-tuned encoder heads now, but encoder + CRF or T5 + structured decoder are still competitive for high-throughput batch jobs.

A from-the-API view

Three idiomatic snippets, one per archetype, using HuggingFace transformers:

# Encoder-only: classify sentiment with a fine-tuned BERT.
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

ckpt = "distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(ckpt)
model = AutoModelForSequenceClassification.from_pretrained(ckpt)
model.eval()

texts = ["The food was incredible.", "The service was terrible."]
inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=True)
with torch.no_grad():
  logits = model(**inputs).logits

probs = torch.softmax(logits, dim=-1)
labels = ["NEGATIVE", "POSITIVE"]
for text, p in zip(texts, probs):
  pred = int(p.argmax())
  print(f"{labels[pred]:>9s}  ({p[pred]:.2%})  {text}")

Pitfalls

Pop quiz

Interview Q&A