To translate a complete sentence, a model can read the whole source before writing the answer. To predict the next token of a growing sentence, it cannot read tokens that have not been generated. These different information patterns help explain encoder and decoder designs.
Before you begin: Understand self-attention, cross-attention, and causal masks.
Separate input representation from output generation
An encoder transforms an input sequence into representations. In a typical bidirectional transformer encoder, each non-padding position can use context from both sides of the input. A task head then uses those representations for classification, token labels, or other outputs.
A decoder generates a target sequence using causal access to earlier target positions. In an encoder-decoder transformer, it also uses cross-attention to source representations. The source and target can have different lengths and token alignments.
“Encoder” does not mean understanding without limitations, and “decoder” does not mean generation without understanding. These terms describe architecture and information flow. Actual capabilities depend on training and the surrounding system.
Trace a translation example
Suppose the source is “The small house is blue.” The encoder can represent each source position using the entire sentence. The decoder begins from its start condition, generates a target token, then conditions later tokens on the source and its earlier target tokens.
During training, the target is shifted so the model predicts each next target token from the correct earlier target tokens. At generation time, its own chosen tokens become the history. A mistake can therefore affect later output. This difference is one reason a low training loss is not the whole evaluation story.
Compare the three families
| Family | Main information pattern | Typical task setup |
|---|---|---|
| Encoder-only | Bidirectional input representations | Classification, token labeling, trained embeddings |
| Decoder-only | Causal sequence representations | Continuation and instruction-based generation |
| Encoder-decoder | Full source plus causal target | Conditional generation such as translation |
These are common setups, not exclusive capabilities. A decoder can be adapted for classification, and an encoder's usefulness for sentence similarity depends on its training objective and pooling method. Choose and evaluate the complete task setup.
Read the mask as part of the model
For three target positions, the first can read one target position, the second two, and the third three under a standard causal mask. The source cross-attention may allow all valid source positions for each target query. Padding positions remain excluded.
If you accidentally use an unmasked target attention layer during next-token training, the model can exploit future target information. A very good training score can then coexist with poor generation when that information disappears. This is leakage caused by the computation graph, not only by a dataset split.
Choose a task representation
For named-entity recognition, you might need one label per input token. An encoder with a token-classification head directly matches that structure. For a free-form summary, you need an output sequence of variable length, which a generative architecture naturally supports.
Still compare real candidates. Sequence length, batch size, quality, deployment hardware, and available pretrained models can matter more than a broad architectural preference. Do not promise that one family is always faster or cheaper without a measured setup.
Diagnose the interface mismatch
You load a base encoder checkpoint and ask it to “write a five-paragraph report.” It returns hidden-state vectors. Is the model broken?
Match the requested output to the loaded component
No. A base encoder produces representations, not an instruction-following text-generation interface. You need an appropriate model head or a generative system trained for the output you want. A checkpoint's architecture, task, tokenizer, and loading class must agree.
Next, we will ask how these models distinguish order when attention itself does not provide a complete sequence-position signal.
Sources
Sequence to Sequence Learning is a historical encoder-decoder reference. BERT and T5 illustrate different transformer training and task arrangements.