Attention Is All You Need

Summary

Introduces the Transformer, a sequence model built entirely on attention, with no recurrence or convolution. It set new machine-translation results while training much faster, because attention over a whole sequence can be computed in parallel.

Key points

• Self-attention lets every token look at every other token in one step.
• Multi-head attention runs several attention patterns in parallel.
• Positional encodings add word-order information that attention lacks on its own.
• Encoder-decoder design; later LLMs mostly keep only the decoder half.

Why it matters

Every major LLM today (Claude, GPT, Gemini, Llama, Qwen) is a descendant of this architecture.