Self-attention, explained from cocktail-party intuition to the full math. Why every attention variant since 2017 is a remix of the same three letters. If you’ve ever stared at this equation and nodded along without really understanding it: Attention(Q, K, V) = softmax(QKᵀ / √dₖ) V You’re in good company. It’s the most consequential equation in modern AI. It powers GPT-4, Claude, Llama, Gemini, and every frontier model you’ve interacted with. It’s in every deep learning course, every transformer paper, every ML engineer’s interview prep. And yet — most people who work with LLMs every day can’t actually explain what Q, K, and V do. They know the shape of the math without the soul of it. This post fixes that. We’re going from cocktail-party intuition to the full equation, step by step, with no hand-waving. (This is a companion piece to my Inside the Transformer post. If you haven’t read that one yet, it walks through the full architecture end-to-end. This one zooms into the single most important mechanism inside it.) Why self-attention matters Before 2017, if you wanted an AI to understand a sentence, you fed it one word at a time. RNNs, LSTMs, GRUs — they all worked this way. Sequential. Slow. Forgetful. Then “Attention Is All You Need” came out in June 2017. One paper. One mechanism. It replaced every prior approach and became the backbone of GPT-2, GPT-3, GPT-4, Claude, Llama, Gemini, and every frontier model since. The mechanism — self-attention — answers a deceptively simple question: When I’m trying to understand this word, which other words in the sentence should I pay attention to? And the answer turns out to be a matrix multiplication. Actually, three matrix multiplications. That’s the whole trick. The intuition, before any math Imagine you’re at a cocktail party with 20 people. Someone says: “The animal didn’t cross the street because it was too tired.” Your brain does something remarkable. To understand “it” , you instantly figure out that “it” refers to “the animal” — not “the street.” You don’t consciously compute this; it just happens. How? You compare “it” against every other word in the sentence. Some words are relevant ( “animal” , “tired” ). Some aren’t ( “the” , “because” ). You weight them accordingly and combine the relevant ones into your understanding of “it” . That’s self-attention. Every token in a sentence looks at every other token, decides which ones matter, and blends them into its own representation. Repeat this for all tokens simultaneously, and you’ve built a mechanism that captures context in one parallel operation. The genius is in how the model decides what’s relevant. That’s where Q, K, and V come in. Meet the three matrices For every input token, the Transformer creates three different vectors by multiplying the token’s embedding by three learned weight matrices. Each vector has a specific job. Query (Q) — “What am I looking for?” The Query vector represents what this token wants to know about the rest of the sentence. If the current token is “it” , its Query might encode: “I’m a pronoun. I need to find my referent. Show me nouns that could be antecedents.” The model doesn’t actually think in English — the Query is a vector of numbers — but that’s the conceptual job. Key (K) — “What do I contain?” The Key vector represents what this token has to offer. It’s a description of the token’s relevant content. If a token is “animal” , its Key might encode: “I’m a concrete noun. I’m a possible antecedent for pronouns. I carry the concept of a living creature.” Value (V) — “What information do I actually contribute?” The Value vector is the token’s actual contribution to the output — the information that gets passed forward if a match is made. Think of it as: the Key is a title , the Value is the contents . A librarian compares your question (Query) to book titles (Keys), finds the matches, and hands you the books (Values). Same token, three vectors, three jobs. The famous equation, decoded Let’s go back to the equation: Attention(Q, K, V) = softmax(QKᵀ / √dₖ) V It’s four steps, and you already understand three of them. Step 1: Score every pair — QKᵀ For every Query, the model computes its match with every Key. Mathematically, that’s the dot product. If Q and K are aligned (pointing in similar directions in vector space), the dot product is large — they match. If they’re unrelated, the dot product is small or negative. The result is an attention score matrix: an N × N grid of numbers, where N is the number of tokens. Entry (i, j) tells you how much token i should attend to token j. For the sentence “The animal didn’t cross the street because it was too tired” : score[it, animal] = high ✅ score[it, street] = low ❌ score[it, tired] = medium Step 2: Scale — divide by √dₖ This step looks weird but matters. If the dimension of your Key vectors (dₖ) is large — say 512 or 1024 — the raw dot products can get enormous. When you feed huge numbers into softmax (next step), you get a distribution where one entry is basically 1.0 and everything else is basically 0. That’s bad. Gradients vanish during training, and the model can’t learn. Q, K, V: The Three Matrices That Quietly Run Every Modern LLM was originally published in GoPenAI on Medium, where people are continuing the conversation by highlighting and responding to this story.