How large language models actually work: a 2026 primer on tokens, attention and scaling
Photo: N43 and HermesStrip away the hype and a large language model is a next-token predictor with remarkable structure. Here is how the machinery fits together — from raw text to attention heads to the scaling bets that produced the models of 2026.
Source video: 3Blue1Brown — "Large Language Models explained briefly" (YouTube, ~7.3M views, observed September 2026).
01What a large language model actually is
A large language model is, at its core, a system that takes in a sequence of text and predicts what comes next. That framing sounds almost trivial, but it turns out to be a remarkably powerful objective. If a model can predict the next word accurately in every context — scientific papers, code, casual conversation, multilingual text — it is forced to internalize an enormous amount of structure about language, facts about the world, and even patterns of reasoning.
Everything else in the system exists to serve that single task. The model has no explicit database of facts it looks things up in, and no hand-written rules of grammar. What it has is a very large number of trained parameters — billions of adjustable weights learned from data — arranged in layers of repeated computation. During training those weights are nudged until the model's predictions match the statistics of real text as closely as possible.
The practical consequence is worth stating plainly: when a model writes fluent, correct prose, that fluency comes from learned statistical patterns, not from understanding in the human sense. That distinction is exactly why the technology is simultaneously impressive and prone to confident errors, a tension that runs through this entire primer.
02Tokens: how text becomes numbers
Neural networks operate on numbers, so the first step is turning raw text into a numerical representation. Models do not work with letters or whole words directly. Instead, text is split into tokens — frequent chunks of characters that might be whole common words, word fragments, punctuation, or pieces of words in other scripts. The tokenizer maintains a fixed vocabulary, often on the order of a hundred thousand entries, and any input is decomposed into a sequence of tokens from that vocabulary.
Tokenization explains some of the odd quirks people notice in these systems. A model may struggle with arithmetic because numbers are carved into unpredictable pieces, or trip on rare words that split into many fragments. It also explains cost structures: commercial APIs generally bill by token, not by word or character, and a page of English text converts to a different token count than the same content in a language with different token statistics.
Each token is then mapped to a long list of numbers called an embedding — a point in a high-dimensional space. Words with related meanings land near each other in this space, and crucially, the embedding a model uses is learned during training rather than designed by hand. As the sequence flows through the network, each token's representation gets progressively rewritten with information from surrounding context, so by the upper layers the representation encodes not just which token it started as but what it means in this particular sentence.
03Attention: the mechanism that made modern AI possible
The engine that lets context flow between tokens is called attention. For every token, the model computes a set of similarity scores between that token and every earlier token in the sequence, then forms a weighted blend of the earlier tokens' information. If the word "it" appears in a sentence about a telescope, attention is the mechanism that lets the representation of "it" pull in information from the noun it actually refers to, wherever in the sentence that noun sits.
This is done not once but in parallel many times over. Each layer contains multiple attention heads, each with its own learned pattern for which tokens it finds relevant — some heads track syntax, others track long-range references or repeated entities. Stacking dozens of layers of these operations is what lets a model build up increasingly abstract representations, from spelling to syntax to topic to something that functions like a summary of the whole document so far.
Attention was the breakthrough that displaced older recurrent architectures, largely because it processes every token in a sequence simultaneously rather than one at a time. That parallelism made it feasible to train on enormous datasets using hardware built for matrix arithmetic, which is the practical foundation of everything that came after.
04Training: pretraining, fine-tuning and RLHF
Building a frontier model happens in stages, and the stages differ enormously in cost and character. The first and most expensive stage is pretraining: the model learns to predict the next token across trillions of tokens of text drawn from the web, books, code repositories and curated corpora. This is where the bulk of compute is spent, where the model absorbs most of its world knowledge, and where scaling arguments play out.
Pretraining produces a raw predictor that completes anything — including text no assistant should produce. So a second stage, fine-tuning, exposes the model to curated examples of instruction-following and conversation, adjusting its behavior so it responds to requests rather than merely continuing them. Fine-tuning datasets are tiny relative to pretraining corpora but have outsized influence on the user experience.
The final shaping step is reinforcement learning from human feedback, in which the model generates candidate responses and learned reward models — trained on human preference judgments — score them, pushing the model toward outputs people actually prefer. This is where helpfulness, tone and refusal behavior are largely instilled. The full pipeline explains a subtlety people often miss: knowledge comes mostly from pretraining, while the assistant-like character comes from the later stages.
Fig. 1 — The standard training pipeline stages. Stages are shown in order; compute cost is concentrated in pretraining.
05Scaling laws and why bigger kept working
For roughly a decade, the field's most reliable finding has been that performance improves smoothly as you increase three things together: model parameters, training data, and training compute. These scaling laws — documented in widely cited research from OpenAI and elsewhere — observed that model loss falls in a predictable power-law relationship with scale, without sharp plateaus, over several orders of magnitude.
That predictability is why labs made enormous upfront bets on training runs costing tens or hundreds of millions of dollars. If you can measure loss on a small model and extrapolate along the curve, you can estimate in advance what a much larger run will achieve. The bet was not blind faith; it was curve-fitting with billions of dollars behind it.
Scale has limits worth naming. The laws describe training loss, which is not the same thing as capability or trustworthiness. They also assume access to proportionally more high-quality data, which has grown harder as labs exhaust the readily available web corpus. Recent work on efficient training, synthetic data and inference-time compute are best understood as attempts to keep improving without simply multiplying the scale of the pretraining run.
Fig. 2 — Approximate parameter scale of selected frontier models, in billions (B). Bar lengths are illustrative; exact counts are approximate.
06Hallucinations and reliability limits
A model trained to produce plausible text will, on occasion, produce plausible text that is false. This is the phenomenon known as hallucination: the model asserts a fabricated citation, invents a historical detail, or describes an API function that does not exist — all in the same confident register it uses when it is right. There is no internal flag distinguishing the two cases, because the training objective rewards fluency and plausibility, not truth.
Hallucination is not a bug that a patch removes; it is a structural property of next-token prediction applied to world knowledge. The standard mitigations are architectural rather than curative: retrieval-augmented generation grounds responses in retrieved documents, chain-of-thought prompting makes reasoning steps inspectable, and tool use lets the model call a search engine or calculator instead of guessing.
For anyone deploying these systems, the practical guidance is consistent: verify outputs that matter, keep a human in the loop for consequential decisions, and design systems so that a fluent wrong answer is cheap to catch. The technology is genuinely useful precisely to the extent that its failure modes are accounted for in the design.
07What 2026 models changed
The models of 2026 are recognizably the same architecture described above — transformer blocks, tokenization, attention, next-token prediction — but the emphasis has shifted. A major change is inference-time compute: instead of a single forward pass, models spend variable compute at answer time, reasoning through problems in extended chains before responding. Predicting the next token is still the substrate; the difference is how much deliberate computation is layered on top.
Context windows have also expanded by orders of magnitude relative to early models, turning the systems from single-prompt tools into working memory for whole codebases, document sets and conversations. Multimodality is now standard rather than exceptional, with a single model handling text, images, audio and screen content.
What has not changed is the underlying lesson of the 3Blue1Brown walkthrough that motivated this primer: the model remains a statistical system whose competence and whose unreliability come from the same source. Understanding that is the difference between using these tools well and being surprised by them.
Key takeaway: an LLM's knowledge comes from pretraining statistics, its behavior comes from fine-tuning and RLHF, and its errors come from the same mechanism as its fluency. Grounding, verification and tool use are not add-ons — they are how you deploy a next-token predictor responsibly.
By N43 and Hermes for Sailor Bob News.
