In-Context Learning: How LLMs Learn From Examples in the Prompt

01The model that changes its behavior mid-conversation

By design, a neural network is supposed to be fixed. Once training ends, its weights are frozen, and the same input should produce the same output forever. Large language models violate that intuition in a strange way: show one a few solved examples inside the prompt, and its behavior on the next question visibly shifts. It translates, formats, classifies, or writes in a style it was never explicitly programmed to handle at inference time. The examples are not stored anywhere; they exist only for the lifetime of the conversation.

Researchers call this in-context learning. It became a household concept almost by accident: the 2020 paper that introduced GPT-3, titled "Language Models are Few-Shot Learners," documented that a model trained only to predict text could carry out new tasks after seeing a small number of demonstrations in its input, with no parameter updates at all. A frontier release like OpenAI's GPT-6 Astra, reviewed in the video above, is in a direct lineage with that finding. Nearly everything users do with such models, from custom formatting to specialized reasoning, rests on this one capability.

02The mechanism: conditioning, not training

Mechanically, in-context learning is a property of the forward pass. When a prompt enters the model, each token is converted into activations that flow through dozens of transformer layers, and attention lets every position read all the positions before it. The prompt therefore acts as a conditioning signal: the examples reshape the activations, which reshape the probability distribution over the next token. Nothing about the model's parameters changes. Two users pasting different examples into the same model at the same moment are, in effect, running two different behaviors on identical hardware.

That is the crucial distinction from fine-tuning. Fine-tuning runs gradient descent on a dataset and permanently rewrites the weights; the new behavior persists after the session ends and is available to every future user. In-context learning changes activations only, so the behavior evaporates the moment the context window clears. Fine-tuning is durable but expensive, slow, and operationally heavy. In-context learning is free, instant, and completely forgetful. Most modern AI products are built on the second property: one frozen model, thousands of behaviors, selected by what you paste into the box.

03Evidence: induction heads

Why would a text predictor do this at all? The best mechanistic evidence points to attention heads that researchers at Anthropic named induction heads, described in a 2022 paper on the connection between in-context learning and these circuits. An induction head scans backward through the context for a previously seen token and boosts whatever came after it: if the sequence "A then B" appeared earlier, a later occurrence of A makes the model more likely to predict B. That single operation, repeated across many heads and layers, is a general copying machine, and it explains how a model can imitate the format, style, and label conventions of the examples in its prompt.

The evidence is more than a plausible story. Ablation studies reported in that line of work show that disabling induction heads measurably degrades in-context performance, and the heads' formation during training coincides with sharp jumps in few-shot ability as models grow. In other words, the capacity that made large models look like few-shot learners is traceable to identifiable circuits, not an unexplained emergent mystery. It is one of the cleanest causal accounts in interpretability research to date, though it almost certainly describes one important component of the behavior rather than the whole of it.

04Few-shot prompting became an engineering discipline

Once practitioners understood that examples steer behavior, example selection became its own craft. Consistency matters: demonstrations that share one template teach the model the template as much as the task. Ordering matters too, sometimes dramatically. A widely cited 2022 study by Min and colleagues found that the order of few-shot examples can swing accuracy across a huge range on the same model and data, and, more surprisingly, that randomly shuffling the labels often barely hurts performance, suggesting the examples communicate format and distribution as much as correct answers. Chain-of-thought prompting extends the same principle by showing worked reasoning steps, which elicits step-by-step behavior at test time.

These findings force a distinction that is easy to lose in marketing copy: capability versus behavior. Capability is what the frozen weights make possible; behavior is what the current context actually elicits. A model that fails a task with one prompt may succeed with better examples, which means a poor prompt can mask genuine capability, and a clever prompt can make a shallow model look deeper than it is. Rigorous evaluation therefore fixes the prompting protocol before comparing models, a lesson every serious benchmark team has absorbed the hard way.

05Why it matters: adaptation without retraining

The practical consequence of in-context learning is that adaptation stopped requiring infrastructure. Before 2020, teaching a model a new task meant assembling a dataset, running a training job on a cluster, and shipping new weights. Today, a product engineer can redefine the model's output format, domain vocabulary, or tone of voice in an afternoon, purely by editing the prompt. One frozen checkpoint serves thousands of distinct behaviors, which is what makes a single general model economical to deploy across many products.

The same property underwrites the agentic turn in software. System prompts, tool schemas, and retrieved documents are all, structurally, in-context learning: examples and instructions placed into the context window that condition a fixed model to behave like a booking agent, a code reviewer, or a research assistant. It also cuts the other way. The same sensitivity that lets a prompt improve behavior lets hostile input steer it, which is why prompt injection is now a first-tier security discipline. A capability this powerful is never only a convenience.

06The limits: fragility and the pattern-matching debate

In-context learning has real constraints. It is format-sensitive: minor changes in spacing, ordering, or template wording can move results more than the choice of examples themselves. It is shallow in documented ways: studies have shown models continuing to follow the prompt's example distribution even when it hurts accuracy on the actual question. And there is an open scientific argument about what the phenomenon even is. One influential line of work models it as implicit Bayesian inference, the model locating a latent task from the examples; a more skeptical reading holds that much of it is sophisticated pattern matching over surface statistics. Both camps have evidence, and the honest summary is that the field has not settled the question.

The other limits are operational. Nothing persists: an hour spent "teaching" a model in a long conversation is gone when the context clears, so in-context learning is the wrong tool for building durable skill. Examples consume context window and therefore cost money and attention headroom. And the effect is unreliable enough that production systems wrap it in validation, retries, and output checks rather than trusting it blindly. Where durable, guaranteed behavior matters, fine-tuning and retrieval still earn their place.

07From GPT-3 curiosity to the defining interface

The history of this idea has an arc worth noticing. In 2020, few-shot learning was a surprising observation buried at the end of a research paper. By 2023 it had become a job title, as prompt engineering turned the craft of example selection into an industry. By 2026, it has dissolved into the interface itself: frontier releases such as GPT-6 Astra are judged largely on how much behavior users can elicit through instructions and examples alone, and the model's system prompt has become the de facto configuration file for a software product. What began as a measurement anomaly in a scaling study is now the primary way humans program these systems.

That is the right lens for reading the hype. When a reviewer demonstrates that a new frontier model "learned" a task on camera, no learning happened in the training sense; the mechanism is conditioning, drawing on capacities that were trained in months ago. Understanding in-context learning is therefore not academic housekeeping. It draws the line between what a model genuinely knows, what a prompt merely elicits, and what an impressive demo quietly leaves out, and it explains both why these systems feel like magic and why they fail in ways that ordinary software never would.