How Large Language Models Actually Work

01 The Prediction Engine

A large language model is, at its core, a system that estimates what token should come next in a sequence. Give it a partial sentence and it assigns probabilities to many possible continuations. Repeating that operation, with a sampling rule that balances confidence and variety, produces a paragraph, a program, or a dialogue. The result can look like reasoning because the training data contains examples of reasoning, not because the model has a human mind behind the text.

The word large refers both to the scale of the training corpus and to the number of adjustable values, called parameters. During training, the model changes those values so its predictions better match text that already exists. It does not store a simple dictionary of answers. Instead, it compresses statistical relationships among words, phrases, facts, styles, and concepts into a high-dimensional function.

02 Tokens Are The Working Units

Before a model can predict, text must become a sequence of token IDs. A token may be a whole short word, a word fragment, punctuation, or a space-marked fragment. Subword tokenization lets one vocabulary handle common words efficiently while still representing names, new compounds, and misspellings. The model therefore does not see a sentence as a stream of meanings; it sees a carefully encoded sequence of discrete symbols.

Token boundaries affect cost and behavior. The same idea can require different numbers of tokens in different languages, scripts, or formatting styles. Code, tables, and long strings often split into many pieces. Each token gets a position in the context window, and that finite window is why a model cannot attend to unlimited history in one pass.

03 Embeddings Turn IDs Into Geometry

A token ID by itself has no useful numerical relationship to another ID. An embedding table maps each ID to a vector: a list of learned numbers that places the token in a space where useful patterns can emerge. Similar contexts can move words and fragments toward similar directions, while grammatical roles, topics, and associations can be represented by combinations of directions.

Position information is added because attention alone does not inherently know whether a token came first or last. Modern systems use learned or mathematical positional signals, then repeatedly transform the resulting vectors. It is more accurate to picture a changing field of representations than a fixed lookup table: the vector for a word after several layers depends on the surrounding sequence.

04 Attention Connects The Context

Attention gives each token a way to weigh other tokens in the current context. A layer forms queries, keys, and values from the representation. A query asks what information this position needs; keys describe what other positions offer; values carry the information that can be mixed in. Similarity between a query and a key becomes a score, and a normalized set of scores determines the weighted blend.

In a causal language model, a mask prevents a position from looking at future tokens. That restriction makes next-token training honest: when predicting the next piece, the network can use the prefix but not the answer. Multiple attention heads can specialize in different relationships, such as nearby syntax, repeated names, or distant references, although the learned behavior is distributed rather than a set of neat human-labeled circuits.

05 The Transformer Repeats Useful Moves

The transformer architecture stacks many blocks that alternate attention with a feed-forward network. Residual connections carry an earlier representation forward, while normalization keeps the numerical scale manageable. The feed-forward part expands and recombines features independently at each position after attention has mixed information across positions. Repetition lets simple operations build progressively richer representations.

At the end, a projection maps the final vector to one score for every token in the vocabulary. A softmax converts those scores into a probability distribution. The highest-probability token is not always selected: temperature, top-k, top-p, repetition controls, and system instructions shape the decoding policy. Generation is thus a loop of neural computation plus a choice about how to sample the next symbol.

06 Training Is A Two-Stage Story

Pretraining exposes a model to enormous collections of text and asks it to predict withheld next tokens. Each error produces a loss value. Backpropagation calculates how the error should change the parameters, and an optimizer applies small updates across many batches. The training objective is local, but learning across varied examples can produce broad capabilities: syntax, translation patterns, factual associations, and procedures.

Fine-tuning changes the model after pretraining. Curated demonstrations can teach a preferred response format, while preference optimization can make outputs more helpful, harmless, or aligned with a product policy. Retrieval, tools, and a system prompt are additional controls around the model; they are not the same as adding knowledge to its parameters. A model can also be quantized or distilled to reduce memory and latency at inference time.

07 Inference Is Powerful But Not Ground Truth

During inference, the model receives a context, runs its layers, selects a token, appends that token, and repeats. The first pass is often called the prefill phase; later steps reuse cached key and value tensors so the whole prompt does not need to be recomputed every time. Latency, memory bandwidth, context length, and batching all affect the practical cost of an answer.

The model's fluency can hide uncertainty. It may confidently complete a pattern with an invented citation, confuse correlated phrases with evidence, or fail on a small change in wording. Its knowledge is bounded by training and any supplied context, and its outputs reflect data gaps and biases. Verification, source retrieval, testing, and human judgment remain necessary when accuracy matters.