How Large Language Models Actually Work: Inside the Transformer Architecture

01 Start With Tokens, Not Words

The first surprise is that an LLM does not read language as a human does. A tokenizer breaks a string into a vocabulary of reusable pieces: a whole common word may be one token, while an unusual name can become several fragments. Each piece receives an integer ID, and that ID is looked up in a learned vector table called an embedding.

Tokenization is both a compression scheme and a design constraint. It lets the model reuse patterns across related words, code, and punctuation, but it also makes some languages and spellings more expensive in context. The model ultimately operates on a sequence of numbers; meaning emerges from the relationships learned among those numbers.

02 Embeddings Put Text on a Map

An embedding turns a token ID into a point in a high-dimensional space. Nearby points often share useful statistical behavior, although “nearby” is not a dictionary definition. The vector for a token is adjusted during training so that the surrounding computation can predict real continuations from real examples.

Position must be added too. The same tokens in a different order can express a different fact, so transformer systems provide positional information through learned or mathematical encodings. After this stage, a sentence is no longer a string; it is a sequence of vectors carrying token identity and location into the network.

03 Attention Connects the Context

Attention is the transformer’s central routing mechanism. For each token, the network creates a query, a key, and a value. A query is compared with the keys of other tokens; the resulting scores determine how strongly their values are blended into the token’s updated representation.

In a sentence about a “bank” and a “river,” attention gives the model a way to weigh nearby and distant clues together. It does not consult a tiny grammar rule that says which definition is correct. It performs many learned comparisons in parallel, across multiple heads, each able to specialize in different relationships.

04 Layers Refine the Representation

One transformer block is not the whole model. Each block typically combines self-attention with a feed-forward network, residual connections, and normalization. Attention mixes information between positions; the feed-forward portion transforms each position’s features through a larger nonlinear space. Residual paths help preserve and refine useful signals as dozens or hundreds of blocks are stacked.

The result is not a neat ladder from nouns to facts. Different layers and heads can carry overlapping patterns, and the behavior changes with training. What matters operationally is that the stack repeatedly turns a token’s representation into one that makes the next prediction more likely.

05 Training Turns Prediction Into Capability

During pretraining, the model sees a context and is asked to predict a hidden or next token. The difference between its probability distribution and the observed answer becomes a loss. Backpropagation calculates how the parameters contributed to that error, and an optimizer nudges billions of weights a little at a time.

At scale, this simple objective rewards much more than memorized word pairs. To predict well, the network learns syntax, recurring facts, formats, styles, and some reusable abstractions about the world represented in its data. It is still not a database with guaranteed truth; it is a statistical machine whose fluency can outpace its verification.

06 Generation Is Controlled Uncertainty

At inference time, the prompt is tokenized and passed through the network. The final representation becomes a list of logits—one score for every possible next token. A softmax turns scores into probabilities, and a decoding strategy chooses one token. That token is appended to the context, and the loop runs again until a stop condition is reached.

Greedy decoding always picks the highest-probability option. Sampling can introduce variety; temperature reshapes how sharply probabilities are concentrated, while top-p limits the candidate mass. These controls do not add knowledge. They change the balance between predictable continuation and creative risk.

07 Why Fluent Output Still Needs a System

Because the model is optimized to continue patterns, a confident paragraph is not proof that its claims were checked. Retrieval-augmented generation can supply documents, tools can perform calculations, and structured output can constrain the form. Evaluation, citations, permissions, and human review provide the parts the raw predictor does not provide by itself.

The transformer is therefore best seen as a powerful component, not an oracle. Its architectural trick—letting every position selectively use information from the context—made modern language interfaces practical. The engineering challenge now is to surround that trick with memory boundaries, useful tools, and honest signals about uncertainty.