The Token Is the Unit of Account for AI

01 Every Bill Starts As A Token Count

When an inference invoice arrives, nothing on it is measured in words, sentences, or requests. Everything is denominated in tokens: the price line reads per million tokens, the context window is quoted as a token maximum, and the rate limit that throttles a launch is counted in tokens per minute. The token is the unit of account for AI services in the same sense the kilowatt-hour is the unit of account for electricity, and every commercial property of a language model feature traces back to how many of them a workload consumes.

That makes token literacy an economics skill rather than a curiosity. A developer who can estimate the token count of a prompt can predict a feature's margin before shipping it, can explain why a launch hit its rate limit at half the expected traffic, and can recognize when a model failure is really a tokenization failure. This article is the unit-accounting view of tokenization: how byte pair encoding manufactures the unit, how price lists and rate limits read in it, why context windows are rented space measured in it, and where the accounting itself breaks the model.

02 Byte Pair Encoding In One Paragraph

Modern models do not read characters or words; they read subword units produced by byte pair encoding. The algorithm is compact: start from a base alphabet of bytes or characters, count which adjacent pairs occur most often in a training corpus, merge the most frequent pair into a new unit, and repeat tens of thousands of times. The result is a vocabulary of subwords in which common words survive as single tokens, rare words shatter into pieces, and everything a model will ever see is a sequence of vocabulary indexes.

The consequences are everywhere once you look. A word like strawberry can split into several tokens while a longer but common word stays whole, which is why letter-counting questions trip models: the model sees chunk indexes, not letters. Capitalized and lowercase variants can be different tokens with different counts. Whitespace usually rides along with the following word. The same sentence also tokenizes differently under different model families, because each provider trains its own vocabulary. The chart below contrasts a plain word-level split with an approximate byte pair encoding of two everyday sentences.

03 Reading The Price List Per Million

Price lists are quoted per million tokens, and they are not flat. Input tokens, the prompt a client sends, are priced well below output tokens, the text the model generates, often by a factor of several times, because generation pays for a forward pass per produced token while prompt tokens are processed in parallel. Providers also sell prompt-caching discounts: repeated prefix content, a system prompt or a retrieved document set, can be billed at a fraction of the standard input rate when it matches a recently served cache entry. The result is that two workloads with identical text volume can land an order of magnitude apart in cost.

This article deliberately prints no per-model price figures. Vendor pricing pages are dynamic application pages that resisted automated verification during research for this piece, and list prices change frequently enough that a snapshot in prose ages badly; readers should price their own workload against the vendor pages in the references on the day they ship. The structural math, however, is stable. Take sentence A from the previous section, about nine tokens, multiply by the applicable per-million input rate, add the generated output at its higher rate, and multiply by traffic: that is the whole costing exercise, and the character count nobody bills for is the trap on the left of the chart below.

04 Context Windows Are Rented Space

A context window is quoted in tokens because tokens are what the architecture actually processes: a model advertised at some large token maximum accepts prompt plus output up to that count, and every added token widens the attention computation that lets distant words see each other. Attention cost grows with sequence length, and the key-value cache that makes generation efficient occupies accelerator memory in proportion to the tokens in play. Long input is not just a bigger number on an invoice; it is a real-time claim on scarce hardware for the life of the request.

That is why long-context capacity is priced and packaged differently, with higher per-token rates or separate tiers at extended windows, and why stuffing everything into the prompt is rarely the free move it appears to be. Quality degrades too: retrieval studies repeatedly find models answering best when relevant material sits near the beginning or end of a long context rather than buried in the middle. Teams that plan token budgets, deciding what deserves to occupy the window and what belongs in a retrieval call, get both the bill and the answers they expected.

05 Rate Limits Are Denominated In Tokens

The second ceiling on any launch is the rate limit, and providers state it in tokens per minute alongside requests per minute. The design is honest accounting: serving cost scales with tokens processed, not with requests, so a hundred short pings and one giant summarization job that happen to share an endpoint do not cost the provider the same thing. A customer can stay under a request-per-minute ceiling and still get throttled the moment long prompts push tokens-per-minute over the line, which surprises teams that only modeled call frequency.

The operational playbook follows directly. Estimate tokens per request, multiply by expected concurrency, and leave headroom before launch day. Move bulky, latency-tolerant jobs to batch endpoints, which providers discount because they queue work off the interactive path. Keep prompts lean, because every cached or trimmed token is both cheaper and less likely to collide with the ceiling. And monitor actual consumption from response metadata rather than assuming, since token counts vary with input content in ways that word counts never reveal.

06 When Tokenization Breaks Things

Some model failures are not reasoning failures at all; they are accounting artifacts. Ask a model how many letters are in strawberry and it may answer wrong, because the model never saw seven letters, it saw several subword tokens that encode the word's sound and shape. The same mechanism explains trouble counting characters in a string, reversing long words letter by letter, and performing certain arithmetic, where multi-digit numbers may split into tokens that sever place values. The model is reasoning faithfully over the units it was actually given.

Once diagnosed, the fixes are procedural. Spell the letters out explicitly, insert spaces or punctuation between characters, or hand the task to a tool such as a code interpreter that counts characters mechanically. Structured formats have their own token economics: verbose markup spends output tokens at the higher output rate, so compact formats save real money at scale. The general lesson is that tokenization is a layer with its own physics, and features that respect it are more reliable than features that assume the model sees text the way humans do.

07 Measure Before You Ship

Token accounting rewards teams that measure. Before a feature ships, tokenize representative prompts with the target model's tokenizer, since counts differ across model families and a word-count heuristic drifts badly on technical text. Log tokens in and tokens out per request from response metadata, attribute them per feature, and price each feature at the applicable input, output, and cached rates. Budget per feature the way infrastructure teams budget per service, with a token cost per successful user action as the headline number. Every one of these steps is cheap; skipping them is how margin quietly disappears at scale.

The honest caveats: list prices and tokenizer vocabularies both change, so a costing spreadsheet is a snapshot, not an asset; caching discounts depend on traffic patterns that only measurement can confirm; and long-context quality effects resist purely economic modeling. The outlook is that this accounting gets easier to automate, with usage metadata, batch pricing, and caching primitives becoming standard parts of the platform surface. The token will remain the unit of account either way. The teams that read invoices in tokens, rather than translating them into words after the fact, will be the ones whose AI features survive contact with production economics.