The Scaling Laws: How Big Is Big Enough for AI?

01 The Power Law

The scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022) describe a power law relationship between compute, data, and model performance. Double the compute and performance improves by a predictable amount. This relationship has held remarkably well — every 10x in compute yields roughly a consistent improvement in loss. But power laws don't continue forever. At some point, the curve must plateau. The question is when.

02 The Data Bottleneck

The scaling laws have three variables: parameters, data, and compute. The Hoffmann 'Chinchilla' paper found that models have historically been undertrained — too many parameters, not enough data. The optimal ratio is about 20 tokens per parameter. But we're hitting the data wall: there are only about 15 trillion high-quality tokens on the internet. A model with 15T parameters (trained at the Chinchilla ratio) would need 300T tokens — 20x more than exists. The data bottleneck may end scaling before the compute or parameter bottleneck does.

03 Will GPT-5 Need 100x More Compute?

If scaling laws continue, GPT-5 would need roughly 10-20x more compute than GPT-4.5. At current GPU prices, that's $500M-$1B in compute alone. This is within reach for the largest companies but creates a moat: only OpenAI, Google, Meta, Amazon, and Microsoft can afford frontier-scale training runs. The implication: if scaling laws continue, AI becomes a 5-company oligopoly. If they break, the field opens up to smaller players. The future of AI competition depends on whether the scaling laws hold.