Google's Custom AI Silicon Is Quietly Rewriting the Economics of Compute

01 Why Custom Silicon Decides AI Economics

The price of artificial intelligence is, at bottom, the price of moving numbers through matrix multipliers. Training a frontier model and serving a billion queries are both dominated by one operation, and the hardware that performs it most efficiently sets the floor on what every AI product can charge. That is why the industry's most consequential competition in 2026 is not happening in chat interfaces but in datacenter silicon.

Google has been running its own matrix-multiply hardware, the Tensor Processing Unit, since 2015, and by most public accounts the majority of its inference workloads now run on TPUs rather than merchant GPUs. What changed this year is who else can use them: expanded external access to TPU capacity turns a proprietary efficiency advantage into a product, and a product into a pricing pressure on the rest of the market. Numbers in this article about TPU performance and cost should be read as order-of-magnitude estimates drawn from public reporting and benchmark literature rather than audited disclosures.

02 What a TPU Actually Does Differently

A GPU is a general-purpose parallel processor that grew up drawing pixels and later learned to train neural networks. A TPU was designed from a blank sheet to do exactly one thing: multiply large matrices of low-precision numbers, quickly, at low power. Its heart is the systolic array, a grid of arithmetic units where data pulses through neighboring cells so that a single fetched operand is reused dozens of times instead of being fetched from memory again and again.

That design trades flexibility for efficiency. A TPU cannot rasterize game graphics and is a poor fit for workloads outside deep learning, but for neural-network math it eliminates most of the control logic, caches, and precision overhead that general chips carry. The measured result reported in Google's 2017 ISCA paper was roughly an order of magnitude better performance-per-watt than contemporary CPUs and GPUs of that era on inference workloads. It is an old measurement, but the architectural logic behind it still holds.

03 The New Economics of Serving Tokens

Training gets the headlines, but serving pays the bills. Once a model ships, every user prompt becomes a recurring cost, and inference now represents the majority of AI compute demand by most industry estimates. The unit that matters is dollars per million tokens served, and that number has been falling relentlessly, driven by better silicon, better utilization, quantization, and brutal cloud competition.

Custom silicon is one of the few levers that attacks the cost curve at its base rather than its edges. Because a TPU owner controls the whole stack — chip, interconnect, and the compiler that maps models onto the array — it can retire inefficiencies a heterogeneous cloud cannot. The chart below shows the shape of the decline; treat the specific values as illustrative of a well-documented trend, not as figures from any single operator's books.

04 Who Else Is Building AI Silicon

Google is no longer alone. Amazon's Trainium chips power a growing share of AWS AI workloads, Microsoft has deployed its Maia accelerators, and Meta designs its own MTIA inference hardware. OpenAI has been reported for over a year to be working with Broadcom on custom accelerators, and every major frontier lab now treats in-house silicon as a strategic option rather than a curiosity.

The pattern echoes an earlier era of the cloud: once a workload becomes predictable at scale, the largest buyers stop renting commodity hardware and start specifying their own. Merchant chips remain essential — Nvidia's GPUs still train most frontier models — but long-term infrastructure contracts increasingly favor providers who can quote a lower cost per token because they own the multiplier.

05 What It Means for GPU Pricing

For a decade the AI hardware market has been a near-monopoly with a single price-setter. Nvidia's share of AI accelerators remains dominant by most analyst estimates, and its gross margins — among the highest in the semiconductor industry — are partly a symptom of that position. Alternatives that clear the performance-per-dollar bar convert directly into negotiating leverage, even for buyers who never leave the incumbent.

The effect is already visible in cloud pricing behavior: reserved-capacity discounts, aggressive bundles, and public benchmark fights over cost per token. None of this means GPU prices collapse; it means the margin structure of AI infrastructure starts to look like the rest of computing, where competition, not scarcity, sets the spread.

06 Risks, Bottlenecks and Limits

Custom silicon is not a cheat code against physics. Accelerator performance at the frontier is increasingly bounded by high-bandwidth memory, not raw matrix throughput, and HBM is bought, not built — every chipmaker queues at the same small group of memory vendors. Interconnect and optics set the ceiling on how many chips can act as one machine, and compiler quality often decides real-world throughput more than silicon does.

There is a software lock-in risk in the other direction: years of CUDA code and expertise do not port themselves, and migration costs are real. Custom programs also live or die with their sponsor's commitment — a chip generation that misses its window becomes stranded capital. These constraints explain why the custom-silicon wave is advancing generation by generation, not overnight.

07 The Legacy of the Custom-Silicon Bet

The TPU was a heretical bet in 2015 — Google was told, in effect, that general-purpose GPUs had already won. A decade later, the program is a case study in how owning your own means of production reshapes an industry's economics. The durable lesson is not that TPUs beat GPUs; it is that at sufficient scale, the compute layer stops being a commodity you rent and becomes a product line you own.

If the current wave of custom accelerators holds its performance-per-dollar advantage, the next few years of AI pricing will be set in boardrooms that design chips, not just buy them. That is a quieter story than any single model release, and a bigger one.