The GPU bet: why parallel silicon became the engine of AI

01A graphics chip walks into a physics department

The GPU was never designed to think. It was designed to draw: to transform millions of vertices and shade millions of pixels many times per second, in whatever video game the player had just loaded. That narrow job description - apply the same small arithmetic operations to enormous batches of data, as fast as possible - turned out to describe a great deal of scientific computing as well. Through the late 1990s and 2000s, researchers in physics, chemistry and finance quietly hijacked gaming hardware for their own linear algebra, a practice that came to be called general-purpose computing on GPUs, or GPGPU.

The decisive demonstration arrived in 2012. A team at the University of Toronto entered the ImageNet competition with a deep convolutional network, later known as AlexNet, and won it by a margin that stunned the field. What made the result possible was not a new law of physics but a hardware choice: the network was trained on two consumer GTX 580 graphics cards, hardware any enthusiast could buy off the shelf. The lesson spread quickly. If a gaming chip could carry a research breakthrough of that magnitude, the gaming chip was now AI infrastructure. Within a few years, the accelerator stopped being a side bet for one company and became the main event for the entire industry.

02Why neural networks love parallelism

Training a neural network is, at bottom, an exercise in matrix multiplication. Each layer of the network holds a large grid of weights; each training step multiplies those grids by big vectors of activations and gradients, then adjusts every weight slightly and repeats. There are no complicated decisions inside the inner loop - no branches, no irregular jumps - just billions of multiply-and-add operations applied uniformly across huge batches of data. Computer scientists call such a workload embarrassingly parallel, and it is almost a literal description of what a GPU is built for.

A modern CPU is the opposite tool: a handful of very powerful cores, each optimized to race through serial, branch-heavy logic as quickly as possible. A GPU flips the trade. It sacrifices per-core sophistication and packs on thousands of small, simple cores that execute the same instruction stream over different slices of data. A single serial task runs slowly on that arrangement; ten thousand identical tasks run beautifully. Deep learning is essentially nothing but ten thousand identical tasks, which is why the parallelism gap shown below is not a marketing claim but an architectural mismatch.

The chart above is illustrative rather than a specific benchmark - exact core counts vary by product and vendors count cores differently - but the orders of magnitude are the story. A 2026-class server CPU has on the order of a hundred cores. A data-center GPU has on the order of ten thousand. The chip did not win because it is fast in general; it won because the workload is uniform, arithmetic-heavy and gigantic - a description that fits graphics and gradient descent equally well.

03CUDA: the moat nobody copied in time

Raw silicon explains only half the GPU story. The other half is software. In 2006, NVIDIA introduced CUDA, a programming model and toolchain that let ordinary developers write general-purpose code for the GPU in a C-like language, without the graphics-API contortions that earlier GPGPU workarounds required. For the first time, a physics student could write a matrix kernel for the same chip that rendered last night's game session, and a deep learning researcher could port an algorithm without a driver programmer in the room.

That decision compounded quietly for a decade. CUDA accumulated libraries, textbooks, course materials, debuggers and an enormous population of trained programmers. When deep learning exploded, nearly every tutorial, framework and research codebase assumed CUDA, and tuned libraries made popular network operations run close to the hardware's peak. Competing silicon could match the transistors, but replicating years of accumulated software and know-how is a much slower project than taping out a chip. AMD's open ROCm stack and the software layers around Google's TPUs have closed some of the distance, but CUDA remains the reason the GPU bet paid off for one vendor in particular, and not merely for parallel hardware in general.

04From gaming cards to data-center accelerators

The hardware did not stand still. The enthusiast card that trained AlexNet evolved into a family of dedicated data-center accelerators: rack-scale parts engineered not for pixels but for training. Along the way, the bottleneck moved. Raw arithmetic throughput - FLOPS - stopped being the whole story, because a training job is only as fast as its slowest data movement. Modern accelerators stack memory directly on the package using high-bandwidth memory (HBM), and NVLink interconnects tie thousands of GPUs into a single training fabric, since one accelerator has not been enough for a frontier model in years.

The scale of the runs changed accordingly. Training runs now use thousands of GPUs for months at a time, in dedicated facilities whose design begins with power delivery rather than the machine hall. The compute consumed by notable training runs has grown by many orders of magnitude since 2012; the chart below shows published estimates on a logarithmic scale, where each unit on the axis represents a factor of ten.

The values in that chart are order-of-magnitude estimates drawn from published trend analyses rather than audited figures - companies rarely disclose exact training compute - but the direction is not in dispute. The engine did not merely get used more; it got reshaped around the workload, and the data center itself got reshaped around the engine.

05The 2026 accelerator landscape

By 2026 the accelerator market is a race with several serious entrants. NVIDIA ships its Blackwell generation and has announced the Rubin line that follows it; Google continues to refine its TPU family, deployed at scale inside its own infrastructure; AMD pushes its MI-series parts; and several large operators design fully custom silicon for their own workloads. The competition is genuine, but the incumbent's advantage in software and installed tooling keeps the market from behaving like a commodity.

Three forces now shape the landscape more than any single chip. Power: a training cluster is limited by the electricity it can draw, and accelerators are judged partly on performance per watt. Memory: the capacity and bandwidth of HBM determine how large a model fits and how fast the compute is fed. Interconnect: whether thousands of accelerators behave like one machine depends on the network between them. FLOPS headlines are the easiest number to publish and often the least decisive one.

06What GPUs cannot do (and what tries instead)

The GPU is not a universal answer. Serial, branchy, latency-sensitive workloads - running a database, serving a web page, executing business logic - still belong to CPUs, and the most parallel chip in the world is wasted on code that cannot be expressed as bulk arithmetic. Inference at modest scale is also awkward: hardware built to chew through months of training can be an expensive way to answer one user's question, which is why a growing family of purpose-built inference chips and compact serving configurations exists alongside the big training parts.

It is also worth being precise about what GPUs do not cover at all. Training produces a model; it does not deliver that model to users. Once training ends, the model enters a deployment world of serving infrastructure, data connections and tool integration - the layer where protocols such as the Model Context Protocol standardize how applications reach databases and APIs. That deployment layer is exactly what the embedded video for this article covers, rather than the silicon underneath it.

The GPU bet, in the end, was a wager that the workload of the future would look like the workload of a video game: huge batches, simple arithmetic, and tolerance for latency in exchange for throughput. It is rare in computing history for a metaphor to survive contact with reality so completely - and rarer still for the winner to owe as much to a software moat as to the chip itself.