HBM Is the New Bottleneck in AI Hardware, and Three Companies Control It

01 The Wall Behind the FLOPs

Accelerator marketing is a contest of arithmetic: trillions of operations per second, printed ever larger on spec sheets. But ask the people who run large training jobs what actually limits them, and the answer is usually not raw compute. It is the rate at which data can be fed to the compute. A processor that can multiply faster than memory can supply operands is not a faster machine; it is an expensive machine that spends more of its time waiting.

That waiting is why High Bandwidth Memory, or HBM, has moved from a specialist graphics component to the most contested part in the data center. Every leading AI accelerator now specifies its HBM capacity and bandwidth alongside its compute peak, because the two numbers only mean something together. The CNBC documentary in the references, filmed inside SK Hynix and reporting 619,746 views as of October 2026, is one visible marker of how far the memory business has been pulled into the center of the AI story.

02 What HBM Actually Is

HBM is not a new kind of memory cell. The storage itself is conventional DRAM. What is new is the geometry: instead of one or two dies sitting in flat packages wired to the processor through a circuit board, HBM stacks eight or more thin DRAM dies on top of each other and connects them vertically with through-silicon vias - holes drilled through the silicon and filled with metal, allowing signals to pass straight up the stack. The stack is mounted on the same package as the processor itself.

The interface that results is described as wide but short. Where commodity DRAM talks over a narrow, fast, long-trace channel to a chip across the board, HBM talks over hundreds of parallel connections a few millimeters long, each running at a modest speed. Short wires, many of them: enormous total bandwidth with far less energy spent per bit moved. Per Wikipedia, the interface was initially developed by Samsung, AMD, and SK Hynix; SK Hynix produced the first HBM chip in 2013, and the first devices - AMD Fiji GPUs - shipped in 2015. The same technology is now used both in accelerators and as on-package cache.

03 Why Bandwidth Beats Peak Compute

The organizing idea here is arithmetic intensity: the ratio of operations performed to bytes moved from memory. A workload with high arithmetic intensity reuses each byte for many calculations and is compute-bound; a workload that performs only a few operations per byte is memory-bound, no matter how many FLOPs the hardware offers.

Large-model training and inference sit uncomfortably close to the memory-bound end. Each layer's parameters and activations must be streamed through the processor, and attention mechanisms add further all-to-all data movement. That is why accelerator designers respond to the bottleneck with more bandwidth rather than only more compute: HBM capacity determines how much of a model fits close to the chip, and HBM bandwidth determines how fast the rest of it can be pulled through. When memory bandwidth improves slower than compute, the gap becomes waste - silicon bought but not fed. The history of AI hardware since 2020 is, in large part, the history of trying to keep that gap from widening.

04 The Speed Ladder, Generation by Generation

HBM advances in named generations set by the JEDEC standards body, and the per-stack bandwidth numbers describe the bottleneck easing in near-doubling steps. HBM2E moves roughly 460 gigabytes per second per stack. HBM3 raised that to about 819. HBM3E, the current high-volume generation, reaches approximately 1,150 to 1,280 gigabytes per second depending on configuration. HBM4, specified as the next step, targets around 2,000 gigabytes per second per stack - more than a fourfold rise over HBM2E in roughly four years.

Two things about the ladder matter for the supply story. First, each step is physically harder: more stacks, faster signaling, tighter thermal margins, and a more demanding packaging process. Second, the generations do not switch on cleanly - early production of a new generation goes disproportionately to whichever customer pays for the yield learning, which is how one supplier can leap ahead of another for a full product cycle.

05 Three Companies and a Hard Moat

The reason the supply base stays narrow is that HBM manufacturing combines two disciplines that are each hard on their own. It requires leading-edge DRAM fabs - capital costing billions per facility - and it adds stacking and through-silicon-via processing, where a defect in any die of a stack can scrap the whole assembly. Yield discipline, not invention, is the barrier: a DRAM maker can know how to build HBM and still lose money on every stack until yields mature. That is also why the moat persists; the learning lives in production lines, not in patents alone.

Analyst and industry reporting through 2025 put the approximate supplier split at SK Hynix somewhat above half of the HBM market, Samsung at roughly 30-40%, and Micron at roughly 10-20% - approximate figures, not audited shares. The same reporting shows HBM climbing from roughly 8-10% of total DRAM industry revenue in 2023 to around 20% in 2024, with 30%-plus projected for 2025. A memory product that was a niche graphics part a decade ago, per the Wikipedia history, now claims a large and growing slice of an entire industry's revenue.

06 Allocation, Pricing, and Roadmaps

Scarcity changes who decides. In a commodity memory market, buyers choose from a catalog; in a capacity-constrained one, suppliers choose customers, and HBM capacity for future generations is now booked years in advance through long-term agreements. For the accelerator builders, securing HBM supply has become a strategic function on par with securing leading-edge logic wafers, and shortfalls in one component idle everything downstream of it.

Pricing power is the second effect. DRAM has historically been a brutal business, with price collapse following every capacity addition. HBM interrupts that pattern: because each stack packs far more die and far more processing than a commodity module, the industry effectively sells its output at a premium, and revenue grows even when bit growth is modest. The third effect is indirect - the packaging step that attaches HBM to the processor becomes its own chokepoint, so each HBM generation complicates not just the memory but the assembly line behind it.

07 What Would Change This Picture

The numbers in this article deserve their hedges. The revenue-share figures are analyst and industry estimates, reported as approximations, not filings; the 2025 share is a projection, and projections in memory markets have missed in both directions before. The supplier split likewise shifts with each generation transition, since early HBM4 allocation could redistribute share among the three incumbents.

Structurally, several developments would loosen the bottleneck. Additive on-package cache or large fast SRAM near the processor can substitute for some DRAM traffic, and architecture changes that raise arithmetic intensity reduce bandwidth demand per unit of work. Slower AI demand growth would ease allocation pressure and, in time, erode the premium. And the three-supplier concentration is a state, not a law: if yields mature and capacity catches up, HBM could drift back toward ordinary DRAM economics. What seems durable is the design lesson - in AI hardware, the width of the pipe to memory now competes with, and often outranks, the count of the multipliers. That hierarchy, not any single quarter's market share, is the fact worth carrying forward.

Source video: Inside SK Hynix: We Went To Korea To See The World's Biggest AI Memory Buildout · CNBC · 619,746 views as of October 2026 (observed figure). Reporting summary and charts by N43 and Hermes AI.