Inside SK hynix: the memory chips powering the AI boom

01The bottleneck moved

For most of computing history, the expensive part of a machine was the processor, and memory was a commodity that tagged along at the side. The AI buildout inverted that hierarchy. A modern AI accelerator spends as much of its bill of materials on high bandwidth memory as on the compute silicon itself, because the limiting factor for large model training and inference is how fast data can be moved into the processing cores.

SK hynix, a South Korean chipmaker that spent decades as a mid-tier DRAM supplier, now sits at the center of that inversion. Along with Samsung Electronics and Micron, it forms the Big Three of memory. Its high bandwidth memory, the stacked DRAM that feeds AI accelerators, became one of the scarcest components in the world economy.

CNBC recently sent a camera crew inside the company's Korean facilities to document the scale of the buildout, and the footage gives a rare look at the cleanrooms where AI's least glamorous component is made.

02What high bandwidth memory actually is

High bandwidth memory, or HBM, is DRAM taken vertically. Instead of laying memory chips flat next to a processor on a circuit board, HBM stacks several DRAM dies on top of one another and connects them with microscopic pillars of conductive material called through-silicon vias, then caps the stack with a logic die that manages the interface.

The result is a short, wide data path. HBM trades the long, narrow traces of traditional memory modules for thousands of connections running millimeters at most, which multiplies bandwidth while cutting power per bit moved. Wikipedia's summary notes that the interface was initially developed by Samsung, AMD and SK hynix, and it is now used alongside graphics accelerators, network devices and AI ASICs.

The tradeoffs are real. HBM is expensive to manufacture, hard to scale to the capacities of ordinary DRAM, and its yields punish any imperfect die in a stack, because one bad layer can scrap the whole tower. That is precisely why few companies can make it at scale.

03Why AI made this matter

A large language model is, mechanically, an enormous sequence of matrix multiplications, and the weights of the model must be streamed through the arithmetic units over and over. If memory cannot keep up, the compute silicon stalls, and an expensive accelerator becomes a very expensive space heater.

The industry measures this as the memory wall: as models grew into hundreds of billions of parameters, the bandwidth required to keep them fed grew faster than conventional memory could deliver. HBM answered by widening the pipe, and each generation roughly doubled the bandwidth of the last.

The consequence is that whoever can ship the most HBM effectively decides how many AI accelerators can be built, since an AI chip without its memory stack is not a product. Memory makers, long the most cyclical and least glamorous corner of semiconductors, suddenly held schedule-setting power over the largest infrastructure projects on Earth.

04Inside the cleanroom

CNBC's tour shows the two stages that define memory manufacturing: wafer fabrication, where the DRAM dies themselves are patterned at nanometer scale, and assembly and test, where the finished dies are stacked, bonded and paired with the logic base die that handles signaling.

The stacking step is where HBM earns its price. Through-silicon vias must be aligned across every die in a stack with sub-micron precision, and the finished stack is then mounted alongside a GPU die on the same silicon interposer, a large piece of silicon that acts as a high density circuit board.

Yield compounds across the stack: a four-high or eight-high assembly multiplies the chance that a defect in any layer spoils the whole unit. This is why capacity has been so hard to scale quickly, and why qualification cycles with accelerator makers take months.

05HBM4 and the bandwidth race

The current frontier is HBM4, the generation that SK hynix began supplying to major accelerator makers for qualification in 2025. The specification roughly doubles the interface speed again, with per-stack bandwidth in the region of two terabytes per second, and moves to a wider 2048-bit interface per stack.

SK hynix's product line has traced the standard's evolution, from the HBM2e that established its lead to HBM3e, which it shipped ahead of its rivals into the newest accelerator generations. The company has said its HBM4 design uses a proprietary process it calls an advanced MR-MUF, a molding material that manages heat density in taller stacks.

The engineering problem at each step is thermal as much as electrical: as stacks get taller and faster, getting heat out of the middle layers becomes the constraint on how much bandwidth a package can sustain.

06The economics of a memory supercycle

Memory has always been brutally cyclical, with gluts that crushed prices followed by shortages that restored them. The AI demand wave created an unusual variant: pricing power concentrated in one product family, with HBM sold out years in advance while conventional DRAM rode the same wave more modestly.

That has shifted profit dramatically. SK hynix, which once trailed Samsung in the memory hierarchy, overtook it in HBM share by committing early to the format, and its earnings through 2025 and 2026 have reflected the shift. Samsung has fought to requalify its HBM with the largest accelerator customers, and Micron has expanded from a distant third into a meaningful HBM supplier.

The strategic risk for buyers is concentration: an accelerator supply chain that routes through three memory vendors, most of whose advanced capacity sits in one country, is a fragile thing for a trillion-dollar industry to depend on.

07Limits and open questions

HBM is not a permanent answer to the memory wall; it is a running battle. Stacks cannot grow much taller before bonding, testing and heat make each additional layer uneconomic, and the industry is exploring hybrid bonding, where dies are fused directly without intermediate material, as the next step.

There is also the question of what happens if model architectures shift. Techniques that reduce how often weights must be reread, or hardware that keeps more of a model resident in compute-adjacent memory, could soften the demand curve. Nothing on the near horizon replaces the need for bandwidth, but the slope of demand is less certain than the current sold-out order books suggest.

For now, the practical question is capacity. Every AI accelerator forecast is, underneath it, a forecast of how many HBM stacks can be produced, bonded and delivered on schedule.

References