NVIDIA Blackwell explained: how the GB200 superchip architecture powers the AI boom

01 The Blackwell architecture and GB200 superchip

Blackwell is NVIDIA's GPU microarchitecture succeeding Hopper, named after mathematician David Harold Blackwell. At its center is the GB200 superchip, which pairs two Blackwell GPU dies with a Grace CPU on a single package using NVLink die-to-die interconnect. Each Blackwell die contains 208 billion transistors manufactured on TSMC's 4NP process, a refined 4nm node. The GB200 delivers 20 petaflops of FP4 AI performance, roughly 2.5 times the H100's FP8 throughput. This leap comes not from a single larger chip but from connecting two dies so tightly that they function as one processor, a design choice driven by the physical limits of reticle-size lithography.

02 Die-to-die NVLink and chiplet design

The GB200's two Blackwell dies are connected by a 10 TB/s bidirectional NVLink interface, fast enough that software treats them as a single GPU. This chiplet approach allows NVIDIA to bypass the reticle limit, the maximum die size a lithography tool can expose in one shot, by manufacturing two smaller dies and joining them. The interposer beneath the dies uses through-silicon vias (TSVs) to route thousands of connections between the GPUs and the Grace CPU. The design trade-off is complexity: packaging the GB200 requires advanced coWoS (chip-on-wafer-on-substrate) technology from TSMC, which has constrained supply. The result is a processor that delivers monolithic performance with the yield advantages of smaller dies.

03 Transformer Engine and FP4 precision

Blackwell introduces the second-generation Transformer Engine, which dynamically selects between FP8 and FP4 precision for each layer of a neural network. FP4, using 4-bit floating point numbers, halves the memory and compute requirements of FP8 while maintaining accuracy through careful scaling. This is critical for large language model inference, where memory bandwidth rather than raw compute is often the bottleneck. The Transformer Engine monitors activation statistics in real time and routes computations to the appropriate precision format, achieving near-lossless inference at much lower energy cost. For training, the engine supports FP8 end to end, while FP4 is primarily used for inference workloads where the precision-accuracy trade-off is well understood.

04 The NVL72 rack-scale system

The GB200 NVL72 is NVIDIA's rack-scale configuration: 72 Blackwell GPUs and 36 Grace CPUs in a single liquid-cooled rack, connected by 130 TB/s of NVLink bandwidth. This system delivers 1.44 exaflops of FP4 performance in one rack, approximately 30 times the throughput of a comparable H100-based DGX system. The NVL72 uses fifth-generation NVLink switches to create a non-blocking fabric where any GPU can directly access any other GPU's memory at 1.8 TB/s. This is essential for models too large to fit on a single GPU, as it eliminates the PCIe bottleneck that limited earlier multi-GPU systems. The rack weighs over 3,000 pounds and requires liquid cooling throughout, with a total power draw of approximately 120 kilowatts.

05 Memory bandwidth and HBM3e

Each Blackwell die includes 192 GB of HBM3e memory with 8 TB/s of bandwidth, a 1.6x improvement over the H100's 3.35 TB/s. HBM3e achieves this through higher clock speeds and wider bus interfaces, with memory stacks manufactured by SK Hynix and Micron. The memory bandwidth is often the decisive factor for LLM inference latency, because the attention mechanism requires reading the full model weights for each generated token. With 8 TB/s, the GB200 can serve a 700-billion-parameter model at over 300 tokens per second per GPU, a throughput that makes real-time conversational AI economically viable at scale. The 192 GB capacity per die means the largest models can run on a single GB200 without model parallelism.

06 Power and cooling challenges

Blackwell's performance comes at a steep power cost. The B200 GPU has a thermal design power of 1,000 watts, and the GB200 superchip draws approximately 2,700 watts including the Grace CPU. The NVL72 rack consumes about 120 kW, requiring direct-to-chip liquid cooling with coolant flowing at rates impractical for air-cooled data centers. This has forced a rethinking of data center infrastructure: new facilities built for AI workloads are designed around liquid cooling from the ground up, with raised floors for coolant distribution and backup power systems sized for sustained high-density loads. The retrofitting cost for existing air-cooled data centers can exceed $50 million per facility, creating a divide between AI-ready and legacy infrastructure that will shape the cloud computing market for years.

07 Competitive landscape vs Google TPU and AMD MI300

NVIDIA faces growing competition. Google's TPU v5e offers roughly half the performance per chip but at a lower cost and with the advantage of Google's deep software stack integration in the Vertex AI platform. AMD's MI300X, with 192 GB of HBM3 memory and 5.3 TB/s bandwidth, competes directly with the H200 and offers a viable alternative for organizations committed to the ROCm ecosystem. Intel's Gaudi 3, while not matching Blackwell's peak performance, targets the price-performance sweet spot for inference. The competitive dynamic is healthy: it puts downward pressure on pricing and forces NVIDIA to innovate faster. For now, Blackwell's combination of FP4 support, NVLink fabric, and the CUDA software ecosystem gives it a decisive lead in training the largest models, where the software maturity of CUDA remains the most durable moat.