Anatomy of an AI Cluster: What a Blackwell Deployment Actually Contains

01WHAT AN AI CLUSTER PHYSICALLY IS

AI cluster usually arrives as a metaphor — a cloud region, a training run, a round number of GPUs. ServeTheHome's tour of a Supermicro-built NVIDIA B200 deployment for Lambda, housed at a Cologix facility in Columbus, Ohio, replaces the metaphor with racks: GPU trays in dense rows, network cabling in overhead trays, storage arrays in their own aisles, and a power and cooling plant sized like a small factory's. The deployment dates to 2025, but its architecture is the template current-generation buildouts still follow.

The physical inventory matters because every capability of the cluster is a property of something you can photograph. Training throughput comes from the GPU trays; coordination comes from the fabric; checkpoint recovery comes from the storage arrays; and the ceiling on all of it is the building's power and cooling contract with the grid.

02THE COMPUTE LAYER

The compute layer is B200 trays — each carrying multiple Blackwell-generation GPUs — stacked into racks at a density that makes colocation contracts complicated. Blackwell concentrated the ratios the industry was asking for: more memory bandwidth per GPU, higher interconnect bandwidth per tray, and power draw per rack that pushed air cooling toward its practical limit.

The arithmetic to internalize is per-rack power. A modern GPU rack draws in the tens of kilowatts, and a row of them approaches what an office building uses. This is why cluster tours spend as much time on busways, switchgear, and coolant plumbing as on GPUs: at this density, the compute layer and the electrical plant are the same engineering object.

03THE FABRIC

The tour's most instructive detail is that the cluster carries two separate networks. Training traffic rides NDR InfiniBand — a lossless fabric engineered for the all-reduce patterns of synchronized gradient descent, where thousands of GPUs exchange gradients in lockstep and any dropped packet stalls a superstep. Storage and management traffic rides 400-gigabit Ethernet, the layer that connects arrays, nodes, and operators without contending with the training burst.

The two-network design answers a real asymmetry. Training synchronization demands latency and losslessness at microsecond scale; storage traffic demands bandwidth and reliability at seconds scale. Compressing both onto one fabric either taxes the training loop or overbuilds the storage path — so production clusters simply pay for both, and the cable trays show it.

04STORAGE

The deployment pairs its GPUs with tens of petabytes of VAST clustered storage, and the pairing is not incidental. Training runs write checkpoints constantly — full snapshots of model weights and optimizer state — because any failure anywhere across thousands of GPUs must cost hours, not weeks. Checkpoint input-output is write-heavy, bursty, and shared across the entire cluster, which is why it wants a purpose-built storage architecture rather than a general-purpose file server.

Storage throughput quietly shapes training wall-clock. If checkpoint cadence outpaces array bandwidth, nodes stall mid-run; if checkpoints are too rare, a crash eats days of progress. The storage aisle of a cluster tour is where the difference between a 90-percent and a 95-percent utilized cluster usually hides.

05POWER AND COOLING

The physical ceiling of the deployment is the facility's power and cooling envelope. Columbus was chosen for the same reasons the industry's buildout map shows clusters clustering there: grid capacity, industrial power pricing, and land. Air cooling still handled this generation of racks with containment and fan walls, but the density arithmetic points at where the industry was already heading — direct-to-chip liquid cooling and rear-door exchangers as rack power crossed what air can economically move.

Cooling is not overhead to minimize; it is a design variable that sets achievable density. Every kilowatt spent moving heat is a kilowatt not spent training, and the ratio degrades with ambition. The higher-density buildouts announced since are, in engineering terms, mostly cooling retrofits with GPUs attached.

06WHY TOPOLOGY MATTERS

The reason fabric and storage choices reach the news is wall-clock training time. Large-model training is a synchronization problem: thousands of GPUs must complete each gradient step together, and the slowest path in the fabric sets the pace for all of them. Non-blocking topologies, tuned all-reduce algorithms, and checkpoint pipelines that never stall the loop convert directly into days saved on a multi-month run.

This is also why cluster design resists commoditization. Two facilities with identical GPU counts can differ by double-digit percentages in effective throughput based on network topology, storage layout, and software-stack tuning. The spreadsheet that matters is not the spec sheet; it is the sustained utilization the topology permits.

07WHAT A TOUR CANNOT SHOW

A walkthrough shows capital and topology; it cannot show the two variables that decide whether the investment pays. The first is utilization — what fraction of purchased compute survives the software stack, the straggler nodes, and the scheduling gaps. The second is economics: the power contract, the colocation terms, and the demand pipeline that determine whether the cluster's output sells above its running cost.

That is the honest limit of the tour format, and the reason cluster tours keep proving useful anyway: the industry's race is now physical enough that its constraints are visible. The 2026 buildout's real story — power limits, two-network fabrics, storage sized for checkpoints — can be read off the cable trays months before it appears in anyone's earnings call.