Why AI Laptops Failed: The NPU Reckoning After the Copilot+ Hype

01 THE PROMISE: A TOPS BUDGET IN EVERY LAPTOP

When the Copilot+ PC program launched in mid-2024, its defining number was forty. A qualifying laptop needed a neural processing unit rated at forty trillion operations per second \u2014 TOPS \u2014 sixteen gigabytes of memory, and 256 gigabytes of storage. The number mattered less for what it measured than for what it organized: an entire procurement cycle, from Qualcomm’s Snapdragon X Elite through Intel’s Lunar Lake and AMD’s Ryzen AI 300, aligned itself around clearing a single benchmark gate. Within a year, laptop makers were advertising NPU horsepower the way they once advertised gigahertz, and retail shelves filled with machines stamped “AI PC” as if intelligence were a certificate that shipped in the box.

The marketing framing was a category error hiding in plain sight. A TOPS figure describes peak throughput under ideal conditions \u2014 INT8 workloads, fully utilized matrix units, data already resident in the right memory. It says nothing about whether any shipped software actually schedules work onto the NPU. Buyers were invited to believe that a silicon capability was the same thing as a product experience, and at launch the gap between the two was nearly total: the flagship Copilot+ features were thin, and the operating system itself rarely touched the accelerator.

This article is the post-mortem of that gap. It examines what the promise assumed, why the accelerators mostly sat idle, and the narrower set of cases where on-device inference genuinely earns its silicon today.

02 THE BANDWIDTH WALL UNDER THE TOPS

The first reason the accelerators idled is physics-adjacent arithmetic. An NPU at forty TOPS doing INT8 math needs to feed roughly forty terabytes of operand traffic per second to its multiply-accumulate arrays. DRAM on a thin laptop delivers a fraction of one terabyte per second \u2014 LPDDR5X on the best Copilot+ hardware peaks near 120 to 277 gigabytes per second depending on generation. The consequence is architectural: any model whose weights and activations exceed what fits in on-chip SRAM becomes memory-bound, and the NPU spends its life waiting on the bus. The forty TOPS headline measures a state the chip can enter only for brief bursts on small working sets.

This is why quantization became the whole game. Shrinking a model from sixteen-bit to four-bit weights cuts bandwidth demand per token roughly fourfold, which is the only way tens-of-billions-parameter models approach practical speed on laptop memory. But aggressive quantization taxes quality unevenly \u2014 reasoning and long-context tasks degrade before casual benchmarks reveal it. The industry discovered that the binding constraint on client AI was never operations per second; it was bytes per second, and bytes per second is set by the memory subsystem that marketing never put on the sticker.

Small models genuinely do fit in SRAM and cache, and they fly \u2014 that is the seed of truth in the NPU story, and Section 06 returns to where it pays.

03 SOFTWARE GRAVITY: NOTHING TO RUN ON THE NPU

A processor without software is furniture. At Copilot+ launch, the marquee experiences were a small set: live captions with translation, several Studio Effects for webcam video, Windows Studio enhancements, Cocreator in Paint, and Recall, which arrived so freighted with security objections that it was delayed, reworked, and rebranded before reaching most users. Meanwhile the applications people actually open all day \u2014 browsers, office suites, chat clients, IDEs, photo tools \u2014 did not schedule inference onto the local NPU at all. The accelerators shipped; the workloads did not.

The reasons are structural rather than lazy. ISVs target the installed base, and in 2024 and 2025 the installed base was overwhelmingly machines without NPUs, or with NPUs of wildly varying capability across three vendors and a long tail of older silicon. Building a feature that requires forty TOPS means excluding most of your customers. Abstraction layers such as ONNX Runtime, DirectML, and later the Windows ML stack lower the porting cost, but they do not manufacture demand, and they add a second performance cliff: generic runtimes often fail to hit vendor-tuned peak numbers on any given accelerator.

Worse, what demand existed was being captured elsewhere. The same ISVs evaluating NPUs were watching token prices fall and context windows grow on the cloud side, which brings us to the deepest force in this story.

04 CLOUD GRAVITY AND THE FRONTIER’S ESCALATION

While client silicon sprinted toward forty TOPS, the frontier sprinted elsewhere. Every leap in hosted model capability \u2014 longer context, multimodal input, tool use, reasoning chains \u2014 raised the ceiling of what a product team wants its AI features to do, and that ceiling ran off the top of what any laptop accelerator can execute. The desirable model of a given year is a two-hundred-billion-plus-parameter construct served from a data center with HBM and liquid cooling; the NPU’s honest envelope is a few billion parameters running quantized. Product roadmaps follow capability, so product roadmaps followed the cloud.

Economics sealed it. A developer integrating one cloud API inherits frontier-grade quality, pays per token, ships zero local requirements, and reaches every device including the billion existing machines without accelerators. The same feature built on-device demands per-vendor tuning, model distribution and update plumbing, thermal negotiation, and a support surface fragmented by silicon generation \u2014 for an experience that is, in most categories, visibly worse. Hybrid designs that offload some steps locally exist, but the split adds complexity that only privacy-critical workloads justify.

The result was a category inversion. The “AI PC” was sold as the place AI happens; it became the place AI waits, while the interesting inference happened in buildings the user never sees.

05 WHERE ON-DEVICE INFERENCE ACTUALLY EARNS ITS SILICON

The honest post-mortem is not that NPUs do nothing; it is that their wins are narrow, specific, and were never worth a whole category rebrand. Always-available workloads top the list: camera framing and eye contact correction, background blur and noise suppression on calls, live captions, and transient transcription. These run continuously, cost milliseconds of latency tolerance, send zero bytes to a network, and would otherwise drain battery through CPU cycles. For them, a low-power accelerator is the right tool and battery-life measurements show real, if modest, gains on systems that use it.

Privacy-bounded tasks come next: local document search and semantic indexing over personal files, on-device classification of screenshots and content, and short-form generation where a small quantized model is genuinely sufficient. Latency-sensitive and offline-first scenarios \u2014 translating a conversation on a plane, dictation without signal \u2014 also hold. The common thread is a model small enough to be memory-feasible and a job the cloud either cannot be trusted with or cannot reach.

Everything else \u2014 coding assistants, general chat, deep research, image generation beyond toys \u2014 still belongs to the data center for the foreseeable future, because capability there compounds faster than laptop memory grows.

06 THE RECKONING, AND WHAT A SECOND ACT REQUIRES

The AI PC era will likely be remembered as a miscalibration, not a fraud. Silicon engineers delivered roughly what was asked: real accelerators at real efficiency. What failed was the inference that capability creates demand. Demand follows software, software follows the installed base and the capability frontier, and both of those pointed away from the local NPU for the first two years of the program. The discount the market now applies to the “AI laptop” label is the price of that miscalibration, and it is paid by every machine whose only AI feature was its sticker.

A credible second act requires three shifts. First, memory capacity and bandwidth must grow until mid-size models run locally at usable speed \u2014 the unification of memory architectures on laptop SoCs is the quiet enabler here. Second, a killer workload must emerge that the cloud structurally cannot serve: something continuous, private, and context-rich over the whole history of everything on the device, not another chat box. Third, the software layer must consolidate so one implementation runs at acceptable efficiency across Qualcomm, Intel, and AMD silicon without per-vendor heroics.

Until then, the practical advice is unglamorous. Buy laptops for keyboard, display, battery, and thermals; treat the TOPS figure as a specification artifact of 2024 through 2026; and let the NPU earn its keep in the quiet corners \u2014 the blur, the captions, the search box \u2014 where it was genuinely useful all along.