Most New Laptops Ship With an NPU, but Few Apps Use It Yet

Most new laptops now include a neural processing unit (NPU), a chip designed to run AI workloads at much lower power than a CPU or GPU. Microsoft's Copilot+ PC certification drove much of that adoption, but few apps use NPUs yet, and Engadget found the NPU sat idle on a reporter's test laptop.

What does an NPU do in a laptop?

Bought a new laptop recently? There's a good chance it has a neural processing unit, or NPU, and a good chance you aren't using it. The chip is a block of silicon designed to run AI workloads at a fraction of the power a CPU or GPU needs. Engadget puts the difference at roughly 2 watts for an NPU compared with well over 100 watts for the discrete graphics in a standard laptop doing the same job. NPUs aren't in every machine yet, but they're now standard in most mid-range and premium laptops.

Advisory firm Gartner estimates that AI-capable PCs, including NPU-equipped laptops, make up more than half of total global shipments. Buyers aren't necessarily asking for them. The push comes from certification requirements. In 2024, Microsoft set a bar that laptop makers needed to hit to qualify for certain new Windows AI features: 40 TOPS, a measure of how quickly a chip can work through AI math, of dedicated NPU performance. Hardware is now moving fast to meet that target, while the software that uses it is lagging behind.

What an NPU actually does

An NPU is built to speed up inference, the stage where a trained model runs and produces results. That differs from training, the heavier process of building the model in the first place, which typically relies on GPUs. Neural networks depend heavily on two kinds of computation, matrix multiplication and nonlinear activation functions. NPUs are designed to execute those operations quickly and efficiently. The tradeoff is some of the general-purpose flexibility that CPUs and GPUs offer.

Background blur during a video call is a good example. Most people have used it, and it can run on either a CPU or a GPU. Both draw a lot of power, which generates heat and drains the battery quickly on a thin laptop. An NPU can handle the same work using a small share of the energy, so effects like that can run continuously without wearing down the battery or making the fans spin up.

NPUs won't help with everything. They aren't useful for training AI models. Running a large local language model such as Llama 70B still requires a discrete GPU with substantial video memory. Services like ChatGPT and Gemini don't need an NPU either, since they run on the provider's infrastructure. For now, the NPU's main role is handling a small set of always-on, low-power tasks, including webcam effects, noise suppression, real-time translation and local search indexing.

Which laptops have them and how they're measured

Why do new laptops include NPUs?

Microsoft's Copilot+ PC certification is the main reason so many laptops now include NPUs. To earn the label and unlock certain on-device Windows AI features, a laptop needs a 40 TOPS NPU, 16GB of RAM and 256GB of storage. The 40 TOPS figure must come from the NPU itself. Performance from a GPU doesn't count. A mid-range laptop with an Nvidia RTX 4050 offers more than 190 TOPS from its tensor cores, but Microsoft doesn't count that toward the requirement.

Qualcomm's Snapdragon X Elite and Plus chips were among the first to reach market with a 45 TOPS Hexagon NPU. Intel's Lunar Lake followed with up to 48 TOPS, and AMD's Ryzen AI 300 came in at 50 TOPS. Intel's newer Panther Lake processors, sold as Core Ultra Series 3, carry a 50 TOPS NPU and are built on Intel's 18A process. Qualcomm has since pushed the figure to as much as 85 TOPS with its Snapdragon X2 Elite, more than double Microsoft's minimum. Each generation has raised the number, even though the software that could use that capacity remains scarce.

Are apps using laptop NPUs yet?

Owners can check whether their laptop's NPU is doing anything. On Windows 11, the utilization rate appears in Task Manager. On an Engadget reporter's 2024 ASUS Zenbook Duo with 32GB of RAM, the reading sat at zero. Engadget expects most casual users to see the same thing. The list of apps that regularly rely on the NPU is short: Windows Studio Effects, Live Captions, Recall and Click to Do, noise suppression in Teams and Zoom, and a handful of Adobe Creative Cloud features.

Running a local model on the NPU

To test how much of an NPU a local AI model would consume, Engadget downloaded Mistral 7B, an open-source model it described as fairly capable and efficient. The reporter ran it through Intel's OpenVINO toolkit on a laptop with 34 platform TOPS across the CPU, GPU and NPU. Only 11 of those TOPS belong to the NPU. The model pushed the NPU to 98 percent utilization while the CPU stayed at 3 percent. Text generation worked, and the laptop stayed responsive, though 11 TOPS left little room for a model that size.

The same model run through Ollama, which uses the CPU rather than the NPU, pushed CPU utilization to 52 percent and held memory use at about 17GB of the machine's 32GB. Engadget said that heavier CPU load would have bogged down the laptop if other demanding tasks were running. That contrast is the appeal of an NPU: It handles an AI workload without affecting the rest of the system. The catch is that getting there required a specific toolkit and technical knowledge. Very little software supports NPUs out of the box, and that gap is the main reason the chips sit idle on most machines.

Este artículo fue escrito con la asistencia de IA.
News Factory APP - noticias agénticas para impulsar tu SEO y AEO.