At last, a good reason to buy an AI PC: Reining in runaway token bills

personal tech

Gartner thinks we’re headed to a hybrid AI model where you offload stuff to the desktop whenever possible

Corporate PC buyers haven’t rushed to buy AI PCs but analyst firm Gartner thinks the machines can now do an important new job: Running AI workloads on the desktop to provide a hedge against tearaway token bills.

The firm on Monday published a Strategic Roadmap for Agentic AI PCs in which Research Vice President Steve Kleynhans argues that enterprise AI tools that run on the desktop “have been slow to materialize.”

“On -device AI has yet to achieve mainstream adoption and remains largely confined to developer and enthusiast use cases,” he added.

He thinks that’s about to change as businesses are paying more attention to the cost of cloud-based AI, and especially “Tokenomics,” the frustratingly imprecise science of trying to figure out how each AI service provider defines a “token” and charges for them at different times.

“As enterprises gain a better understanding of AI cloud cost dynamics, many are looking to AI PCs as a potential offset,” Kleynhans wrote. “Enterprises are becoming increasingly concerned about the economic sustainability of cloud-centric AI strategies as token consumption and associated usage costs continue to rise, driving greater interest in a hybrid strategy, by identifying workloads that can be processed more efficiently at the edge or on the device itself.”

The analyst admits “there is no consensus yet on the level of cost benefit” but feels the potential for savings is “clear.”

The main reason for his optimism is advances in development of small language models (SLMs), small reasoning models (SRMs), and targeted domain-specific language models. He’s not alone in that opinion: The Register recently covered the likes of Microsoft and Google using smaller models for some tasks.

Kleynhans thinks some of these smaller models will happily run on today’s AI PCs, which pack neural processing units capable of at least 50 TOPS performance.

Again, he notes that polished tools for enterprise users are yet to materialize. But he thinks that the likes of OpenClaw, plus on-device AI tools such as Claude Cowork, Microsoft Scout, and OpenAI Codex, will show enterprise buyers what’s possible – and spark greater interest in AI PCs.

“Local AI models will support speech, chat, image, audio, and text generation, as well as application and model orchestration,” he predicts. “SLMs and SRMs will power always-on personal assistants and agents, fundamentally changing how users interact with their devices.”

The analyst thinks “Many routine tasks will be executed locally, with personal agents coordinating work across applications, models, and services operating both on the device and in the cloud.”

The analyst therefore offered two numerical predictions:

By 2029, 30 percent of enterprises will be using AI PCs to reduce their cloud AI token costs;

By 2030, 70 percent of the corporate PC installed base will be capable of running some local GenAI workloads.

All those desktop AI workloads will complement clouds, which will continue to handle the most demanding workloads. But Kleynhans thinks mature AI models “will increasingly migrate to the endpoint as they become optimized for smaller systems, transforming the PC from a simple endpoint into a critical component of the broader AI infrastructure.”

Another stimulus will be the increasing power of AI PCs, which Kleynhans thinks will become ten times more powerful by 2031.

The analyst therefore thinks PC buyers need to think about AI PCs as part of an organization’s IT infrastructure and develop an ROI model based on token cost displacement.

“Initially this will be most appropriate for developers but make the discussion part of any new AI deployment for all employees,” he suggests. Another of his recommendations is to start this effort in earnest once third-gen AI PCs appear in 2027. Starting to experiment with SLMs and SRMs is another activity Kleynhans thinks is worthwhile. ®