For years, commercial AI has mostly meant CNNs, recommender systems and LLMs, with GPUs doing the heavy lifting. That picture is changing fast as agentic AI, reasoning models and mixture‑of‑experts (MoE) architectures go mainstream.
Agents: orchestration needs CPUs
An agent is not just another model. It’s a higher‑level orchestrator that:
- sends tokens to one or more models
- inspects the results
- makes decisions with
if‑then‑elsestyle logic - calls APIs, runs tools, issues follow‑up queries
This branching, decision‑heavy control flow is exactly what CPUs are good at – and what GPUs are bad at. In classic LLM serving, the CPU mainly acts as a host and PCIe traffic manager, so one CPU can feed multiple GPUs. Typical clusters ended up around 1 CPU : 4 GPUs.
With agentic workloads, the CPU is no longer just a host; it executes substantial logic. Industry voices now talk about CPU:GPU ratios drifting toward 1:1, which is already driving up demand – and prices – for high‑core‑count, power‑efficient server CPUs.
Reasoning, MoE and SLMs: still GPU‑centric
Other emerging model types still lean heavily on accelerators:
- Reasoning models run extra inference steps and may expose their chain of thought. OpenAI’s O‑series and DeepSeek‑R1 are early commercial examples.
- MoE models replace some dense FFN layers with many parallel expert FFNs plus a learned router that picks which experts see each token.
- SLMs (small language models) are compact, domain‑specialized models that run independently.
All of these can be used by agents, but the expensive tensor math still lives on GPUs/NPUs. The agent layer above them – planning, sequencing, tool use, JSON‑driven workflows – is what’s pulling the CPU back to center stage and forcing a rethink of AI server architecture.
Source: Semiconductor Engineering
