Arm is pitching its AGI CPU as the bit of silicon that keeps AI agents from tripping over themselves.
At Hot Chips 2026, Arm Cloud AI Business Unit vice-president of marketing Eddie Ramirez and Arm senior director of product management Jeff Defilippi walked through the Arm AGI CPU and its customer pull from the AI crowd.
According to Arm it shipped 1.5 billion Neoverse cores into datacentres, with 500 million of those arriving during the past nine months. It claims more than 60 per cent share as the host CPU in AI server platforms and says more than 120,000 companies now deploy Arm in the cloud.
The outfit reckons agentic AI is driving the surge. Arm says AI agents’ token usage could rise 24 times by 2030, which means the humble CPU is suddenly back in fashion after years of accelerator worship.
Arm splits enterprise automation tasks roughly 50-50 between CPU and accelerator. Software development tilts 60-40 towards the CPU, while financial analysis leans 40-60 towards accelerators.
The CPU earns its keep in tool use, orchestration, planning and reasoning. Accelerators handle LLM inference, while CPUs take the hit from IPC, I/O, concurrency, branch prediction, cache, memory latency and bandwidth.
The AGI CPU is based on Arm’s Neoverse V3 core architecture. It packs up to 136 cores per chip, 2MB of private L2 cache per core and clocks of up to 3.7GHz.
The chip is built on TSMC’s 3nm process and uses a dual-chiplet design. Each chiplet carries more than 50 billion transistors, while the complete CPU is rated at up to 300W.
ARM has put memory and I/O on the same die to target sub-100 ns memory latency. The chip supports 96 PCIe Gen6 lanes, CXL 3.0, AMBA CHI extension links, and DDR5-8800 memory with up to 6 TB per chip.
The Neoverse V3 core is a 10-wide machine with a 384-plus out-of-order window, a 10-wide dispatch and eight-wide retire. It has 64KB instruction cache, 64KB data cache, eight ALUs, three branch units and a 10-cycle L2 load-to-use path.
Arm is not doing this as one giant lump of silicon. Each AGI CPU uses two chiplets, each with up to 70 cores, although four are disabled for yield, leaving 136 active cores per CPU.
Each chiplet gets 140MB of private L2 cache, a coherent CMN interconnect, 1MB per core of shared system-level cache and security and control subsystems. The chiplets hook into DDR5, PCIe Gen6, CXL, UCIe and a coherent mesh network with multisocket support.
Arm is pushing reference 1U dual-node and 2U2P servers, while partners including Supermicro, ASRock and Lenovo are lining up tray designs. Supermicro is offering rack-scale 2U four-node and 1U four-node systems with liquid cooling support.
At rack scale, Arm says the design can pack up to 8,160 CPU cores across 30 1U servers inside a 36kW power envelope. The company claims more than twice the performance per rack versus an x86-based system in the same 36kW design.
Demand expectations have been nudged upwards, too. Arm has raised its customer demand forecast for AGI CPU to $2 billion from the earlier $1 billion view for 2027 and 2028.
The AGI CPU is only part of Arm’s wider attempt to make itself unavoidable in AI infrastructure. The same Armv9 foundation shows up in Nvidia Vera and Rosa, Microsoft Azure Cobalt, Google Axion, AWS Graviton and Fujitsu Monaka.
