Arm Now Sells Hyperscaler Server Chips

Arm’s CEO, Rene Haas, showed off the company’s new, chiplet-based AGI CPU multicore processor chip (Fig. 1) targeting hyperscale data centers that are delivering agentic AI support to the cloud. Chiplet technology allows developers to address the reticle limit. The chip has garnered supporters such as Meta, OpenAI Microsoft, and NVIDIA, as well as companies that offer data center server solutions like Lenovo and Supermicro.

The AGI CPU consists of two identical chiplets with 68 Neoverse V3 cores, for a total of 136 cores. The cores support the Armv9.2 instruction set architecture (ISA). This includes dual 128-bit SVE2 (Scalable Vector Extension 2) units for each core. The ISA works with bfloat16 and INT8 MMLA instructions for artificial-intelligence (AI) acceleration, but separate GPGPUs or AI accelerators will be used for heavy-duty training or inference chores.

Each core also has a 2-MB L2 cache, and they can run at speeds up to 3.7 GHz. The chip is based on TSMC’s 3-nm technology with a TDP of 300 W.

The PCIe Gen 6 interface has 96 lanes and supports CXL 3.0. CXL supports memory expansion using the PCIe interface. Regular memory is provided via external DDR5-8800 DRAM. Each core has a 6-GB/s bandwidth with a capacity limit of 6 TB per chip. The system is designed to deliver sub-100-ns latency.

Arm is allowing vendors like Supermicro and Lenovo to deliver rack-based solutions, but the company did highlight what these systems will look like built around a 2-node, 1U configuration (Fig. 2). This packs in 272 cores per blade, with a 30-blade rack having 8,160 cores. An air-cooled version draws 36 kW. Arm’s partner, Supermicro, has a liquid-cooled, 200-kW solution that packs in 336 CPUs for a total of 45,696.

Chasing the Agentic AI Dollars

Many x86 data center servers have been paired with GPGPUs like NVIDIA’s Blackwell. The AGI CPU looks to replace the x86 platforms from Intel and AMD. These server CPUs represent a significant cost when it comes to chips and power needed to run the cloud data centers. By building and selling its own chip, Arm looks to acquire a significant chunk of that money.

Using Arm processors in the data center isn’t new. Amazon’s Graviton is employed by Amazon Web Services (AWS) and Ampere’s Arm-based CPU can be found in Microsoft’s cloud servers. These vendors have used Arm’s IP to develop their own chips. However, Arm’s AGI CPU may be of interest because they wouldn’t have to support the development effort in the future, although incorporating custom IP may or may not be beneficial.  

Keep in mind that Arm named the chip AGI CPU. Agentic AI tends to be all the rage these days, but the chip isn’t really an AI powerhouse along the lines of something like NVIDIA’s GPGPUs. The AGI CPU is a high-performance CPU that can support agentic AI applications in conjunction with AI accelerators, though they will not be doing the AI heavy lifting. They will be running the conventional code to perform the agent’s actions implemented via conventional programming.

Getting Your Hands on an AGI CPU

The AGI CPU isn’t a likely candidate for embedded systems and will probably not be found outside of higher-end servers at this point. A lot of these will plausibly be made; Meta is just one of the major AI cloud providers that’s adopted the AGI CPU for its Arm-based solutions.

It will be possible to get blades and rack systems from the likes of Lenovo and Supermicro. Still, users will have to determine how economical they will be compared to other Arm and x86 solutions.

Arm’s announcement is significant but limited. The company isn’t talking about future directions other than those aiming at the same AI data center target (Fig. 3). Moving down the food chain may meet with limited success and would definitely impact its current customers that have established solutions based on Arm IP.

Likewise, Arm will continue its Compute Subsystem (CSS) program that will allow customers to replicate and extend the capabilities of an AGI CPU-style solution. Whether anyone would want to replicate an AGI CPU will depend on what additional features could be included to make such an approach worthwhile.