Google said that it will start offering companies access to a new series of public cloud services powered in part by a unique infrastructure processing unit (IPU) that it designed and developed with Intel.
The companies said the E2000—previously code-named Mount Evans—is designed to offload and isolate a wide range of networking, storage, and other cloud infrastructure chores from the CPUs housed at the heart of servers, making data centers more secure and efficient in the process. The ASIC-based IPU—Intel’s first— belongs to the same class as the data processing units (DPUs) built by the likes of AMD, Marvell, and NVIDIA.
Google is now offering access to the IPU, first announced at Intel’s Architecture Day last year, in a preview of its latest public cloud service, the C3 series. The series is an evolution of its existing C2 instances that are suited for AI and other high-performance workloads. The new instances also will be the first to be powered by Intel's fourth-generation Xeon CPU, code-named Sapphire Rapids, which is set to be widely released in early 2023.
Google, the world’s third-largest public cloud provider, played up the capabilities of the E2000, saying that some early customers are seeing performance gains up to 20% when running workloads on the C3 servers.
Nick McKeown, SVP of the networking business at Intel, touted the new IPU-powered service from Google as “a first of its kind in any public cloud.”
Dreaming of DPUs
Intel has positioned the IPU as a pillar of its strategy to win favor with cloud and technology giants that are increasingly investing in custom silicon as an alternative to Intel’s server processors. The company has several generations of IPUs in development, with its roadmap lasting through 2026.
But Intel is far from alone in its ambitions. The E2000 IPU is being rolled out to directly counter NVIDIA’s Bluefield and AMD’s Pensando DPUs in what has become one of the biggest battlegrounds in the server chip market.
Today, general-purpose CPUs sit at the heart of the data center in more ways than one. In addition to running applications, they’re often responsible for managing software-defined networking, storage, as well as cryptography and other security workloads.
But using the CPU to coordinate data traveling through the network or carry out other infrastructure-related workloads in the data center wastes a large portion of the CPU’s total capacity.
The E2000 can be attached to a cloud provider’s servers via 16 lanes of PCIe Gen 4. Once attached, the IPU (and the accelerators inside that set it apart from related class of server hardware called SmartNICs) takes over auxiliary tasks such as managing storage infrastructure and networking from a host CPU. Doing so leaves more of the CPU’s clock cycles to run applications or—in the case of the cloud—client workloads.
An IPU also offloads workloads in a way that physically isolates them from a host CPU. This helps improve security between customers that are renting out computing power or other resources from the same CPU.
Inside the Package
Between the high-performance Arm CPU cores and accelerator IP inside, the Intel-Google IPU offers more than enough processing power to act as a sort of server within a server for infrastructure purposes.
The E2000 can be configured with up to 16 CPU cores built on Arm’s Neoverse N1 processor design. The N1 also happens to be the same core at the heart of Amazon’s second-generation Graviton server CPU.
The CPU cores are supplemented by a programmable packet-processing engine that runs at up to 200 GB/s for networking purposes plus Ethernet ports to match. There’s a 1-GbE port for out-of-band management.
For networking purposes, the E2000 also has hardened logic that runs the RDMA over Converged Ethernet (RoCE) protocol. RoCE supports remote direct memory access from server to server without any involvement from the CPU or operating system (OS) running on top of it. Offloading data movement from the CPU in a server brings gains in both performance and power efficiency, while keeping latency in check.
On top of its core networking features, the E2000 is designed to speed up storage-related workloads, too. The chip supports NVMe and NVMe over Fabrics (NVMe-oF) protocols for accessing flash storage in data centers.
Google said its IPU-powered service offers 10X more input and output operations per second (IOPS) and 4X the throughput of its predecessor when combined with Google’s new “Hyperdisk” block storage technology.
The IPU also includes crypto accelerators that secure data at line rate, or the speed of the rest of the network. This hardware—along with the data-compression engine in the E2000—are adapted from the QuickAssist technology in Intel’s Xeon Scalable CPUs.
Intel has the option to sell the “hyperscale-ready” IPU, as executives have called it, to other customers besides Google Cloud. The company has previously said it expects those shipments to start in 2023.