Intel's 32nm, Sandy Bridge-EP (Efficient Performance) Xeon E5-2600 (Fig. 1) is now the top of the line server platform. It's 8 cores deliver 16 threads via Intel's HyperThreading support and it also supports Turbo Boost 2.0 overclocking. The chips also support Advanced Vector Extension (AVX) floating point math (see Intel's AVX Scales To 1024 bit Vector Math). The multi-socket E5-2600 comes in 17 different configurations ranging in price from $198 to $2,050. There is a single socket version named the E5-1600 that targets small servers and workstations. The release of this latest chip has opened the floodgates to a host of related announcements from vendors using or supporting the Xeon E5.
Fig. 1: Intel's 8-core Xeon E5-2600 deliver 16 threads via Intel's HyperThreading support.
The chip is 434 mm2 with 2.27 billion transistors (Fig. 2). The processors support up to 768 Gbytes of system memory and runs up to 3.3 GHz. It works with the "Patsburg" C6000 chipset. The combination was code named Romley.
Fig. 2: Xeon E5-2600 has 2.27 billion transistors. The 8-cores are in the center.
Each core has a 32 Kbyte L1 data cache, a 256 Kbyte L2 cache and up to 20 Mbytes of L3 cache. Inside the chip is a high speed, bidirectional ring that links the L3 cache slices with the quad memory controllers, I/O controllers and dual 8 Gtransfer/s QPI (Quick Path Interconnect) links. The memory controllers support up to three DDR3-1600 sockets for a total of 12 DIMMs but more on memory later. The I/O controller provides 40-lanes of PCI Express Gen 3.0 support and it eliminates the need to run through an off-chip interface. The C6000 adds x8 PCI Express Gen 2 links and slower devices such as USB 2.0, 3 Gbit/s and 6 Gbit/s SATA ports and SM Bus for system management.
Fig. 3: The QPI links tie a pair of Xeon E5-2600 together into a 16-core system.
Intel Integrated I/O (Intel IIO) and Intel Data Direct I/O (Intel DDIO) Intel DDIO allows I/O traffic from devices like Intel's X540 Ethernet controllers to be routed directly to the cache. Intel Integrated I/O (Intel IIO) provides support for PCIe non-transparent bridging. The on-chip ring, I/O support and on-chip PCI Express links provide a significant I/O performance boost over earlier Xeon chips.
The C6000 supports TPM (Trusted Platform Module) while the Xeon E5-2600 includes support for Intel's Advanced Encryption Standard New Instruction (AES-NI14) instructions. This allows it to keep up with encrypted network traffic as well as providing general security support. The chip also supports Intel's Trusted Execution Technology (TXT15).
New Xeon E5 Systems
Companies such as Dell and Kontron have new, multi-chip systems that host Xeon E5 processors. Dell's new 12th generation PowerEdge servers (Fig. 4) have a pair of sockets for the Xeon E5. The 1U and 2U rack mount PowerEdge 620 720 support PCI Express Gen 3.0 including multiple GPGPUs that may require up to 300W of power. The new Select Network Adapter is a daughtercard that holds eith four 1G Ethernet ports or a pair of 10G ports and two 1G ports. The PERC (PowerEdge RAID controller) uses LSI's RAID chip supports up to RAID 60. It can also run LSI's CacheCade firmware. CacheCade can blend solid state disk (SSD) storage as a cache for hard disk storage. The new system has more 2.5-in drive slots as well as PCI Express slots designed to handle PCIe-based solid state storage.
System management has been improved with Dell's Lifecycle Controller and Identity Module. The agent-free management system streamlines system configuration and upgrades. The Identity Module is software that allows systems to be customized while being easy to maintain. It can incorporate features like custom thermal algorithms. The Identity Module also supports an LCD front panel allowing front panel configuration.
Fig. 4: Dell's 12th generation PowerEdge servers supports a pair of Xeon E5 processors and up to 768 Gbytes of DDR3 memory. It has pluggable slots for PCI Express devices.
Kontron targets the embeddd space with a range of products for the Xeon E5 including the AT8060. The AT8060 is an AdvancedTCA single board computer with a pair of 10G Ethernet ports. It has an AMC bay with PCIe x8 Gen 2 support as well as SATA connectivity.
Fig. 5: Kontron's AT8060 is an AdvancedTCA blad with 10G Ethernet and a pair of Xeon E5 processors.
The AT8060 supports up to 128 Gbytes of DDR3 memory. It also has integrated SAS support. An optional rear transition module (RTM) supports a pair of hot swap, 2.5-in SAS drives.
Storage For The Xeon E5
768 Gbytes of RAM is a lot of memory. Getting that onto a dozen DIMMs is not easy but there are options. Inphi is looking to have its interface chip on LRDIMMs (Load reduced DIMM). The Xeon E5 can handle 3 DIMMs per channel but to get a lot of memory on the DIMMs requires either very dense memory chips or more chips.
This approach can lead to multirank DIMMs (Fig. 6). These provide higher density devices and a number of 32 Gbyte LRDIMMs are available now with 64 Gbyte versions on the horizon. Dual row and even quad row configurations are possible using Inphi's chips and only expose the processor to a single load per DIMM regardless of the number of memory chips. Using "tall" DIMMs allows lower cost memory chips to be used to make higher capacity DIMMs.
Fig. 6: Inphi's interface chip allows four ranks of memory to be placed on a single DIMM.
The Xeon E5 systems are likely to have a lot of flash memory storage as well. Products like Micron's P400e (Fig. 7) are designed to deliver storage throughput necessary for cloud computing.
The P400e is a 6 Gbit/s SATA drive. It comes in capacities up to 400 Gbytes. Even the high end version only use 5W of power. It has a write capacity of 175 Tbytes. The MLC drive targets entry enterprise applications where fast boot times and reading speed is important.
Fig. 7: Micron's P400e solid state disk drive provides the kind of fast access storage that Xeon E5 systems will require.
Intel's Xeon E5 has been long awaited and will be challenge to AMD's 16-core Opteron processors. AMD's chips are single threaded to the top end Xeon E5 and the top end Opteron both run 16-threads at once. The latest crop of Intel chips target dual processor systems with future versions expected to target 4-way (socket) systems. AMD's Opteron line supports up to 16-way systems.