Ethernet continues to reign atop the list of high-speed interconnects, even as it moves into the 10-Gbit/s space (see "High-Speed Serial Technology Drives Board Interconnects" at www.electronicdesign.com, ED Online 15348). Combine this with the emerging PCI Express I/O virtualization, and the result is Intel's new 82598 10-Gbit/s Ethernet controller and its cousin, which uses the 1-Gbit/s 82575 chip (see "I/O Virtualization," ED Online 15358).
As part of Intel's I/O Acceleration Technology, these adapters specifically target servers where virtual-machine monitors (VMMs) will host operating systems in multiple virtual machines (VMs).
With current technology, the VMM manages shared peripherals instead of having the VM access the peripheral directly. That's because current adapters provide no I/O virtualization support. Instead, a virtual peripheral is delivered to the VM, which creates a performance and management bottleneck. The solution is hardware I/O virtualization.
Hardware I/O virtualization Hardware I/O virtualization isn't new, but its implementation at the PC level is. Part of the reason for this lag is the cooperation required across the hardware. In particular, device adapters must be part of the solution by providing virtual interfaces that are independent of each other from a control standpoint (Fig. 1).
In Intel's case, the 82598 offers 16 virtual-machine device queues (VMDq). This maps well to the 32 transceiver and 64 receiver physical queues per port. Each VM using the adapter will be allocated one VMDq. An operating system in a VM will use the virtual device as it would a real device.
The key difference versus that of a virtual device provided by the VMM is that the operating system will communicate with the Ethernet adapter directly. Also, security measures are in place so that the VM can't get outside its memory sandbox. Essentially, the VM and the hardware will be able to share only the VM's memory, even though the operating system thinks it has access to all physical memory.
Additional hardware that handles address remapping between the device and physical memory control the DMA support used by the network adapter. The support is comparable to that found within a VMM-enabled processor, which doesn't allow a VM to access memory outside its restricted arena. In fact, the VMM will typically combine its control of a VM and any hardware virtualized devices, such as Intel's Ethernet adapters.
The other piece of the puzzle is interrupt splitting, because the hardware must now notify a particular VM for most interrupts versus the VMM. The hardware needs to differentiate between a VM that's currently running, possibly on a different core each time, versus one that's active but not currently running.
In the former case, the interrupt can be delivered to the VM. In the latter, the VM needs to address the interrupt when the VM begins running again. This is analogous to software interrupts for applications, but the implementation tends to be significantly more complex.
Features like direct virtual interrupt and DMA support are usually implemented for performance reasons, but other areas can benefit, too. For example, transmit fairness and head-of-line blocking can be implemented more easily in hardware with the virtual support incorporated into the chips.
Splitting ethernet chores Virtualization is a major feature of the 82598 and 82575 chips, but it isn't the only feature. The chips don't incorporate a full TCP/IP offload engine (TOE), but the TOE still does more than the typical Ethernet adapter. Like the virtualization, it's tuned for multicore processors.
One interesting feature is the ability to split or replicate incoming packets (Fig. 2). Typically, packet processing is based on header information that includes routing data like the Internet Protocol (IP) destination address. Deeper packet processing is possible. But usually when data arrives at a host node, an application will handle this level of filtering.
Header size is often a small fraction of a packet, and the two are often separated for processing. This allows a larger amount of header data to fit more efficiently into a processor's cache. So, in this case, what was once the application code's responsibility is now handled by the hardware.
Another significant feature is receiveside scaling, which distributes interrupts and loads across multiple cores. This type of load balancing can make more effective use of multiple cores.
Check out the tables for more details on the chips' other features. Chip features are usually situated on pre-virtualized hardware, including the Network Controller-Sideband Interface (NC-SI) and SMBus, remote boot loading via PEX and iSCSI, and support for range of standard Ethernet interfaces.
The chips and adapter can support any compatible PCI Express-based host processor. However, some features will only be available with Intel-based processors because of the interaction between various hardware components. Designers will need to keep a close eye on compatibility with the host processors.
Intel www.intel.com