PCI Express

Why the need for PCI Express?
As processor clock speeds increase, parallel buses such as PCI become harder to implement. Signal skew and fan-out restrictions restrict the bandwidth achievable on a parallel bus. PCI Express (PCIe) has arisen to break the bandwidth barrier while maintaining software compatibility with the popular PCI bus.

The problem that PCIe aims to solve is the inability of conventional parallel buses to scale to today's processor speeds. As illustrated in Figure 1, a parallel, multi-drop bus such as PCI must keep many signal lines synchronized in order to reliably send information. Minor variations in signal trace layout and load capacitance from line to line affect signal propagation times, resulting in skew among the signals. Bus timing must accommodate skew by positioning the clock signal within a time window that allows the other signal lines to stabilize to their final values at all points along the bus before the clock arrives.

Skew in a typical design is only a few nanoseconds. As bus clocking frequencies increase, however, those nanoseconds become an ever larger proportion of the bus's cycle time, resulting in a shrinking clock window. In addition, skew tends to increase with increasing signal frequency, which shrinks the window even more. Careful control of signal trace layout and loading can ease the situation, but ultimately these effects place a hard limit on the achievable bus clock rate. The PCI bus has reached that limit.

BEYOND PCI COMPATABILITY
While PCIe is able to transparently support existing system software, it also offers some useful features that new software can utilize. One of the most important is the ability to assign attributes to packets, such as no-snoop, relaxed-ordering and priority. The system can use these attributes to optimally route packets through the I/O system. Thus, PCIe can easily support Quality of Service (QoS) features needed for such applications as video and VoIP.

How Does PCIe Maintain Compatibility With PCI?
The segmentation and reassembly at the physical layer is a key component of the PCIe bus design. The hardware-based formatting helps to keep the upper layers of the system software (see Figure 2) isolated from the physical structure. Other isolation components are in place at both the link and transaction layers, so that the driver, operating system, and applications software are completely isolated from the PCIe bus structure. This allows PCIe to replace PCI in hardware design without requiring any software changes.

The link layer, for instance, serves to ensure reliable delivery of data packets across PCIe. It attaches a packet sequence number and a cyclic redundancy check (CRC) character to the data packet. The CRC allows the link layer hardware to detect transmission errors. In the event that a data error occurs, the link layer hardware automatically resends packets that have been corrupted. The sequence number allows the hardware on the receiving end to properly reassemble data blocks even if they arrive out of order because of such resending.

The transaction layer interacts with the system software to receive memory read and write commands and initiates the command and data packets at the link layer, marking each with a unique identifier. It also receives response packets from the link layer and uses the identifier to send the response to the correct software element.

In its interaction with the system software, the transaction layer supports the three PCI address spaces—memory, I/O, and configuration—and handles both 32-bit and extended 64-bit memory addressing. It is thus able to fully mimic the load-store architecture and flat memory space of PCI. The transaction layer also includes a Message Space, which PCIe uses to handle all the side-band signals that the PCI bus requires, including interrupts, power management requests, and resets. This use of the Message space, in essence, provides "virtual wires" to replace these signals.

How does PCIe Work?
PCIeeliminates the problem of skew by switching from a parallel bus design to a serial, point-to-point connection, as shown in the diagram of Figure 3. The serial connection of PCIe uses differential signaling operating at a 2.5 GHz clock speed. Even though the connection is point-to-point, the system CPU can use the PCIe bus to connect to multiple peripherals simply by incorporating a switch in the path. The switch can also allow peripherals to communicate directly with one another in peer-to-peer transactions and may also support multiple simultaneous connections.

One or more serial connections, called lanes, may link two devices. Each lane operates at 2.5 GHz and uses 8b/10b serial encoding so that it is self clocking. A single lane provides a connection bandwidth of 200 Mb/s, nearly twice that of conventional PCI. Further, PCIe is scaleable in bandwidth by utilizing multiple lanes in a link.

As many as 32 lanes may be used in a link. During system boot-up or during a plugand-play configuration, the two PCIe interfaces negotiate the number of lanes and the operating frequency they will use to form their link.

Hardware at the physical layer splits data bytes across the available lanes for transmission and reassembles them at the receiving end. This allows the physical layer to handle data in a byte-parallel format when communicating. The use of headers eliminates the effects of skew among lanes by allowing the reassembly to properly handle packets arriving at different times.

Developing with PCIe
Numerous devices using the PCIe interface-have become available, as have silicon cores for custom designs. Designers interested in using devices should ensure that the offerings have been certified to PCISIG specifications. A good place to start is the PCI-SIG Integrator's List, found at: (http://www.pcisig.com/developers/compliance_program/integrators_list/ pcie/), which identifies components and system building blocks that have been proven compliant.

Developers seeking controller cores for custom devices should check if the core has been used in a device that has passed compliance testing. For PHY cores, look for expertise in high-speed board design. In both cases, the more software and design support the vendor offers, the easier it will be to use PCIe to break the bandwidth bottleneck of PCI-based systems.