PCI took over the desktop and has muscled its way into embedded systems. The serial PCI Express will eventually replace its parallel cousin in almost every avenue. Initially, bridge chips will provide access to a PCI Express switch fabric, but the lay of the land will change as more processors and peripheral interfaces replace PCI interfaces with PCI Express. At this point, PCI Express is more interesting to embedded developers in general.
PCI Express brings significant benefits yet raises new problems. The 2.5-GHz signaling frequency delivers high bandwidth that can be increased incrementally by adding more bidirectional lanes. Such speed is significantly higher than even the 133 MHz for PCI-X 1.0 or 66 MHz for PCI 2.1. What that means, then, is a call for new circuit-board design rules, as well as new test equipment to analyze these higher-speed systems. Most cases don't require new board and socket technology. It's primarily a matter of looking at the details along with proper preparation. For example, mid bus pads can be designed into PCI Express circuit-board runs so that it's possible to use test probes to examine a PCI Express lane.
Nader Saleh, president and CTO for Catalyst Enterprises, indicates that designers familiar with InfiniBand and Fibre Channel have had luck with PCI Express designs because they're used to dealing with gigahertz signals. The complexity of dealing with high-speed signals and the PCI Express protocol has delayed development by six to nine months while designers move up the learning curve. Fortunately, such companies as Catalyst, Tektronix, and Agilent are delivering new test equipment like Catalyst's SPX-16E Exercister, which handles x16 links. Designers used to megahertz designs must now deal with details like clock jitter.
PCI Express, unlike Advance Switching (AS) for PCI Express, makes compatibility paramount (see "Advanced Switching It Is"). An AS system can incorporate PCI Express devices using a tunneling mechanism (see "Advanced Switching For PCI Express: The Future Looks 'Fabric' Fast," Electronic Design, June 23, 2003, p. 36). But native AS is a packet-based system. PCI Express supports PCI and PCI-X by retaining a memory-mapped architecture. This means that PCI software doesn't change even though PCI Express utilizes packets just like AS.
A good example of this transparency is when PCI Express is used to connect a pair of PCI or PCI-X systems. A board is plugged into each system. One board has a PCI Express forwarding bridge, and the other has a reverse bridge. The boards are joined by a PCI Express link that uses fewer lines and can go further than a PCI/PCI-X bus. No software needs to be changed to perform transfers across this link.
IT'S IN THE SWITCH PCI Express appears to a device or host processor as a memory interface identical to PCI and PCI-X. Internally it's a packet-switched system that encapsulates memory reads and writes into packets. Reads are translated into request packets with address and size information. The response to these packets is the data requested.Packets get routed through PCI Express switches. Data flows through the switch fabric without regard to the number of lanes joining each device. The initial connection from host to switch may be an x16 connection, while the endpoint has an x1 connection. In between, the switches may have an x4 connection.
This ability to mix and match connection bandwidth is a key advantage over PCI and PCI-X. Judicious use of switch capacity can optimize system performance and cost. Typically, a PCI Express switch will enable the grouping together of a variable number of lanes. For example, an x32 switch may allow an x16, an x8, and eight x1 connections or any similar combination.
PCI Express offers a maximum of 256 switch levels, but the practical implementation will be lower than this limit. Also, a switch typically adds 100 to 200 ns of delay. Therefore, a designer will need to determine that maximum delay between endpoints for a particular system.
Many other enhancements are included in addition to higher speeds and greater expansion options. PCI Express virtual channels allow systems to support isochronous (or time-dependent) data transfers and various quality-of-service (QoS) levels. Hot-plug support is standard, as is power management. There's also data-integrity support, although end-to-end CRC is an optional feature. Still, most PCI Express switch chips implement a full complement of features.
BRIDGING The first wave of PCI Express products incorporates a range of PCI Express bridges (Fig. 1). These bridges typically link PCI Express to PCI and PCI-X buses.NEC Electronics Corp., PLX Technology, and Texas Instruments are some of the major vendors delivering PCI Express bridge chips. Not all bridge chips are created equal, yet most are relatively flexible in terms of support and bandwidth. The table shows various bridge configurations that can be used within a PCI Express system. Some bridge chips, like PLX Technology's PEX 8104 PCI-X-to-PCI Express bridge, support all configurations.
Bridging technology is finding its way inside chips as well. Chips for interfaces like Gigabit Ethernet and Serial ATA are already available with PCI and PCI-X interfaces. Moving forward, transparent PCI Express bridge chips on these interface chips reduce the bill of materials while quickly providing a native PCI Express chip. This allows more time to transition to a native PCI Express implementation. It's analogous to incorporating a PCI Express bridge on a PCI Express board containing a PCI/PCI-X device.
IN THE CHIPS The emergence of PCI Express in the server and workstation market are helped by the separation of the processor and peripherals via a chip set that typically includes a north bridge and south bridge. These are ideal places to put PCI Express support. Keep in mind that these bridge chips differ from PCI Express bridge chips.Intel, Silicon Integrated Systems, and Via Technologies offer bridge chips that interface to a Pentium processor and PCI Express. An x16 connection is primarily directed at the video interface, while fewer PCI Express links handle peripheral interfaces. Less than 16 links is suitable, even with high-speed peripherals such as Ethernet and Serial ATA.
Via's PT890 and K8T890 north-bridge chip sets work with the VT8251 south bridge. All have PCI Express links. The north-bridge chips include an x16 and an x4 connection designed for video and peripherals, respectively. Via's 66-MHz, 16-bit Ultra V-Link connection joins the north and south bridge. This connection is sufficient to handle the x2 PCI Express link in the south bridge, which is primarily intended for low-speed PCI Express peripherals.
Silicon Integrated Systems' SiS656 north bridge links a Pentium 4 to an x16 PCI Express link. It's designed to work with the SiS965 south bridge, which contains an x1 PCI Express link along with a typical collection of built-in peripherals.
Advanced Micro Devices' HyperTransport-based Opteron and Athlon 64 will typically interface to PCI Express via a HyperTransport-to-PCI Express bridge (e.g., the SiS756 with its x16 PCI Express link and support for an SiS965 south bridge). A number of devices will interface directly to HyperTransport, such as video and high-speed communication interfaces like InfiniBand and Gigabit Ethernet. Besides, using this type of PCI Express bridge is no different than using a north or south bridge.
VIDEO THE PCI EXPRESS WAY The jump from PCI to AGP (advanced graphics processor port) allowed PC graphics to get a performance boost, but at the expense of a specialized interface. Motherboards typically offered a single AGP slot, making multiple monitor support difficult until the advent of dual monitor adapters. The x16 PCI Express slot now provides the needed performance for high-end graphics, which makes PCI Express the way to go (unless the system has HyperTransport floating around).Major video controller vendors ATI and Nvidia use PCI Express x16 as the platform for high-end graphics in lieu of the current AGP x8 standard. Nvidia possesses a range of PCI Express-based chips, starting with the GeForce PCX 5950 at the high end down to the entry-level GeForce PCX 4300.
IT'S IN THE CARDS Migration to PCI Express means lots of hardware changes. Board sizes change. Connectors change. But the software stays the same.PCI-SIG and PICMG bring a host of PCI Express adapter standards that handle x1, x4, x8, and x16 slots. The standards and sockets are designed so that a card with a lower number of lanes than the socket can still be plugged into the system. PCI Express autonegotiation will determine the card's capabilities and perform accordingly. Still, don't expect systems to have slots with the maximum number of lanes per slot. There's a cost associated with the PCI Express switch chips, so it pays to design a system to match its intended environment. An x1 slot supplies more than enough bandwidth for low-end peripheral and data-acquisition cards.
All card formats are getting a face lift. The PCI Mini Cards used in laptops have a corresponding PCI Express Mini Card standard from PCI-SIG (Fig. 2). It incorporates an x1 PCI Express connection, USB 2.0 link, and SMBus link. The latter is intended for management options. These cards can be used in embedded devices as well.
Likewise, the PCMCIA card and CardBus standards have a cousin in ExpressCard (Fig. 3). ExpressCard employs an x1 connection and pairs with a USB 2.0 and SMBus link. Sound familiar? ExpressCard comes in two form factors, both of which are smaller than a standard PCMCIA card.
PCI Mezzanine Cards (PMC) are getting a boost from the VITA-42 XMC standard for PCI Express. Cards using the standard can be employed on compatible VME and CompactPCI cards with XMC mezzanine slots.
PICMG's Advanced Mezzanine Card (AMC) standard will support a host of high-speed serial interfaces, including PCI Express, AS, and Ethernet. AMCs target AdvancedTCA carrier boards.
Additionally, the PCI-SIG is working on a Server I/O Module. This design compares favorably to the proprietary blade server solutions for enterprise-class servers. It's designed to deliver better hot-plug support plus improved power and cooling distribution. The boards fit into a compact cartridge.
Systems that take advantage of the PCI and PCI-X bus architecture like PC/104 will need to develop a new approach, because PCI Express doesn't lend itself to stacking. This is only a problem when using a separate motherboard. It will be interesting to see how or if PC/104 moves to PCI Express.
SO, WHEN? PCI Express is here now. Servers, workstations, and laptops will lead the way this year in adoption, generating plenty of demand for PCI Express chips. This includes board-level products that utilize AdvancedTCA and CompactPCI form factors.Expect PCI and PCI-X to travel on the same path as ISA. Both will remain fixtures on motherboards that will typically have x16 down to x1 PCI Express slots. The number of PCI and PXI-X slots will slowly drop to zero over the next 10 years.
Embedded designs will take a bit longer. PCI Express switch chips are readily available, but most developers would rather not contend with bridge chips. They would rather have CPUs and peripherals with PCI Express support. In the long run, this actually simplifies system designs, because it's simply a matter of choosing the appropriate processors, peripherals, and PCI Express switch chips.
On the horizon is the next generation of PCI Express. It will require new physical layers and may operate at 6.25 Gbits/s, more than twice the current standard. Backwards-compatibility will be mandatory for this standard. Look for standard ratification in 2005, though it will take a while for hardware to appear.
If you haven't looked at PCI Express yet, then it's recommended that you do so now. Don't wait for the next generation of PCI Express. Your competition isn't.
| Need More Information? | |
|
Advanced Micro Devices www.amd.com Agilent Technologies www.agilent.com ATI Technologies www.ati.com Catalyst Enterprises www.catalyst.com Express Card www.expresscard.org Intel Corp. www.intel.com NEC Electronics Corp. www.necus.com Nvidia www.nvidia.com PCISIG www.pcisig.com |
Personal Computer Memory Card International Association www.pcmcia.org PICMG www.picmg.org PLX Technology www.plxtech.com Serial ATA Working Group www.serialata.org Silicon Integrated Systems www.sis.com Tektronix www.tektronix.com Texas Instruments www.ti.com Via Technologies www.via.com.tw VITA www.vita.com |
| PCI EXPRESS SPECIFICATIONS | |
| Point-to-point, bidirectional links Switch-fabric interface Peer-to-peer data transfer Hot plug Virtual channel support QoS |
|
| Low-voltage differential signaling (LVDS) 2.5 GHz per lane 8B/10B encoding 128-Gbit/s maximum x1, x2, x4, x8, x12, x16, x32 bidirectional lanes |
|
| Forward and reverse bridging Non-transparent bridging QoS |
|
| Switch chips Bridge chips End-node integration |
|
| BRIDGING BASICS | |
| Connects PCI Express to a PCI or PCI-X device. Used to link legacy devices to a PCI Express system. | |
| Connects PCI Express to a PCI or PCI-X host processor. Typically found in a high-availability, multiple-host system. | |
| Connects a PCI Express device to a PCI or PCI-X bus. Typically found on a PCI or PCI-X card or a motherboard with a PCI/PCI-X host and a PCI Express slot. | |
