For modern high-performance electronic systems, it's common practice to break the system down into pluggable, line-replaceable modules. This gives system designers flexibility to connect subsystems with a variety of functions. It additionally allows for easy repair and upgradability.
Backplanes often are used to electrically interconnect modules with each other via a multilayer printed wiring board (PWB). The most frequent packaging implementation of the backplane is where modules are plugged in from the one side and are oriented parallel to each other (Fig. 1, left).
Other implementations use midplanes, which are similar to backplanes except the modules are inserted into both sides of the board rather than one side. For example, midplanes interconnect modules (such as switch cards, line cards, processor cards, or disk arrays) on the front side of the midplane to modules on the back side. A midplane architecture may offer designers more flexibility to optimize their designs for many variables, from minimizing trace length to user serviceability.
Specialized connectors, common to backplane and midplane implementations, are used to provide the blindmate separable interface. Connectors of this type must permit a routing channel in a direction normal to the card face to both escape the connector's footprint and allow for through routing channels.
As signal densities increase, routing these systems will boost the number of signal layers and result in thicker, higher-cost circuit boards. Thicker boards also adversely affect electrical performance by increasing the via stub, impedance, and footprint crosstalk. Front and rear cards plug into the midplanes with various orientations. One specific midplane implementation has front and rear cards oriented orthogonally with respect to each other (Fig. 1, right).
Orthogonal system architectures can bring significant advantages for applications requiring the need for future speed upgrades using the same midplane. Because most of the trace, and its associated limitation in electrical performance, is moved from the midplane to the daughtercard, upgrades are simplified.
Greater interconnect performance is accomplished by just upgrading the modules. In addition, because the midplane has minimal trace length, standard FR-4 boards can be used instead of more exotic high-performance materials. All of these factors will result in lower system cost. Another sometimes overlooked benefit is that this midplane construction doesn't determine the physical-layer network topology, enabling the system to be reconfigured to meet the customer's future performance needs.
By minimizing traces routed through connector fields across the midplane, the overall skew and impedance control within the differential pair is more manageable. The resulting system is a well-controlled link in terms of impedance, skew, and attenuation management, with significant cost advantages over traditional backplane or midplane designs.
A few obstacles have prevented the potential mainstream adoption of orthogonal midplane architectures, though—the thermal challenges in aircooled systems and the nonlinear insertion loss seen with short traces and standard backplane connectors. If it isn't addressed, the nonlinear insertion loss will diminish the signal-integrity benefits. But with careful attention to choosing the correct connector, the advantages of orthogonal architectures can be realized.
STANDARD BACKPLANE CONNECTORS
The electrical performance of a backplane interconnect is determined by linking together all of the individual components in the channel. In an orthogonal architecture, it's possible to improve the footprint and connector by using a differential shared plated through-hole (DSPTH) configuration.
In addition, designers can improve transmission effects by removing the plated through-hole (PTH) stubs. Also, optimizing vias for the increased pitch between signal pairs dramatically lowers the crosstalk values in the footprints. Still, how the individual elements on the link interact isn't obvious.
In an orthogonal architecture using a traditional connector system, two connectors are usually placed on opposite sides of the midplane with a small amount of trace interconnecting them. Figure 2 illustrates the routing for a typical midplane in an orthogonal system.
The PTH normally has a poor impedance match to the connectors and midplane traces. The close proximity of these two elements means the reflections (or return loss problems) will interact. The reflected energy from each impedance mismatch won't dissipate before reflecting off the next impedance mismatch. The routing diagram shows the proximity of the impedance mismatch on the midplane. The daughtercard footprints/traces, connectors, and midplane traces all have different impedance structures, resulting in a number of reflections.
Figure 3 shows a differential time-domain-reflectometer (TDR) plot, with each element of the link labeled, and a frequency-domain measurement plot of a traditional orthogonal interconnect without DSPTH technology. Each of the link's elements are labeled. The PTH, or via, on the daughtercard and the backplane each has approximately 50-mil stubs, and both use an 18-mil finished hole diameter. The impedance of the two connectors is approximately 105 Ω the daughtercard trace impedance is centered at 100 Ω and backplane-etch is approximately 95 Ω Many available publications discuss the resonance caused by a via stub. From this work and other simulations, it can be assumed that the resonance of a 0.05-in. via stub will be above 10 GHz, most likely in the region of 15 GHz. Yet when this link is analyzed with a vector network analyzer (VNA), there's increased insertion loss at frequencies as low as 2 GHz. This is due to the accumulation of impedance discontinuities.
As the via stub grows (and the return loss increases), the interaction between the different impedance mismatches results in a nonlinear channel. The frequency-domain measurements of a similar channel are displayed, with the stub length increased to 0.120 in. and the backplane connectors interconnected with 2.5 in. of trace.
The dramatic change in the insertion loss between 1.8 and 2.4 GHz for the 2.5-in. trace should be noted. In a span of less than 600 MHz, the insertion loss increases by 4 dB. It's difficult to compensate for abrupt changes in the insertion loss of the channel.
Channels with linear loss are easier to recover than nonlinear channels. Higher-order equalization (decision feedback and multitap feed forward) must be used to recover nonlinear channels.
As the complexity of silicon devices increases (to implement these equalization techniques), transceiver power consumption also elevates. Channel recovery in digital systems is never optimal due to the limited resolution of filter coefficients and the coarse time steps that they're spaced on.
The impedance discontinuity at the compliant pin via results in a high-loss, nonlinear insertion loss even for relatively short midplane traces. New interconnects are now on the market, such as Amphenol TCS's Crossbow Matrix connector, that offer innovative solutions to orthogonal implementations.
An interconnect system that can connect front and rear differential pairs through a common via in the midplane eliminates the need for short jumper traces and two connector footprints. The common via, or DSPTH technology, eliminates via stubs as signals pass directly through the midplane for all signals. Figure 4 shows a TDR plot and frequency-domain plot of an orthogonal interconnect with DSPTH technology.
Looking at the performance of the interconnect from the daughtercard through the connector to the next daughtercard, the impedance control is much better. Compared to the insertion loss shown in Figure 3, this plot is linear with minimal resonances.
Stub elimination is a well-documented electrical benefit, but the shared hole also saves the cost associated with backdrilling for stub removal. A further cost benefit is often achieved by the signal-layer count reduction in the midplane due to the elimination of midplane-connector routing.
Laminate and conductor loss associated with the short bridging traces is also eliminated. Further advantage is gained by orienting the rear-side daughtercard edge so any in pair skew induced by the right angle portion of the connectors is cancelled, rather than doubled.
It's possible to have the same connector share vias in the midplane when implemented orthogonally. The differential pair pitch in the X axis must equal the pitch in the Y axis, and the number of pairs per column must equal the number per row.
Designers should look for connector features that eliminate the need to compensate for skew and its negative electrical impacts. Connectors are available with crosstalk at less than 1% at 50 ps and 20 db below transmission across the entire link (past 20 GHz).
It's best to observe the individual component improvement mentioned above at the link level. Figure 5 shows the simulated insertion loss results of orthogonal midplane interconnect using standard and optimized connectors.
The blue line represents the transmission of a traditional connector in a midplane using short (1-in.) traces to connect connector vias from both sides of the midplane. While the trace is short and there's only a small amount of attenuation, the impedance discontinuities for each via lead to a highly nonlinear result. On the other hand, the DSPTH connector has a relatively linear response resulting in a >2-dB advantage at 5 GHz and >10 dB at 7 GHz.
THE COOLING CHALLENGE
While the enhanced performance and cost benefits of orthogonal architectures are demonstrated, thermal issues related to cooling orthogonal systems present another challenge. Cards on the front of the midplane require a separate airflow path from cards on the back.
Moreover, horizontally oriented cards require side-to-side cooling, which can be problematic in enclosed racks. Cooling solutions for a chassis depend on the rack level requirements, which in part are defined by how the room in which the racks that hold the chassis is laid out. Designers can consider two primary cooling architectures: side-to-side and front-to-back.
Side-to-side airflow is typical for wiring closet applications in which equipment is backed up against or mounted to a wall. The need for rear access is typically discouraged and thus difficult for an orthogonal architecture that requires rear access. On the upside, if the room isn't densely packed, there can be more freedom to accommodate two different airflow directions—in part because equipment in the room could be positioned away from obstructions.
Front-to-back airflow architectures are typically required in data rooms where racks are placed next to each other and the room is organized so distribution of cooling air becomes more effective (i.e., hot and cold aisles). Generally, rear access is possible but limited. This type of setup can cause problems with horizontally aligned cards that have a side-to-side airflow path, unless a sufficient amount of clearance is provided on the side of a chassis. There are several options for front-to-back cooling of a midplane system.
- Through the midplane: Orthogonal architectures using connectors with shared plated through holes reduce or even eliminate the routed areas between slots. This allows for greater perforation of the midplane to improve air flow through the midplane. This extra space can improve through-the-midplane cooling. However, this type of cooling is usually limited to lower power and power density levels, in part because of the challenges in packaging air movers and dealing with the preheated air from the upstream cards. For through-the-midplane cooling, air flow must pass into the front panel of front-side mounted cards, which can be a challenge for some types of cards, such as those with front-panel ports.
- Around the midplane: For vertically arranged front-side cards, an air-only cooling solution can be considered, where a series of air movers is packaged much like a front-side line card slot—either integrated in the chassis or alongside it. Relatively new high-pressure 1RU-type axial fans are well-suited and can pass the air around the side of the midplane. The air then makes a 90° turn behind the midplane and flows across the rear-mounted cards, again assumed to be horizontal in this case. After the air passes across the cards, it's discharged out the rear at the far end of each card.
- Away from midplane: This method transports the heat to another rack or room location where the heat can be dissipated. One possibility is a liquid cooler (Fig. 6) that requires 3RU of rack space to dissipate about 500 to 750 W of heat using a dielectric fluid. A non-refrigerated system dissipates the heat using a liquid-to-air heat exchanger arranged for front-to-back airflow. Liquid cooling in this fashion also provides a means to scale the thermal system without essentially affecting the electronics packaging.
As OEMs look to squeeze more bandwidth out of traditional, lower-cost systems, many are turning to orthogonal architectures. Orthogonal system architectures can have significant advantages in overall channel performance, offer cost advantages over traditional backplane or midplane designs, and provide exceptional headroom for future performance upgrades.
As with any complex electrical packaging challenge, many design factors must be considered and carefully balanced in weighing traditional and advanced solutions to orthogonal packaging challenges. The introduction of Crossbow Matrix is one example of the importance of an architecture-based solution for today's system designers.
Optimizing component technology may result in a sub-optimized system. Designers should seek solutions that, whether by incorporating existing components or innovating new component technologies, will create an optimized system solution.