With literally billions of ports, Ethernet lays claim as the most ubiquitous networking technology worldwide. Ethernet started out simply as a popular local-area network (LAN) technology, and it slowly developed into a networking method for metro-area networks (MANs). Now, with 40- and 100-Gbit systems on the horizon, Ethernet will no doubt quickly move into wide-area networks (WANs). At the recent Ethernet Technology Summit in San Jose, I got a first-hand update of what’s happening in the 40G/100G Ethernet space.
A combination of factors is driving the need for 40G/100G networks, spearheaded by the steady increase in Internet traffic. Cisco Systems projects a hundred-fold increase in Internet traffic from 2008 to 2020, which equates to 55.1 exabytes/month. (An exabyte is 1018, or 1 billion Gbytes.) The leading pressure point is video, not only from new Internet Protocol TV (IPTV) in consumer homes, but also video on demand (VOD), videoconferencing and webinars, and cellular video traffic from YouTube, Facebook, and other sites. This ultimately puts increased stress on data centers and all of the connecting networks.
Higher data rates are also needed for virtualization and cloud computing, low-latency services, and more 10GE customer connections in data centers. Not only are 1GE lines being aggregated into 10GE lines, but now 10GE’s increased usage is pushing the need to aggregate 10GE traffic.
With the upward climb in traffic among all sectors, carriers are desperate for a solution. Coming to the rescue are 40G/100G systems. In fact, AT&T, Verizon, and other carriers have already implemented some proprietary 40G and 100G solutions. Still, they’re waiting on some standard methods that will drive down the cost of this expensive optical technology.
Before extolling the future of 40G/100G, let’s make it clear that Ethernet isn’t all-encompassing. For the Internet backbone, long-haul networks that interconnect all of the WANs and MANs prefer the ITU’s Optical Transport Network (OTN) standard. Designated ITU G.709, it uses dual polarization and differential quadrature phase-shift keying (DQPSK) modulation to achieve 100 Gbits/s over long distance using single-mode fiber. Over time, the 40G/100G Ethernet networks will tie into these long-haul networks. Look for 40G/100G Ethernet in data centers for data aggregation, as well as in new and upgraded WANs and MANs.
Moreover, 40G/100G will also replace legacy Sonet/SDH, which has an upper limit of 39.813 Gbits/s and was originally designed for voice traffic anyway. It’s not packet-based, making it less desirable in this TCP/IP-based packet world.
The IEEE standard for 40G/100G, 802.3ba (see the table), offers a wide range of different, mostly optical versions. Backplanes, copper, and fiber physical-layer (PHY) media are covered, each usually featuring multiple lanes of 10 or 25 Gbits/s (although a single serial version of 40 Gbits/s is being developed, too). Most of these leverage the current 10-Gbit/s technology.
Some versions use multiple fibers, while others employ coarse wavelength division multiplexing (CWDM). For the latter, each lane or channel uses a different wavelength (λ) of light on a single fiber.
Just as important as the media are the connectors used for attachment. The CFP module, for instance, is a 148-pin electrical connector that has 12 optical I/O ports. Then there’s the quad small form-factor pluggable (QSFP) module with four optical ports, which is used for 40G MMF and single-mode fiber (SMF). And, the CXP module handles 100G over multi-mode fiber (MMF) and offers two sets of 12 optical I/O ports. Future versions, CFP2 and QSFP2, are in the works for 100G.
We can also look for more photonic integrated circuits (PICs). All of the optical components reside on a single chip, including an array waveguide (AWG) that does the multiple wavelength filtering. I’m amazed by the copper versions. Do they defy physics? With multiple cables using new wires, insulation, packaging, and signal processing, 40G and 100G are possible.
Some of these new PHYs should finally be ratified this June, with products following shortly thereafter. The consensus at the conference was that despite the forthcoming products, 40G/100G’s adoption will be slow, likely over a 10-year period, as demand increases and prices come down.
Other hot topics at the conference were carrier Ethernet and Fibre Channel over Ethernet (FCoE). Carrier Ethernet, developed by the Metro Ethernet Foundation, makes Ethernet more suitable for critical enterprise applications (e.g., quality of service). Thus, Sonet/SDH telecom carriers can provide Ethernet services to customers. Overall, it’s easier to monitor, control, manage, and administer both MAN and WAN versions of Ethernet.
FCoE makes it possible to carry the popular Fibre Channel (FC) storage-area network (SAN) protocol over plain-old Ethernet. For years, FC has been the most popular SAN interconnection network. It has its own standards (T11) and support organizations. However, with all services using Ethernet, it’s more economical to support only one network on the same cable rather than separate data and storage networks.
FCoE simply encapsulates the FC protocol in Ethernet frames. The figure shows a converged network adapter (CNA) that operates to 10 Gbits/s. It handles both regular LAN Ethernet traffic as well as the Fibre Channel connection to a SAN on the same cables.
Ethernet can carry almost any other networking protocol as well. It already is the primary carrier of TCP/IP, and most popular industrial networking protocols (ModBus, DeviceNet, etc.) are now being routinely carried on hardened Ethernet LANs in factories and plants. A popular phrase at the conference was “Anything over Ethernet (AoE)”. Look for plenty of examples of that to come.