This file type includes high resolution graphics and schematics.
Everything you thought you knew about powering data centers is passé. The future of the grid is spelled IEC 61850, and many pieces of the infrastructure are available. And, greater efficiency and innovation are still on the way for the common ac-dc and dc-dc digital power supply (see “The Future Of Power Efficiency,” and “Innovation In Digital Power Supplies”). This news comes from a data center architect, the largest maker of hardware for Internet protocol (IP) communications, and executives from the largest European power supply makers.
Revolution In The Data Center
At a recent Silicon Valley IEEE Power Electronics Society (PES) meeting, MegaWatt Consulting president and CEO K.C. Mares briefed members on the latest trends in the data center and their effects on power distribution and conversion. Mares also is the CTO of the Unique Infrastructure Group, the company developing the Reno Technology Park, which will be the largest dedicated data center campus in North America with onsite renewable energy generation.
At MegaWatt, his services included datacenter site selection, design, and efficiency projects for Facebook, Google, Yahoo, Equinix, the National Center for Atmospheric Research, Lawrence Berkeley National Laboratory, the U.S. Department of Energy, and others. Overall, he has played a key part in the design of data centers worth cumulatively more than $10 billion.
Mares described a data center that’s generations away from aisles of cabinets full of blade servers, sitting on raised computer floors powered by multiple isolated and nonisolated voltage step-downstages from “front ends” to “bricks” and “POLs,” all surrounded by temperature-control and humidity-control systems that maintain a shirt-sleeve environment for the humans who work there.
About the only thing that remains from this rapidly passing image of the data center is the power consumption, and that’s going up due to growing total capacity. Several factors are influencing this evolution, Mares said. These include solid-state memory replacing disc storage, processors with ARM cores replacing Intel devices for the heavy lifting, and modularization of server arrays. Furthermore, more subtle in its effect is the realization that the cloud provides its own backup, which results in less demand for “six-nines” reliability.
At the meeting, Mares held up his iPhone and said, “Basically, this is a server.” In terms of processing power and storage, the smart phone is the equivalent of a typical blade. The amount of heat it generates, of course, is somewhat smaller. Yet regardless of how much heat it generates, its environmental heat tolerance is far above what a human worker can long tolerate. There is no reason not to pack a large number of iPhone-size servers into boxes of far less internal volume than today’s cabinets and provide only enough internal cooling to allow the internal temperature to rise no higher than 140°F. Full cooling, he said, is only necessary in the battery room.
There only was one objection from that room full of power engineers when Mares said that humidity control wasn’t necessary either. “How can you avoid electrostatic discharge (ESD) failures without some humidity?” one engineer asked. “Shoe straps,” Mares replied. There was more discussion, but the point was that 100% enforcement of anti-discharge shoe strap discipline was considerably less expensive than installing humidity control.
Finally, Mares said that all of these changes will have an effect on uptime, but the redundancy of data in the cloud more than compensates for going from 0.999999 reliability to 0.99999 reliability at any one data center.
The Cisco Grid
Ignoring “smart meter” controversies for the most part, Cisco notes that utilities are already investing in communications networks that improve situational awareness and facilitate control, automation, and system integration. The objective is to “smooth out” peak load demand, minimize the need for “spinning reserves,” and avoid building new plants and putting in more conductors.
Cisco reckons that four characteristics combine to make a grid “smart,” and these elements reside in the layer of digital superstructure that can be added to the traditional grid’s analog infrastructure: observability, configurability, automation, and integration. Observability means providing real-time awareness of the grid state; configurability means being able to drive grid elements to new states; automation means doing this largely without human intervention; and, integration means system-wide interconnectivity.
Obviously, this kind of scheme isn’t implemented all at once. Cisco says the utilities should start with automating substations. In a way, substation automation has been happening ever since the introduction of the microprocessor relay. The critical need now is to extend secure, scalable communications infrastructure beyond individual components. This will allow new applications for control and automation, such as condition-based and predictive maintenance programs.
Beyond the substations, the task is to maintain a variable energy supply in the wake of growing and uneven energy demands. Monitoring is provided by operational effectiveness metrics and system performance indices: traditionally the System Average Interruption Duration Index (SAIDI) and System Average Interruption Frequency Index (SAIFI).
At a finer granularity, to manage outages, utilities deploy protection relays that provide for fault location identification and restoration. These relays greatly depend on deterministic communications for precise timing and fast network convergence to deal with voltage or phase imbalances. As for the infrastructure that communications and data will run on, it’s already common for utilities to deploy their own dark fiber between sites for peer-to-peer protection communications.
In fact, with the advent of IEC 61850 standards, station bus communications have already begun migrating to Ethernet for intra-substation protection, which facilitates interoperability and scalable network design, as well as reduces operating expenses. Another common communication need at the substation level is identification of energized lines, especially ones that are down and may be a public safety hazard.
All this requires communication networks that can maximize situational awareness of grid conditions at all times. Remote workers need data and voice communications to report conditions to control centers and to first-responders. Substation technicians at remote sites require redundant, survivable communication paths.
Backing up these communications concepts, Cisco offers its Connected Grid portfolio of products and solutions designed specifically for a Smart Grid network. These products include the Cisco 2010 Connected Grid Router (CGR 2010) and the Cisco 2520 Connected Grid Switch (CGS 2520). These platforms are optimized for use in power substations and meet substation compliance standards including IEEE 1613 and IEC 61850-3 (Fig. 1).
1. Cisco sees design opportunities across many network levels using IP. Note that there isn’t necessarily any utilization of the tactically vulnerable conventional Internet in the critical functions at the left side of the diagram.
IEC 61850
While Cisco’s vision concerns itself with the communications infrastructure of the evolving Smart Grid, much of the opportunity for new products lies in the hardware for controlling transmission and distribution. In an effort that started in the mid-1990s, this is being standardized as IEC 61850, which standardized the design of automated electrical substations. It’s part of the IEC Technical Committee 57’s reference architecture for electric power systems. Its abstract data models can be mapped to various data protocols that run over TCP/IP networks and substation local-area networks (LANs) using high-speed switched Ethernet.
IEC 61850 is intimidating. The complete set of documents runs to 1400 pages. Electrical power experts from 22 countries wrote it, and it assumes intimate knowledge of how the components of an electrical power distribution network work. Happily, there is a document that makes it more accessible to electronic product designers. Released in 2009, “Understanding and Simulating the IEC 61850 Standard” by Yingli Liang and Roy H. Campbell of the Computer Science Department of the University of Illinois at Urbana Champaign is a model of clarity. (And it runs only 12 pages.)1
IEC 61850 And IEDs
Intelligent electronic devices, unfortunately abbreviated IEDs, are at the core of the standard. Essentially, IED refers to microprocessor-based controllers for power system equipment that can receive or send data or control commands from or to an external source. While that makes an IED basically a PC, probably running Linux, it may also have some dedicated logic for domain-specific processing.
IEDs can be classified by their functions, such as relay devices, circuit-breaker controllers, recloser controllers, or voltage regulators. In a substation, the internal network is connected to the outside wide area network via a secure gateway (Fig. 2).
2. IEC 61850 envisions a substation network connected to the outside wide area network via a secure gateway. Outside remote operators and control centers can use the abstract communication service interface (ACSI) to query and control IEDs in the substation. One or more substation buses connect all the IEDs inside a substation.
The IEC 61850 standard defines an abstract communication service interface (ACSI) to be used by remote operators and control centers to query and control devices. One or more substation buses, in the form of a medium bandwidth Ethernet network, connect all the IEDs inside the substation.
The substation bus carries all ACSI requests and responses as well as generic substation events messages (GSE). A separate, high-bandwidth Ethernet process bus handles communication inside each bay and connects the IEDs to any dumb devices. A substation would typically have a single global substation bus and multiple process buses, one for each bay.
Most interactions inside a substation automation system are data gathering and setting, data monitoring, or reporting and event logging. To accomplish all that, the standard defines a communication structure (Fig. 3). There are four communication profiles in addition to ACSI: the generic object oriented substation event profile (GOOSE), the generic substation status event (GSSE), the sampled measured value multicast profile (SMV), and the time synchronization profile.
3. The IEC standard defines a communication structure with four communication profiles: ACSI, the generic object-oriented substation event (GOOSE), the generic substation status event (GSSE), and the sampled measured value multicast profile (SMV).
As we have seen, ACSI services enable client-server interaction between applications and servers. GOOSE provides a fast way to exchange data on the substation bus. GSSE provides a method for substation-level status exchange.
Applications request all ACSI services, and servers respond to them. In the IEC 61850 data model, the server is the common point that links physical devices and logical objects. Usually, an IED hosts one instance of a server, though it can run more.
Each server has one or more access points, which are the logical representation of a network interface controller, and each server hosts several files or logical devices. Clients can manipulate files in the server as if they were talking to an FTP server.
ACSI services include querying an object set, getting or setting data values, controlling system objects, report manipulation, log manipulation, and other services, including file upload and download.
This file type includes high resolution graphics and schematics.