Those who are of, shall we say, a certain age will recognize the quote in the title, from the iconic 1968 sci-fi movie “2001: A Space Odyssey.” The scene involved an interchange between a human—astronaut Dave Bowman—and a machine—HAL 9000, a sentient computer—using voice interaction (Fig. 1). Unfortunately for Dave, HAL's response was “I'm sorry, Dave. I'm afraid I can't do that,” followed shortly by “Dave, this conversation can serve no purpose anymore. Goodbye.”
Although the human-machine interface (HMI) is becoming more intelligent, this conversation luckily hasn't yet taken place (so far as we know). HMI provides a means of controlling, monitoring, managing, or visualizing device processes. An industrial example is a control panel that allows an operator to interact with a machine in a graphical, visual way. With controls and information graphically displayed on the screen, the operator can use either a touchscreen or external buttons to control the machinery.
Ranging from simple segmented displays to high-resolution LCD panels, HMIs can be located on the machine, in battery-operated handheld devices, and in centralized control rooms. In process control, HMIs connect sensors, actuators, and other equipment on the factory floor to supervisory and control systems.
HMI Overview
The HMI provides interaction between the operator and the process or equipment being controlled, so it requires two-way communication that goes well beyond an on-off switch and a light. A well-designed HMI delivers information that helps users with their decision-making processes.
An HMI system’s effectiveness is determined by its processing power, ability to render complex images, fast response time to user inputs, and flexibility in handling various levels of operator interactions. Current approaches rely heavily on GUI technology, which coincidently was first demonstrated in 1968, the year of “2001.”
HMI Functional Bocks
Figure 2 shows an HMI ecosystem, which contains multiple functional blocks: an applications processor; power management; wired and wireless communications; and a variety of systems to interact with an operator, such as knobs, switches, a monitor (perhaps with touchscreen capability), and so on.
Processor Selection
Knowing the performance requirements is important when choosing an embedded processor for HMI applications. Particular care should be taken to include a realistic estimate of computationally demanding tasks like video processing, and display functions such as 2D or 3D graphics. A scalable family of devices built around a common core can maximize code reuse across applications and shorten development time.
For example, the Sitara AM57x microprocessor family can cover the range of HMI applications. The family includes devices with single or dual ARM Cortex-A15 cores running at 1.5 GHz, with performance as high as 10,500 Dhrystone MIPS. There's also a large peripheral set, including integrated C66x DSPs, multiple communication options, and a dedicated video-acceleration engine.
Input Devices
An industrial environment is likely to contain a broad range of chemicals and pollutants, including water, dirt, oil, dust, solvents, and acids. When designing input controls (e.g., switches and knobs), it's important to provide protection against these contaminants, which can otherwise prove lethal to electronic equipment.
Although HMI systems have long employed protection methods such as rubber boots on switches, the constant movement over time renders the seals prone to failure. The mechanical switches are cheap and provide tactile feedback, but are subject to wearout after extended use. Manufacturers are therefore developing “smart” switches that provide a similar user experience without mechanical motion.
Haptic Systems
One burgeoning field involves the replacement of mechanical switches with haptic technology. Haptics is the science of applying touch (tactile) sensation and control to interact with computer applications. A haptic device provides a sense of touch and gives user feedback.
The reference design shown in Figure 3 uses inductive sensing mounted on a PCB underneath the switching surface. Since it can detect extremely small deflections (on the order of microns), the buttons can be simple etched outlines or raised bumps on a continuous sheet of metal. The assembly can be effectively sealed against contaminants and grounded to protect against electromagnetic interference (EMI).
Haptic designs have other advantages, too. The haptic feedback can be customized to a desired profile; button sensitivity is programmable; and operation is possible in a wide variety of conditions, including harsh environments, under water, or where the operator is wearing gloves.
Haptic systems are used in a wide variety of HMI applications, including process control, building automation, mobile devices, point-of-sale (POS) equipment, and home appliances.
Touch-Free Capacitive Sensing
In certain hazardous applications, operators may be required to interact with explosion-proof controllers, which are housed in sealed metallic enclosures with thick glass windows to display local readout and programming functions. With traditional HMI technology, the operator must open the enclosure to change a setting or touch a switch. This may call for special procedures and even require shutting down a process.
Technology can come to the rescue in the form of capacitive switching. A capacitive switch is made by forming an isolated sensor plate out of a conductive area on a printed circuit board (PCB) and applying a charge to it. A capacitor will form whenever a grounded, conductive object, or an object with a dielectric constant different from air, comes close to the sensor plate.
Since the human body is essentially at ground potential, moving a finger close to the charged plate acts as the ground plate. A circuit that detects this capacitance can be used to construct a contactless capacitive switch. Capacitive-sense technology enables “touch-through-glass” operation and allows the operator to interact with a controller without having to open the enclosure.
Figure 4 shows an example of such a sealed system. The numbers are simply decals to indicate where to place the finger; the switches are on the PCB, which is mounted one or two millimeters from the glass surface. The capacitive switch can detect a touch through glass up to 10 mm thick. Like inductive sensing, it also works if the operator is wearing gloves.
The heart of a capacitive-sensing system can be a single specialized microcontroller such as the MSP430FR2633, a 16-bit RISC device that includes a capacitive-sensing interface with up to 64 channels.
Capacitive Proximity and Gesture Sensing
It’s possible to expand the technology behind capacitive switching beyond detecting an on/off event. Since the capacitance increases as the finger approaches the sensor, this can be used to detect distance. Even though the variation is nonlinear, there’s a detectable difference that’s sufficient for proximity detection.
Proximity sensing can play a valuable role in reducing power consumption. The system can wake up the HMI when it detects the presence of a user, allowing the power-hungry components to remain in sleep mode until they’re needed. When used on a display screen, this can increase its lifetime considerably, since LCDs and backlights lose contrast and brightness as they age.
A further refinement of capacitive sensing is the gesture sensor, which employs multiple independent proximity sensors for up/down and left/right detection. As a finger moves over the sensors, it changes the capacitance of all four sensors. A multichannel detector reads all four values; then software can calculate speed and direction based on the difference in readings. For more information on this topic, see this article.
Displays
HMI displays can be divided into categories, depending on the complexity of the industrial process being monitored. Of course, this is a moving target—the WHUXGA format ranges up to 7680 x 4800. The table shows typical graphics requirements for differing HMI form factors.
Touchscreens combine both input and output functions, and touchscreen controllers provide a low-power, high-performance solution. The controllers, which offer programmable 8- or 12-bit resolution, measure touch pressure. They can also carry out preprocessing of touchscreen measurement data, reducing the load on the host processor.
Touchscreens using capacitive-switch technology provide higher quality and a better experience than resistive displays. A Sitara processor such as the AM437x, with its ARM Cortex-A9 core, features an integrated display subsystem (DSS) for easy connection to an LCD touchscreen; a reference design is available here.
High-end screens are able to display a huge amount of detailed information, but that doesn't mean they should. Too much information is as bad as not enough information. Proper HMI design aims to minimize the cognitive load of the operator—the work needed to find data, understand it, forsee potential problems, and take action.
Best practices in HMI design begin with an intimate understanding of the tasks to be performed (Fig. 5). Typical display guidelines include:
• Arrange information into logical groups.
• Keep the display simple by eliminating unnecessary clutter.
• Use color, motion, sound, etc., sparingly.
• Be consistent in the use of symbols, naming conventions, placement of navigation button, etc.
• Use analog representations where possible. For example, an analog clock face, while seeming to be less precise, displays information in context (“how long until quitting time?”) differently than a digital clock.
To reduce cognitive load, touchscreens are often combined with haptic systems to provide vibratory feedback when a button on the touchscreen is tapped. Research has demonstrated that combining touchscreens with haptics or tactile feedback can reduce user input errors by 20%, increase input speed, and lower cognitive load by 40%.
Voice Interaction: The Next Step In HMI?
What of voice-based HMI, as used by the HAL 9000 at the beginning of this article?
It involves two elements: voice recognition and voice synthesis. Voice recognition must be robust, be capable of real-time analysis, and have low power to enable always-on listening. On the hardware side, that usually requires an audio codec feeding a powerful, low-power DSP, such as TI's C55x family, to clean up the raw audio for use by the voice-recognition engine.
On the output side, options include a voice-synthesis engine combined with a high-resolution display.
In consumer HMI, such systems are becoming commonplace in applications like Apple's Siri, Google's Assistant.ai, and numerous telephone banking systems. Voice control is also making its way into automobiles, helping to reduce fatalities. Challenges that remain include multiple languages, accented speech, and operation in high levels of ambient noise, a particular concern for industrial applications.
A few vendors are promoting voice-activated control for selected applications. However, much work needs to be done to make it robust enough for industrial applications, where a single misunderstood command could be disastrous.
Conclusion
Designing an HMI system for an industrial application is a complex endeavor involving hardware design, software design, ergonomics, cognitive psychology, and more. On the hardware side, new technology continues to increase reliability and reduce risk, ultimately leading to reduced operator workload.
Looking for parts? Go to SourceESB.