For the PDF version of this article, click here.
The vehicle interior continues to evolve with added capabilities for information and entertainment; i.e., infotainment. While these embedded wireless communication, navigation, enhanced audio and visual entertainment systems provide additional useful functionality in the vehicle, they also contribute to higher driver workload. This workload increases even more as drivers bring mobile electronics devices into the vehicle to use as part of the vehicle system. As more capabilities and features are introduced in the coming years, it is even more important for automakers and their suppliers to provide user-friendly environments that ease the workload on the driver to interact with these new technologies and features.
The mantras, “Hands on the wheel, eyes on the road” and “Driving is the primary task,” are the guiding principles for providing optimal human-machine interface (HMI) to reduce the driver's cognitive workload. To accompany the ongoing increase of in-vehicle infotainment and mobile device features, voice recognition and wireless connectivity technologies such as Bluetooth provide opportunities to support workload-optimized HMI designs for certain tasks.
Human-system integration (HSI) is the key to providing these user-friendly HMI systems. Engineering activities such as: human factors, systems, software and hardware along with industrial design and market research must be coordinated to provide a human-centered design experience resulting in an enjoyable HMI for the driver and other passengers. Human factors engineering addresses the physical and cognitive workload on the driver while designing interactivity that is intuitive for a broad range of users. Systems engineering provides the vehicle capabilities to support the control of in-vehicle devices. Software engineering provides simulation models to be used for product development and will generate auto-coding for the final product. Industrial design provides a design synthesis giving the vehicle a unique experience. Market research provides end-customer wants for feature usage and defines new trends.
HMI DEVELOPMENT
In the initial design concept stage, industrial designers and human factors engineers work together to design an optimized control solution for the user task. The design is input to a computer-aided design model for further analysis and optimization.
After the user controls have been defined, the interactivity is designed and modeled. These models are user tested with the vehicle environment in mind. This process may take a few iterations depending on the feedback analysis. Primary importance is placed on the high usage tasks and tasks that trap the user in a long series of tasks. Any task that traps the user must be locked out or broken up into smaller tasks that force the user to take breaks.
Development of in-vehicle HMI's to support the implementation of infotainment features presents several challenges to automotive manufacturers and suppliers. The increase in features requiring driver interaction by inputting information or selecting options can require significant driver attention with manual controls (touch screens, buttons and switches). Additionally, the consumer desire to bring portable electronics into the vehicle puts a further burden and complexity on providing a convenient and safe vehicle HMI. As new systems are developed with a reduction in the number of controls, giving the visual impression of simplicity, it may add to the user's workload because the controls need more physical interaction to complete the same task.
These challenges can be addressed with the proper deployment of voice recognition and vehicle wireless connectivity systems in conjunction with the in-vehicle HMI. The wireless connection systems can provide a gateway to portable devices through technologies such as Bluetooth to enable transfer of information (audio, data) as well as hands-free control of the portable device (song selection, number dialing). Voice-recognition technology can provide the hands-free control of these features as well as a convenient methodology for entering information (phone numbers, destinations). It can also be used as a shortcut for a complex interactive task, reducing the interaction with controls.
VOICE RECOGNITION MARKET ACCEPTANCE
To investigate the consumer interest in voice recognition, Visteon recently completed a Global Infotainment Market Research study. In this study several products were tested with and without voice recognition and the quantitative results indicated a strong preference for a voice-enabled HMI in the U.S. market. For example, as shown in Figure 2, 73% of the respondents preferred a voice-enabled HMI for a wireless phone module over scrolling through a phone's address book or simply answering an incoming call.
Also qualitatively, respondents indicated that voice control was an essential element in their future consideration of infotainment products. They said voice recognition complimented the traditional button/display-based HMI, and simplified the entry of complex information such as phone numbers and destinations.
AUTOMOTIVE ENVIRONMENT VOICE RECOGNITION
Voice is a key mode of input for in-vehicle HMI. Voice HMI in currently deployed applications is largely compromised or directed by the limitations of the voice-recognition engine. Recognizing this, Visteon has invested heavily in core voice-recognition technology to design voice HMI with usability as the main focus including a voice-recognition engine that supports the HMI well. Speech as a mode of input/output has its limitations too, for example in situations when the car is fully occupied with passengers who are engaged in a lively conversation.
The front-end signal processing provides the gain and spectral normalization of the audio input, the extraction of the users verbal command information from the noisy vehicle environment, and the spectrum analysis of the input for speaker-independent recognition.
The core recognizer provides the statistical analysis of the information to determine the match with possible grammar available for the given applications feature set. The core recognizer includes vehicle-based acoustical models (including support for Lombard effect) to support high-recognition performance in noisy vehicle environments.
The application provides the interpretation of the recognizer results and transfers this into network messages to control or transfer information based on the user voice command. The application also manages the feedback (both audio and visual) to the user based on the protocol.
VOICE-RECOGNITION HMI DEVELOPMENT
Voice-recognition grammar (the commands issued by the user to control features) and grammar structure are developed to be intuitive to the end consumer in a given market. The initial part of grammar design is the identification of words and phrases preferred by the market. Localization studies to identify them involve conducting interviews and collecting field data. Once initial command grammars and structures are developed, a simulation model is developed quickly and usability tests are conducted. These tests cover performance, perceived ease of use, preference ratings and workload assessment.
VOICE HMI BENEFITS
Voice-recognition HMI designs provide several benefits in reducing the workload of a driver to control infotainment systems. Voice recognition mitigates the driver distraction while controlling an infotainment device by eliminating the need for the driver to take his or her eyes off the road. Voice recognition can also reduce the task execution time for various controls. (Visteon HMI studies have shown a significant reduction in complex task times associated with using voice recognition vs. standard manual controls)
Voice HMI deployments are also able to allow the automotive OEM to deploy geographically tailored systems by language to meet unique HMI behaviors. An example of this is the way of speaking phone numbers; in the United States it is spoken as a sequence of single digits but in France it is commonly spoken as a sequence of double digits. Visteon voice-recognition systems deployed in these countries use the appropriate grammars.
Focusing on the user and usability factors while ensuring that the voice recognition engine is able to support the performance requirements has been key to Visteon's success in deploying wireless connectivity systems. The use of structural analysis tools for HMI design and the exhaustive quality assurance (QA) procedures in place have ensured that the product development cycles are short while the quality is high.
WIRELESS CONNECTIVITY
There is an increasing demand for automotive drivers and passengers to operate mobile electronic devices safely and conveniently inside the car. A leading factor has been the need for convenient operation of phones in-vehicle to meet customer concerns for safety as well as meeting legislated requirements in several markets. Additionally, the faster cycle times of the consumer electronic industry has made it difficult for automotive OEMs to deploy integrated solutions that meet the rapidly changing consumer expectation for features. Vehicle electronics need to be adapted to the standard of consumer electronics by deploying solutions that enable wireless connectivity to mobile devices and provide associated HMI to control these devices safely and conveniently.
COMBINING VOICE RECOGNITION AND WIRELESS CONNECTIVITY
Visteon has developed a wireless voice-activated phone system called Mach Voice Link (MVL) as shown in Figure 4, which is an integrated hands-free system for operating a mobile phone while driving a vehicle. It uses the driver's own mobile phone and calling plan, avoiding duplicate billing inherent in other phone systems. MVL uses Bluetooth wireless technology to connect to the phone so that no physical connection is required. It employs Visteon voice technology to quickly and accurately recognize spoken commands so that the driver can control the phone without touching it. A microphone in the vehicle picks up the driver's voice, while the other party's voice is heard through the audio system speakers.
This product provides:
- Wireless connection to a consumer mobile device (phone, other Bluetooth devices).
- Data transfer from customer mobile device (such as mobile phone book transfer) with associated in-vehicle HMI for safer use.
- Voice-recognition command and control of mobile devices, as well as other in-vehicle features such as audio system, climate control or navigation.
- Speaker-independent (i.e., no end-user voice training is required) voice recognition deployed in various languages with associated localized HMI.
- Acoustic echo cancellation and noise reduction for superior call quality.
- Seamless pairing of Bluetooth device to the vehicle to ensure consumer ease of use.
ACOUSTIC ECHO CANCELLATION AND NOISE REDUCTION
Although the automotive passenger compartment is an excellent application for voice command and control technology, the acoustic environment is not. A moving vehicle generates many sources of ambient noise, including wind, tire and climate control blowers.
The final hands-free call-quality performance of the system is determined by a combination of the noise/echo algorithm along with key components for the system such as the microphone and the audio system. A systems engineering approach to microphone placement, microphone selection (directional vs. current omnidirectional), vehicle acoustic tuning, etc. will enable the module to meet customer expectations for hands-free call quality.
ADVANCED VOICE RECOGNITION AND WIRELESS CONNECTIVITY
Beyond basic hands-free calling, voice recognition supports more advanced features such as direct dial address book and song selection via artist name or song title. A wireless connection opens the door to the vehicle and voice recognition helps to enable it.
Visteon's initial production system supported an address book function through two main approaches. The first was a manual process where each individual contact was programmed into the vehicle using “voice tags” and a phone number. The voice-recognition engine would use this “voice tag” to determine which contact to call. Although this approach allowed the driver to dial a contact without taking his/her eyes off the road, the process was manual and required the maintenance of two phone books. A second approach allowed the driver to access his/hers phone's address book through a vehicle display and scroll button. This approach did not use voice recognition and required the user to review a scrolling list while driving.
Direct dial address book combines the better of these two approaches. The phone's address book is transferred into the vehicle. The voice-recognition system builds grammar for accessing the contacts via simple voice commands. This approach:
- Enables the user to maintain a single address book.
- Through voice activation, allows drivers to select a contact and dial the number while keeping his or her eyes on the road and hands on the steering wheel.
- Eliminates the time required to create user-programmed “voice tags” for address book entries.
- Allows the user to store multiple phone numbers for a single contact.
- Allows user to know the caller identity without glancing at the phone or display.
SUMMARY
The consumer electronics market will continue to evolve at a rate faster than the automotive OEM design cycles can support. While consumer electronic devices provide exciting features for drivers, they introduce several challenges for OEMs. “Driving as the primary task” is a critical focus for automotive manufacturers to ensure a user-friendly and safe driving experience. Voice recognition and wireless connectivity technologies such as Bluetooth, combined with proper human-system integration, can enable customer access to their mobile and embedded devices while at the same time providing for workload optimized in-vehicle HMI designs.
ABOUT THE AUTHORS
Maha Kadirkamanathan is manager of Infinitive Speech Systems at Visteon Corp. Bill Klingler is infotainment platform manager at Visteon Corp. John Kosinski is advanced HMI engineer at Visteon Corp. and Joe Jira is infotainment product manager at Visteon Corp.