Author Eric Starkloff
We’ve all followed the stories about Toyota’s recent recalls. The current estimate is that these recalls will cost Toyota up to $2 billion. The damage to Toyota’s quality brand, which has been built over several decades, also is likely to be very costly.
While it’s premature to speculate as to all the causes of these recalls, they will likely include gaps in the engineering process, the culture and corporate management, and how the company responded to the defects once they were known.
Regardless of the causes, the visibility of Toyota’s crisis will create more focus across many organizations on quality. Test, which provides the essential function of ensuring quality during product design and manufacturing, has always been challenging to justify as a strategic investment.
ALL MODELS ARE WRONG, BUT SOME ARE USEFUL
This often-quoted phrase, attributed to statistician George E.P. Box, reminds engineers about the need to test our assumptions and verify models with real-world data. As device complexity continues to increase, and as we push the physical limits of mechanical and electrical systems, this reminder is as relevant as ever.
Modeling and simulation are powerful tools in all engineering disciplines. But increasingly, test is being used throughout the product development cycle—from research through final production—to verify and enhance modeling techniques.
In early research, measurement is used to create the models of subcomponents. Complex components, like RF semiconductors, often have inaccurate models. Only through real-world measurement can more accurate designs be built. During the product development step, test is used to compare a prototype’s actual performance to the predicted performance of system-level models.
Hardware-in-the-loop (HIL) testing combines modeling with real-world data. An HIL system simulates parts of an embedded system so its dynamic performance can be tested in a variety of operating conditions. To test a motor control system, an HIL test system uses real-world I/O to control the inputs and measure the outputs of the controller while simulating the motor. HIL testing can provide a fast and cost effective way to test complex embedded systems under many different conditions to ensure correct and robust operation.
Ford uses an HIL system based on NI LabVIEW and PXI to test prototype control systems for fuel-cell vehicles using PXI I/O cards while simulating other parts of the system including various sensors and actuators. Ford uses this system to rapidly demonstrate prototype controllers in real-world conditions.
LINKING DESIGN AND PRODUCTION TEST
Test is performed throughout the product development process. Yet all too often, it is done with different testing platforms and techniques at each stage. This makes it difficult to correlate problems encountered in manufacturing or in the field with validation data. The challenge is that the testing requirements of these different stages often differ in significant ways.
In the early stages of product design, quick measurements are taken to verify a prototype. In verification testing, the product goes through a very thorough suite of tests to test different potential operating conditions, such as HIL. And, in production, the goal is to test just enough to ensure quality and to keep the manufacturing process in control.
While the measurement and test automation needs vary across these use cases, a common testing platform with shared components can address them all. Having a single platform enables better correlation and traceability of data from a fault encountered in production or in the field, back through to validation.
HOW MUCH IS TEST REALLY WORTH?
Too often, test is viewed as a necessary evil. We ask how much it costs. Instead, we should be asking what test is worth. Test improves a product’s performance, increases quality and reliability, and lowers return rates. It is estimated that the cost of a failure decreases by a factor of 10 when the error is caught in production instead of in the field and decreases by a factor of 10 again if it is caught in design instead of production.
By catching these defects and collecting the data to improve a design or process, test delivers value. If you understand the value of catching defects through test, you can make more educated investments. Most companies under-invest in test, yet paradoxically spend too much—in slower product development, longer manufacturing cycles, and expensive repair and recall costs.
Improving your results requires a strategy for testing throughout product development that includes people, process, and technology. The right people are required to develop and maintain a cohesive test strategy. In test, this can be particularly challenging due to the growing experience gap in the field of test engineering. Process improvements are required to streamline test development and reuse throughout product development. And, technology always offers new ways to solve the challenges in testing complex products. The key is tracking and incorporating new technologies to improve test system performance or lower costs.
These elements can elevate your testing function from a cost center to a strategic advantage. Toyota’s issues give us another motivation—what is the risk to our company if we don’t look at test strategically? As test engineer, you must drive this strategic view and justify the investments. If you don’t, your company might be in the headlines for the wrong reasons.