Hardware failures or software bugs can have devastating effects in real-time computer systems. In submarine navigation systems, for example, equipment failure can mean loss of life or costly submarine repairs. In commercial applications such as interactive TV, failures can both infuriate customers and deplete the revenue streams of service providers.
Regardless of the application, system designers who want to enhance reliability must evaluate their suppliers more closely. As a result, supplier compliance with quality standards such as ISO 9000 is becoming crucial.
In general office and scientific computing environments involving PCs and workstations, computer users can often handle software bugs or hardware failures. A board swap typically resolves hardware failures, and users can eliminate software bugs by obtaining fixes and other upgrades (via floppy disk or bulletin board) from the vendor. While bothersome, these types of failures aren’t critical in most general-purpose computing environments.
Embedded-system designers don’t typically enjoy the luxury of simple field repairs. For example, a software bug found in an interactive TV set-top box after deployment can result in a company recalling millions of installed units to replace a simple memory IC.
Upgrades can even be painful in low-volume applications. A latent bug that appears in a remote real-time control system–an oil pump in an Arctic Circle drilling rig, for instance–may result in millions of dollars in lost oil revenue and require an expensive service visit.
The Quality Movement
Over the past decade, design engineers have begun to examine vendor quality issues more closely. Most designers, however, focus on hardware, and software often escapes the stringent quality requirements. With the complexity of software in embedded systems expanding rapidly, the same rigid quality standards must be applied to both the hardware and software selection process.
Cross certification between hardware and software suppliers is paramount, since system reliability is largely determined by the interaction between components, not just the individual components. More and more designers today rely on the ISO 9000 standard to ensure that vendors have robust quality programs in place.
ISO 9000 provides a consistent set of metrics by which an individual company’s quality or total quality management program can be judged. The standard covers 20 operational sectors of an organization which a company must address to achieve certification.
While the standard certainly covers manufacturing and engineering sectors, it also focuses on other disciplines within an organization, such as accounting, personnel and training. To operate in an ISO 9000-compliant environment, a company must infuse the quality philosophy into every nook and cranny of the organization.
A company can be certified to one of four levels of ISO 9000 compliance:
ISO 9001
–Model for quality assurance in design/development, production, installation and service.ISO 9002
–Model for quality assurance in production and installation.ISO 9003
–Model for quality assurance in final inspection and test.ISO 9004
–Quality management and quality system elements.
While companies such as low-end component manufacturers, distributors and service organizations may be satisfied with a lower level of compliance (such as ISO 9004 alone), technology design and manufacturing companies should be held to the ISO 9001 level, which encompasses all aspects of ISO 9000.
Today, embedded-system designers can buy both real-time hardware and software building blocks from ISO 9001-certified suppliers. Designers can also buy bundled ISO 9001 real-time hardware and software from the same vendor.
Though many embedded-system designers still prefer to roll their own CPU boards and real-time kernels, a growing number use off-the-shelf products to cut development times and focus on application development. Purchasing ISO 9001-qualified boards and software, either separately or as a bundled product, simplifies the system-level ISO 9001 qualification process. Designers still have to incorporate the boards and software into their quality systems, but having complex components like the CPU boards and real-time OS prequalified certainly gives them a head start.
Procurement and Design
During the procurement process, an ISO 9001-certified company must evaluate every component vendor using a vendor certification procedure. A vendor supplying a simpler component, such as a connector, is evaluated against the same criteria as the supplier of a 32-bit microprocessor or software–specifically, the capability to meet the product requirements.
The procurement process ensures that suppliers have quality procedures in place. In the case of boards hosting hundreds of components, such detailed vendor evaluations are the only way to provide a reliable end product.
In the case of two certified companies, the fact that each has received ISO 9001 certification simplifies the supplier/vendor relationship. For example, a supplier of real-time CPU boards can integrate and sell real-time operating systems and development tools from another company without performing the in-depth supplier evaluation and product verification that would be required for a noncertified company.
Verifying a supplier’s internal procurement procedures is an important, yet relatively straightforward, process whether the supplier is ISO certified or not. The design and test process, which injects a company’s knowledge base into a product, is much less tangible and more difficult to measure. It is here that knowing a supplier has developed and manufactured its product in accordance with ISO guidelines gives system designers the greatest comfort.
Both hardware and software suppliers must develop extensive product planning, development, production and shipping guidelines to meet ISO 9001 requirements. Both also aspire to produce products with zero defects. This implies shipping products that not only meet outgoing inspection tests, but will work in virtually any real-world environment. To achieve this zero defect goal, both hardware and software companies must implement testing and validation programs that eliminate marginal areas of product design, those that may fail in a product’s worst-case scenario.
Hardware/Software Metrics/Testing
A software company can test and validate its products through the use of third-party validation suites and code profile tools. Software vendors, for example, can use the Plum Hall Validation Suite to validate its compilers prior to release. Test and analysis software is also available for verifying the quality and optimization level of application code.
Metrics from these validation and profile tools are collected throughout the development process as well as each time a product is updated and released. These metrics can identify improper language use, nontransportable code or noncompliance with internal and external standards.
Close attention to these metrics assures that as additional code and features are added, the overall code quality will not diminish. These procedures also help ensure that no bugs appear later.
Embedded-system designers basing their systems on software developed under an ISO 9001 program can be assured that this design and measurement process relies on completely independent design and test groups. The design group develops software within the confines of a certified set of procedures. The test group measures and verifies each step of the process independent of the design and implementation process.
Hardware companies must also implement procedures that verify the operation of their equipment under worst-case conditions. A real-time VMEbus CPU board, for example, must pass not only stand-alone tests that verify baseline component level functionality and VMEbus compliance, but also system-level tests conducted in a fully populated VME enclosure that verify functions such as multiprocessor data transfer, arbitration and interrupt response. These system-level tests, performed under heavy loading conditions, are the key to exposing marginal designs.
Customer Input
A customer new to quality programs may be surprised by how widespread the effects of an ISO 9000 program can be. Compliant programs not only result in reliably manufactured products, but also mandate that the certified supplier adopt a philosophy based on serving the customer. More succinctly, the customer often gets a say about how the hardware or software vendor goes about the conception and development of new products.
A key part of any ISO 9001 program is a systematic review of customer requirements, which is undertaken for all new product development. Both hardware and software suppliers should seek customer input during the planning stage and validate the design with strategic customers as the design progresses through engineering to manufacturing.
In the area of customer support, software vendors, as well as hardware vendors who resell and provide internal support for third part software, sometimes face a more difficult task than do vendors of hardware alone. Hardware problems tend to be finite; software problems tend to be abstract and may escalate as customers develop their applications. Still, to meet ISO 9001 guidelines, software suppliers must implement support procedures that ensure that any customer problem can be addressed.
Within hardware and software companies, technical customer support must be governed by a set of procedures. These procedures clearly state how any problem will be resolved and who will resolve it. The procedures mandate training programs for the support engineers and define how calls from customers are reviewed, summarized and distributed to middle management.
Though many of the benefits that emerge from implementing an ISO 9001 program or buying from ISO suppliers aren’t quantifiable, many can be measured. Since obtaining ISO certification 18 months ago, for example, Heurikon has documented a 0.1% failure rate for its boards. This in-warranty failure rate has been flat from month to month. Before establishing an ISO 9001 program, the in-warranty failure rate occasionally dropped as low as 0.1%, but fluctuated on a month-to-month basis with significantly higher rates.
ISO 9000-certified suppliers offer other, less tangible benefits to embedded-system designers. For example, the designer is assured of the supplier’s senior management commitment. This commitment, together with a strong middle-management team that can infuse the quality philosophy throughout the organization, is key to any comprehensive quality program. Such commitment ensures that the benefits of dealing with certified suppliers extend to not only design, manufacturing and shipping, but also packaging, accounting and even marketing.
Increasingly, doing business at just about any level in the embedded-systems food chain requires some kind of quality program. And among large companies, ISO 9001 is becoming the quality program of choice. By selecting ISO 9001 suppliers, embedded-system designers not only enhance their product quality, but also make it easier to meet their customer’s requirements.
If you are faced with implementing a quality program, don’t despair. It takes a lot of hard work, but it definitely pays off. Companies who implement ISO 9001 programs typically realize documented quality improvements that directly impact the financial bottom line.
At the same time, employees who embrace the quality philosophy at these companies can expect to see improvements in their own work processes. As many companies are discovering, ISO 9000 is just part of doing business today–and a positive part at that.
About the Authors
Bob Glover is the Quality Manager at Heurikon Corp. Before joining the company, he was Quality Manager at Rayovac for seven years. Mr. Glover earned a B.S. degree in chemistry from Wake Forest University and an M.S. degree in biochemistry from the University of Wisconsin. Heurikon Corp., 8310 Excelsior Dr., Madison, WI 53717, (608) 831-0900.
Ike Brown is the Director of Operations, Quality Assurance, at Microware Systems Corp. He has been with Microware for more than three years, and recently led the company’s program for ISO 9001 certification. Mr. Brown attended Drake University.
Dave West is the Software Integration Manager at Microware Systems. Mr. West has been employed at the company since 1984, after graduating from Drake University with a B.S. degree in business administration.
Microware Systems Corp., 1900 N.W. 114th St., Des Moines, IA 50325, (515) 224-1929.
Inspection
Copyright 1995 Nelson Publishing Inc.
September 1995
|
|