In the April 1996 issue of EE, we discussed when to use a systems integrator (SI), what relationship goals to define and how to select the right company to partner with in developing your test system. Briefly in review, the magnitude and difficulty of the specific requirements can drive the process for qualifying and contracting with an SI. Qualification of a third-party integrator may only involve meeting the third party, evaluating the proposal and approach, and checking references. Or, it may be a more extensive process involving credit checks and financial reviews, investigating corporate officers, understanding development processes, reviewing staffing and looking at equipment capitalization.
Although cost is an element in selecting a qualified SI, it should not be the only factor. Do not use pricing as the most important selection criteria. Any competitive price differences may be offset by poor performance caused by misaligned competencies or underbidding.
Using third-party SIs can solve specific testing challenges by leveraging your existing staff. A competent SI can be an extension of your company’s existing resources when placed under the proper direction using various project management processes.
Once you decide to use an SI, you are ready to begin the project. Many times, a solution provider and an SI will work together to deliver the product. A solution provider is an organization with sufficiently broad technologies and strengths that it can supplement the capabilities of an SI.
Your Responsibilities
Although important, selecting a qualified SI is only the first step in the process to subcontract all, or part, of the solution. Both you and the integrator are responsible for the success of the project. A common scenario is outlined in Figure 1.Neither you nor the selected integrator can guarantee success independently. The integrator can be primarily responsible for designing (the system functional specification [SFS]), executing and implementing the requirements that you specified. The integrator may assist in documenting these requirements by writing the system requirements specification (SRS) which documents your needs. You, however, must ensure that all your goals are clearly communicated in the SRS to the integrator. If your organization is new to developing and documenting specifications or if your organization is capacity limited, a solution supplier or integrator can help you develop the requirements document.
You must designate a single point of contact between you and the integrator. This single point of contact is the project or program manager.
Finally, you must ensure that your company is ready for the test system and that it fits into your company’s processes. Too many companies develop systems on tight schedules only to discover there is inadequate facility power in the desired production area when the system is received.
If the test station results in changes to the normal mode of production, special training may be required or procedures updated to reflect these enhancements. Failure to plan for these changes can result in production disruption and a perception that the test-system development was not successful.
Implementation Steps
To ensure that the project is implemented successfully, there are seven key steps to follow. These steps are illustrated in Figure 2. In general, it is important that these steps be followed in the order presented in the figure.
Failure to inspect and approve the integrator’s design before implementation at design reviews increases the risk that a key requirement was overlooked and that the system will not meet your needs when supplied. For example, failure to document the system requirements may result in a system that tests your product but is too slow, too large to fit in your production line, or too power hungry.
Designing, installing and supporting the system are the responsibilities of the SI. If you selected a quality integrator and are practicing good project-management processes, these steps should be executed with minimal risk.
Strong Project Management
One of the biggest impacts to project success is the use of good project management. Although many project problems will be avoided with a good SRS, no specification can scrupulously document all possible requirements, scenarios and permutations.
During project development and implementation, it is inevitable that issues will arise requiring trade-offs and clarifications. The guidance provided by the project manager can have significant impacts on project costs, the schedule and overall technical performance.
It is very important that the project manager have the authority to make expedient key decisions, the expertise to facilitate these decisions, and the support of other departments within the company who hold a stake in the success of the solution. When using an SI, it is common to assign an internal project manager who may be technically competent but lacks an understanding of how to run a project. As a result, the project is executed in a vacuum relative to other departments and the resulting system does not meet overall company objectives.
Proper project planning is particularly important when leading-edge technology is involved or when the schedule is compressed. Project planning is documented and covers areas of schedule, scope and resources.
A key part of the project plan is a definition of project structure, roles and responsibilities, risk assessment and budget. These must be carefully analyzed and addressed by the project manager.
Before awarding the contract, the project manager will require insights into the thoroughness of the integrator’s project plan. To facilitate this insight, the project manager will probably require a work breakdown schedule (WBS) in sufficient detail to indicate probable success by the integrator. To facilitate execution and implementation, this WBS should provide measurable milestones for project tracking. These milestones may be tied to progress payments.
Once the SRS document is complete and development starts, the integrator should present regular status reports. Along with summarizing progress, these reports should reveal key decisions and trade-offs being made as part of the project development and implementation.
It is critical that you review these decisions and make sure you agree with them. Otherwise, the delivered system may be technically correct and meet your specification but not work well in your company’s environment.
An SFS is developed after the SRS and describes how the requirements in the SRS will be executed and implemented. Among the many technical details, this SFS should include how acceptance will specifically be accomplished. This is also called an acceptance test procedure.
Depending on complexity, the SFS can be made in two or more presentations, represented by a preliminary design review phase and a critical design review (CDR) phase. At the conclusion of the CDR phase, you should be able to accept the design that will be implemented from the SFS.
Document Your Requirements
A complete SRS that describes all the factors needed for the test system to meet your objectives is a major factor in successful projects. The SRS includes not only a list of the technical objectives and constraints, but also all management requirements such as schedule, budget, flexibility for future changes and integration with existing production processes. Even when solutions are developed in-house, thorough documentation of the requirements should be practiced.
When writing the SRS, it is better to over-document than to under-document your requirements. Remember, the integrator does not work in your environment on a daily basis and does not have an intimate understanding of your product. Features and processes you take for granted are not known, and may not be taken into account by the integrator.
On the other hand, the SRS should not describe how to meet the design needs unless this is a requirement you want to impose. Doing so restricts the integrator’s options and may result in higher cost, lower performance and lack of innovation that could lead to a better solution.
Review the Design
In principle, always insist that the test station be fully designed and documented in an SFS before fabricating the hardware or coding any software. Move through the preliminary and CDR phases with detailed reviews by your own technical people and the eventual users of the system to ensure that the design meets all requirements. This review should cover all key aspects of the design.
Hardware documentation should include a rack layout, a system block diagram and interconnect, cable drawings, schematics of all boards, parts lists and conceptual drawings for any custom mechanical items and fixtures.
Software documentation will vary based on the environment (object-oriented or traditional language), the tools used (such as a test executive) and the complexity of the design. Generally, it should include some form of module specifications, data dictionaries for key variables, and a structure chart to define the internal design.
The software should also be documented from an external viewpoint, or what is seen by the user. This requires a description of the user screens, sample reports and data storage.
A draft user’s manual can help convey this information at a design review. This manual typically contains sections for an operator (uses the system), a test engineer (supports and modifies the system for new test requirements), and a system manager (manages system-level issues including security, data bases, major changes, configuration management and users). This approach allows you to “run the system” on paper, getting a feel for actual operation and potential changes that would make the test system better fit your company processes.
It is important that you review all information, looking for in-scope improvements, problems with the design or possible confusion about your requirements. This is a good time to have typical operators review the user interface.
Changes made at this stage will generally have little or no impact on project performance measures. Changes occurring later can significantly impact cost, schedule or scope.
Summary
Outsourcing test solutions by using an SI or solution provider can present you with tremendous cost, schedule and technical benefits. Quality integrators can work with you in a partnering relationship, and they can be considered, in many ways, like part of your staff.
Using good project-management techniques is integral to a successful project for both you and your SI. In today’s and tomorrow’s changing business climates, it makes a lot of sense to use third-party SIs to implement test solutions.
About the Authors
Paul Hiller is the President of Symmetrix. He started his test-system company, Test Quality Co., in 1981 in California and changed the name to Symmetrix after moving the operation to Austin. He received a B.S. degree in business administration and an M.S.E.E. degree from the University of Wyoming. Symmetrix, 1301 Capital of Texas Hwy., #C200, Austin, TX 78746,(800) 560-8378.
Joseph McDonough is a Field Project Manager at Hewlett-Packard. He holds B.S. and M.S.E.E. degrees from California Polytechnic University. Hewlett-Packard, 1421 S. Manhattan Ave., Fullerton, CA 92631, (714) 758-5929.
Figure 1.
Your Responsibilities
Integrator Responsibilities
Specify Requirements (SRS):
Project Goals
Company Processes
DUT Specifications
User Requirements
System Administration
Test Approaches, Speeds
Accuracies, Tolerances
Software, User Interface
Repeatability
Fixturing, Interface
Data Storage
Acceptance Criteria
Schedule, Budget Requirements
Environmental Requirements
Support, Spares
System Design (SFS):
Meet SRS Requirements
Meet Project Goals
Test Techniques
Test Implementation
Solution Configuration
Documentation
Instrumentation Selection
Software Design
Custom Hardware Design
User Interface Design
Review and Decision Making
Change Process
Design Reviews
Acceptance Test Plan
Milestones
System Implementation
Execute Contract
Implement Approved Design
Order Parts and Integrate
Develop Software and Integrate
Develop Acceptance Test
Execute Acceptance Test
Integration Into Company Processes
Configuration Management
Security Requirements
Support
Assign Project Manager
Customer-Furnished Equipment
Problem Resolution
Installation and Training
Execute as Contracted
Ship Solution
Install Solution
Provide Training
Provide Documentation
Warranty and Support
Provide Contracted Support
Figure 2.
Copyright 1996 Nelson Publishing Inc.
July 1996