However, there also is a rising expectation in the quality of service and performance of these cellular services. An increasing number of households now depend solely on a cellular telephone, with no fixed line installed. The traveling user has realistic expectations that devices will work anywhere while roaming nationally and internationally.
The business user increasingly relies on mobile access to e-mail while mobile TV already is available in some markets. The merging of cellular and global positioning system (GPS) technologies is set to enhance the safety and navigation capability of cellular users.
Modern devices are taking on a wider range of formats, from classic candy-bar to PC data card to powerful Windows-based PDAs. Ensuring it all works reliably and consistently is challenging—but manageable.
For devices using 3rd Generation Partnership Project (3GPP) technologies such as GSM and UMTS, the answer lies in the extensive test and trial programs conducted at many stages during a device’s lifecycle. During the development process, extensive testing is conducted on all parts of the design: protocol stack, RF section, Subscriber Identity Module/Universal Subscriber Identity Module (SIM/USIM), and audio. Following completion of the development phase, the new device must go through the certification process.
So What Is the Problem?
Before cellular devices can be sold to the major network operators, they must pass the conformance tests and interoperability trials required by either of the two major certification bodies. In the Americas, this is the PCS Type Certification Review Board2 (PTCRB); in Europe and many other parts of the world, it is the Global Certification Forum3 (GCF).
This testing covers RF performance, signaling protocol, SIM/USIM, and audio aspects as well as the underlying enabling protocols behind the new applications such as the Multimedia Messaging Service (MMS) and Java. To test just the signaling protocol of a 3G device can, depending on the capability of the device, require more than 500 test cases to be executed.
The RF test cases must be performed in all supported frequency bands at a range of operating temperatures and battery voltages. So although there are not as many individual RF test cases, it can take hundreds of hours to run them under all conditions.
Why Do It?
The certification process has proven effective in improving the quality of service offered to users. Opening a recent GCF meeting, a senior manager for one of the European network operators stated that certified terminals suffered half the rate of dropped calls on their network compared to noncertified ones.
How Is It Done?
The conformance testing must be carried out in an approved lab. There are a number of specialist labs around the world that offer certification test services using test equipment approved by GCF and PTCRB. The number of authorized test labs has increased over the past 18 months, resulting in more competition and, consequently, lower hourly rates. However, device certification still is an expensive process, requiring hundreds of hours of test time.
Larger manufacturers, with sufficient numbers of models introduced each year, can justify the expense of establishing their own labs with independent management separate from the development teams. Smaller manufacturers make use of the commercial test labs, often using more than one to achieve the test coverage and time scale required.
Either way, there is a significant management overhead involved in the certification process, which lies in the direct time-to-market path of a new device. It is essential that the certification process happens smoothly and predictably to allow all aspects of the product launch to be planned accurately. A stalled product launch due to test failures can result in a new design being scrapped.
As in many complex developments, the earlier that realistic testing can take place, the sooner faults can be detected and corrected. It is well known that the cost of error correction gets higher the later it happens.
For that reason, a precertification test phase should be considered to increase the number of errors detected before the device leaves the lab. Ideally, precertification testing will use the same type of test equipment as that used in a formal approval lab, such as PTCRB- or GCF-listed equipment, or equipment capable of executing the same test cases. The cost of this extra test phase needs to be set against the potential losses made by late delivery of new products.
Once this investment has been made, it becomes more practical to conduct cyclic regression tests on every new software build, especially if automation capabilities are available. This precaution will prevent the buildup of multiple errors that result in a big-bang approach to final testing prior to release.
Impact of Test Failures
Test failures at the certification stage are among the most expensive failures, causing bottlenecks while the test lab diagnoses the cause of a fault. Many test labs have a single system to cover each type of testing such as protocol and RF due to the high cost of the
test systems.
Full conformance test systems are expensive due to the specialized nature of the equipment. In addition, the systems must be continuously revalidated to accommodate changes in the standards and functionality added to the conformance test regime by the certification bodies.
Device Design and Development
For a smaller manufacturer, the typical design cycle of a device involves buying a chipset, licensing the protocol stack, and developing a customizable user-interface, possibly based on a standard operating system.
The main development activity is to successfully integrate the various components and then verify that the design meets the specifications. This is greatly helped by ensuring that the suppliers already have tested the protocol stack and chipset, but this increases the cost and is not always available.
In this scenario, the development and integration team, unlikely to have access to an in-house conformance test system, will rely on standard instrumentation to do as much testing as possible before sending the device to an external test lab for certification. This increases the risk that the device will not pass first time.
Larger manufacturers tend to use multiple chipset sources, including both in-house developments and bought-in ones, usually with an in-house protocol stack and user interface. There typically will be multiple design teams working in parallel on different device types covering a model range, usually based at different locations around the world.
The company’s in-house test lab will provide the equipment and expertise needed to achieve the appropriate certification, depending on the market to be addressed. However, even the largest test labs usually would expect to subcontract some testing to other labs. This can be done to handle temporary overflows due to nonaligned delivery from multiple design centers or small gaps in test coverage.
One hundred percent test coverage would require the purchase of equipment from practically all approved test-equipment vendors since nearly all have some unique test cases. This is a situation the GCF is working to eliminate. A centrally managed certification test lab, often a multisite operation itself, represents a significant investment for the large manufacturers.
The efficiency of such a large operation relies on the quality of the designs being received for certification. Poor, nonconformant designs will result in unpredictable test-cycle times, making it difficult for the lab manager to schedule resources accurately.
Field Trials
Conformance tests ensure that a device complies with the relevant core specifications under repeatable laboratory conditions. Field trials guarantee that the device will work correctly in real-world conditions using a variety of infrastructure elements from different vendors.
Field trial-qualified networks in specific locations are used to execute a predefined test plan to ensure consistency from one device to the next. The field trials form an essential companion to the conformance tests, providing an important balance between lab-based and field-based testing.
More About the Protocol
Test Cases
To illustrate the level of detail that the certification process involves, let’s take a more detailed look at some of the UMTS protocol test cases, just a subset of the tests that a 3G device will need to pass. These tests are described in detail in the 3GPP technical specification TS 34.123.
The first part of this specification, which runs more than 5,000 pages, contains the prose-form of the test cases. For each test, the following are specified:
• Conformance Requirement—relates back to a specific feature detailed in the core specifications.
• Test Purpose—what is being verified.
• Method of Test—test conditions, test parameters and procedure, expected signaling sequence.
• Test Requirement—the condition that must be achieved to pass the test.
For these UMTS tests, the standardization body has taken the additional step of providing a working implementation of the test cases in part 3 of TS 34.123. The test cases have been implemented in Tree and Tabular Combined Notation-2 (TTCN-2), a standard test language, and made available in abstract form that is not dependent on a specific system simulator.
These test cases then can be compiled using a TTCN-2 compiler and linked with a TS 34.123 standard-compliant target adapter to allow them to be executed with a particular manufacturer’s system simulator. The test cases are supplied in suites covering different aspects such as Media Access Control (MAC), Radio Link Control (RLC), Radio Resource Control (RRC), and Non-Access Stratum (NAS). They are verified by an industry group.
For the 2G GSM/GPRS (General Packet Radio Service) standards, the protocol tests are specified in 3GPP 51.010 in a similar way to TS 34.123. For these test cases, however, there is no standard implementation available so the test-equipment manufacturers must develop their own test suites.
PTCRB and GCF
The primary role of the PTCRB and GCF is to manage the certification process for new cellular devices, including any variants. Although the certification test cases used by the PTCRB and GCF are similar, the procedures adopted by these bodies are quite different.
The PTCRB is a body led by cellular network operators mostly from North and South America. The GCF is a wider industry forum including members from network operators and handset vendors with test labs and test-equipment manufacturers as associate members. Both bodies hold quarterly meetings covering policy and technical matters.
The main technical difference between the two bodies can be traced to the frequency bands used. PTCRB certifies devices primarily intended for use in the 1,900-MHz and 850-MHz bands and GCF for the 900-MHz and 1,800-MHz bands plus 2,100-MHz FDD I band used for Wideband Code Division Multiple Access (WCDMA) 3G devices.
Both bodies keep track of the status of all certification test cases and the associated validated test systems although by different means. The PTCRB uses a spreadsheet and the GCF a database, both available only to members via their respective websites.
Using this information, labs determine which test cases are applicable and what equipment can be used. This is not a straightforward process, with many test cases being added each year and, occasionally, test cases being removed.
Test Equipment
Special test equipment is required to perform the conformance tests. This equipment will be listed by the PTCRB and GCF as validated for the execution of a specific set of tests.
The test systems usually are grouped into protocol test systems or system simulators, RF test systems, SIM/USIM simulators, audio test systems, and application-enabler test systems. There are a number of suppliers of these systems including Aeroflex, Anite, Anritsu, AT4Wireless, Comprion, Rohde & Schwarz, and Spirent. The systems for all of these companies have been assessed by independent validation organizations.
Conclusion
The certification process can seem like a long and winding road, but the proven compliance of certified devices brings benefits to both consumers and network operators. Consumers gain from the improved interoperability of their devices, especially when roaming onto different networks. Network operators benefit by reducing the amount of approval testing they need to perform themselves, relying instead on the results of the certification process. Device manufacturers need to plan the time and effort they put into the certification process to ensure new-product introductions meet schedule and cost requirements.
References
1. Global Mobile Suppliers Association, www.gsacom.com
2. PCS Type Certification Review Board, www.ptcrb.com
3. Global Certification Forum, www.globalcertificationforum.org
About the Author
Phil Medd is a product manager for Aeroflex Wireless. Mr. Medd graduated from Nottingham University in 1977 with a degree in electrical and electronic engineering. Since then, he has worked in electronic system design and programming, embedded software development for GSM handset and base station test systems, and development of test systems for satellite communications systems and cdma2000 cellular handsets. Aeroflex Wireless, Progress Business Centre, Whittle Parkway, Slough, Berkshire SL1 6DQ, UK, +44 (0) 1628 604455, e-mail: [email protected]
March 2008