At times, it has seemed as if the momentum around 3G was almost completely stalled. Yet a closer look reveals that only the overblown 3G hype has died. Forward-looking participants continue to invest toward 3G deployments. According to a recent Business Week article, European operators will order roughly $1.5 billion in 3G equipment this year. That number is expected to climb to $5.5 billion in 2004. Spain's Telefonica, for example, has already installed 750 3G-capable base stations in 21 cities. It has kickoff plans for early next year. Test networks also are up and running in 24 European countries. Within the coming year, initial commercial 3G services are expected to launch in Britain, Finland, and Italy.
Obviously, the telecommunications business sector is striving to move forward with third-generation service offerings. During this time, major technological challenges must be addressed and overcome in nearly every facet of the industry. It is imperative for both carriers and service providers to give their subscribers a smooth transition to new 3G-based applications and services. At the same time, they must guarantee the integrity of their third-generation infrastructures and handsets.
A successful transition to 3G will require the ready availability of rigorously tested, 3G-capable handsets. The development and testing of handsets has been held up, however, by a lag in the demand for 3G services. No one can predict when the upswing will occur for 3G market demand. This uncertainty has fed a general atmosphere of postponement among many industry players. "Wait and see" has erroneously appeared to be the safest choice. Unfortunately, given the relatively long time cycles needed for developing handset products, the 3G window of opportunity has become tighter with each delay or postponement.
As a result, operators and their handset suppliers are facing a difficult squeeze. They want to meet their 3G rollout objectives and jumpstart the market with advanced 3G handsets. At the same time, they want to make sure that they are not at risk for any handset problems that can cause loss of reputation and/or costly recalls. Thankfully, a solution does exist. The answer lies in compliance testing using comprehensive test suites to rigorously exercise complex 3G protocol stacks.
The Third Generation Partnership Project, or 3GPP, has taken the first step through the high-level definition of hundreds of industry-standard test cases for 3G products. It is appropriate for the creation of "prose" test-case definitions to be addressed via an industry-wide standards-setting organization like 3GPP. Unfortunately, 3GPP's implementation of specific real-world test suites could lag significantly behind the carriers' increasingly critical 3G-deployment schedules. The experts within the wireless-test industry segment best handle the critical next step of implementing actual test programs and platforms.
An effective test strategy will thoroughly exercise 3G device designs and functionality across a range of normal and abnormal operating conditions. In complex communications structures like 3G systems, this need translates into the explicit testing of protocol stacks in hundreds of different specific and repeatable situations.
Handset developers, on the other hand, will perform low-level module tests and then move on to integration testing. To ensure correct operation, however, the handset needs to be thoroughly tested as a complete entity.
In light of the increasing complexity of 3G systems, providers cannot afford to rely only on operational testing as they roll out new products and services. If they did, they might have handsets in the field that will not function properly as new services are added. This scenario could potentially lead to costly product recalls and loss of competitive position. The only viable solution is to conduct comprehensive pre-deployment testing of handsets using the 3GPP-defined test specifications.
Here are examples of the 3GPP test coverage that is required by GCF:
- Idle mode
- Basic procedures:
- MO call
- MT call
- Packet- and circuit-switched data
- Lower-layer protocol RLC, MAC
- Cell selection and re-selection
- Handovers
- Simultaneous services: speech + data
- Interoperation between the third generation and 2G or 2.5G
For handset manufacturers to assure the compliance of their product offerings, all of the test cases must efficiently run on target testing hardware. Handset manufacturers and 3G service providers cannot afford to wait for 3GPP to develop idealized abstract test programs. Such programs would then need to be adapted for specific test platforms.
CASE IMPLEMENTATION In OSI protocol conformance testing, Tree and Tabular Combined Notation (TTCN) has become the de-facto standard for the creation of the test cases that are used. It is a well-established, highly structured programming language. TTCN has already gained acceptance in the wireless world as the standard for implementing GSM test cases.As a highly abstract language, TTCN permits the implementation of very precise test cases. Such cases treat the underlying device under test as a "black box." Plus, its inherently abstract nature enables the test cases written in TTCN to provide platform independence with regard to test systems. Theoretically, a test suite that was written in TTCN for any particular application can be used to accurately check that specific application in any test-system environment (FIG. 1). In actual practice, however, the abstract TTCN code must ultimately be converted to the appropriate code for running on and controlling the target test platform (assembly, "C", etc.). The TTCN code must be viewed as a "means to an end" rather than an end in itself.
The prose language that is used for the test specification can be subject to some degree of interpretation. Yet the TTCN code utilizes a very precise low-level syntax. This syntax is based on symbols that are interpreted only by computer. It must therefore be totally unambiguous. The creation of the test code requires very specific skills and rigorous disciplines. To ensure consistency between the test specification and each test case, prose definitions must be accurately translated into abstract TTCN syntax notation.
In addition, the test-suite implementers must have an understanding and awareness of the issues that involve the ultimate target platforms. These platforms will be used for running the tests. TTCN is an unambiguous translation language that can provide an accurate re-creation of prose specifications. The development of TTCN code without any regard for the target hardware, however, can lead to test programs that do not make the most efficient use of available hardware resources.
Another key issue must be taken into consideration: the need for configuration control of the TTCN test cases. Handset manufacturers and wireless operators must be able to confidently use the test cases and systems to accomplish critical development and deployment objectives. As a result, it is vital that the TTCN code be maintained under rigorous change-control procedures. Communication of change processes also is important. Manufacturers and operators need to know exactly what modifications are planned and when they will be available (FIG. 2). Otherwise, handset manufacturers could risk losing significant time and money. Picture what would happen if the conformance-testing environment suddenly and unexpectedly deviated from the assumptions under which they had been conducting their product development.
It can be difficult to achieve such carefully controlled change management within a process that is entirely driven by committee-based decision making. One approach, which was adopted by Anite, is to control the entire test-system environment while adhering to the stringent 3GPP prose test specifications. This approach eliminates many of the uncontrolled variables in the process. It also assures tight change control for the test-case users. For Anite, this method also lets it act as a buffer. The company can dampen any risks of uncontrolled changes while providing its customer base with stable testing environments that incorporate beneficial changes on a rational, controlled, and clearly communicated basis.
STATE OF THE PROCESS The 3GPP is a broad-based consortium. It brings together a number of telecommunications standards bodies as "organizational partners" to produce globally applicable technical specifications and reports for a third-generation mobile system. Currently, however, 3GPP also has taken on the task of translating high-level "core specifications" for mobile devices into "prose test specifications." Ultimately, these specifications will be translated into TTCN test cases.In parallel, the Global Certification Forum (GCF) is evaluating the test specifications. The forum also is prioritizing them within an overall implementation scheme. This scheme is designed to provide operators with standard test regimes that will support the introduction of 3G handsets. Because of the need to encompass a number of existing and evolving standards within multi-mode 3G handsets, the sheer amount of total test cases represents a massive undertaking.
For example, 800+ GSM/GPRS test cases have already been specified to cover operation across the 850-MHz, 900-MHz, 1800-MHz, and 1900-MHz bands. If each band is included, the number multiplies to up to ~1500 tests. In addition, over 700 specified W-CDMA test cases have been selected from the many thousands of possible W-CDMA functions. As W-CDMA and its feature-rich handsets appear, the technology's many different data rates and coding schemes will all need testing. Test cases must therefore include multiple paths (e.g., repeating the same test at 9.6 and 64 kbps). Producing a combined W-CDMA/GSM/GPRS handset will likely require the performance of thousands of conformance tests.
The abstract nature of TTCN provides a good environment for creating "ideal" test cases. When it comes down to real-world testing, though, the TTCN test cases have to be able to run efficiently on the physical testing hardware. In order to provide optimal performance, the test processes must take into account the specific resources and capabilities available in the target test platform(s).
Given the overall complexity of 3G wireless technology and the criticality of subtle timing issues, it is vital that the test cases and testing hardware be "tuned" to maintain a consistent testing environment. Otherwise, variations in test results could result from uncontrolled variables in the test environment—thereby masking errors or creating false error conditions in the device under test.
THE FAST PATH Some test and measurement providers continue to be very concerned about the amount of effort that is still needed to create and debug the detailed TTCN code. In addition, they worry about the inherent risks of not having sufficient test cases to support the growing market need for 3G deployments.One strategy being implemented is an independent and more aggressive schedule to develop TTCN test-case code. This code will conform explicitly to the 3GPP prose definitions. The strategy's goal is to provide a critical mass of test cases for supporting full-scale 3G rollouts before the governing bodies release their test cases. This comprehensive initiative does not just focus on producing TTCN code. It also includes independent third-party validation of the test cases. In addition, it optimizes these cases for real-world testing on target test-system platforms.
Other options also have been considered, such as the splitting of tests across numerous platforms. This approach would attempt to spread the load of test-case debugging. It has been rejected, however, as it would require that terminal developers have access to multiple test systems. Such access is unlikely to be economically viable. Plus, this strategy relies on the availability of TTCN from the standards bodies.
TUNING PERFORMANCE In this quagmire, a test and measurement solutions provider offers a key benefit: its ability to tune the TTCN code to mesh with specific hardware. A solutions provider should be able to optimize the hardware's efficiency to improve cost, features, and performance. The resulting devices should supercede those that can be built by overlaying abstract TTCN code onto generic hardware. Anite, for example, does not treat the development of 3G TTCN test-case code as an end in itself. Instead, its development process has been aimed at providing comprehensive test platforms that are targeted to meet the real-world 3G testing requirements of manufacturers and operators.No matter which approach or strategy wins out, the drawbacks of waiting are clear. Yet the network providers must either wait for adequately tested handsets or launch services with a large risk of problems. Unfortunately, some network operators cannot afford to wait. Having paid for expensive licenses, they must start earning revenue or fail. The only alternative is to launch products without having performed sufficient testing. Of course, making this choice is liable to lead to product recalls, poor performance, and disappointed customers. These problems will negatively affect both service providers and handset makers. Most importantly, they also will tarnish the reputation of 3G.