Testing Times—Clearing The Validation Hurdles

Motorola's MVME6100 embodies the complete collection of technologies in the first wave of the VME Renaissance, namely PCI-X onboard interconnects, PCI-X mezzanine sites and the very fast 2eSST protocol across the backplane.

All of this is how it should be, but there is a 'but'. Having a product with such a large number of new technologies imposes special challenges at the test and validation stage of the development cycle.

By far the most challenging new technology on the MVME6100 (see Figure 1) is the implementation of the 2eSST protocol on the backplane. Some time ago Motorola undertook a project codenamed 'Tempe' to create a PCI-X to 2eSST VME bridge chip. The resulting chip, now called the TSI148, was turned over to Tundra Semiconductor for distribution and support to the VME market. The MVME6100 is the first VME product to incorporate the TSI148.

Since ASIC development is often the long lead item in the development of a new product, it was important the chip worked correctly first time. To increase this probability the intelligence in the TSI148 was tested long before it was encapsulated in a chip.

Two major approaches were employed: simulation and emulation. The logic of the TSI148 was simulated on as many as thirty high-power Unix workstations running test vectors continually for many months. These simulations uncovered many defects that were corrected and retested long before the chip went to fab. The down side of the simulations was that it would literally take days to execute logic that would run in less than a second of real time. The second approach used to validate the logic of the TSI148 chip before it went to fabrication was emulation. In the case of emulation, a special VME test board was created containing an FPGA where the TSI148 would normally reside. The logic of the TSI148 was then downloaded into the FPGA and test cases were run. In most respects, the FPGA behaved as the TSI148 would and this allowed test vectors to be run in real time, a considerable cycle time reduction over the simulation environment. When a test vector in the simulation environment would identify a problem, the FPGA board would be used to cross-check the defect, and vice versa.

After the TSI148 came back from the fab a second VME test board was created, this one containing an actual TSI148. The only noteworthy components on this second test board were the TSI148 itself and two PMC sites connecting to the PCI-X side of the chip. In this way the TSI148 could be exercised with a variety of processor types by simply snapping on a different ProcessorPMC module.

Tundra Semiconductor participated in the test process by employing the second test board and a Motorola PrPMC 800 module to perform additional independent testing of the Tempe ASIC, which they called 'Gauntlet testing'. Tundra's Gauntlet test focused on the robustness of the VMEbus interface in the presence of timing variations, varying degrees of bus loading and noise. The Gauntlet test included four types of tests, each run for three days without interruption. The arbitration test exercised the arbiter and performed simple data transfers between target boards. The block test performed VME block transfers. The test-and-set test was designed to test VMEbus semaphore activity as well as the read-modify-write cycle. The fourth test exercised the VMEbus priority interrupt bus, ensuring that all seven VME interrupt levels can be enabled and disabled, generated and handled.

Once testing on the live TSI148 test board indicated that the chip was stable, Motorola produced a series of MVME6100 prototype boards. This was the first successful integration of the hardware associated with the MVME6100.

Meanwhile, more than six months before the hardware unit test plan was being executed on the first real MVME6100, the software was being developed for the product. With embedded products there is a bit of a 'chicken and egg' situation regarding the hardware and the software. Hardware really needs working software to test the board, and software cannot develop until they have a working board.

Developing a very early prototype of the MVME6100 that was created without a VME bridge chip solved this challenge. In this way the software team had a stable environment to develop and test their code long before the first real MVME6100 would come together – in fact, even before the TSI148 chip went to fab. The only aspect of the code that the software team could not develop and test using the pre-proto board was the functionality associated with the VME bridge itself. To solve this the software team used the FPGA test board that behaved like the TSI148 chip itself.

As the hard and software engineers completed their testing, MVME6100 boards and MVME6100 software became available for integration test. A series of test cases to validate hardware functional requirements including front panel buttons, LEDs, CPU and memory devices, I2C and FLASH interfaces, WDT/NVRAM/RTC devices, serial and Ethernet I/O interfaces, system status registers and the PCI, PCI-X, and VME busses were performed. The test cases were based either on the MVME6100 firmware, VxWorks BSP (Board Support Package), or Linux BSP. Other test cases focused on specific firmware and OS requirements, such as boot and FLASH options, memory configurations and PMC options.

Several third-party PMC modules were tested on the MVME6100's 2 PMC sites, as well as on the PMC sites of Motorola's PMCspan module. The MVME6100 supports up to two PMCspan modules for a total of six PMC sites. The PMC modules were then exercised using the firmware or operating system software to ensure they functioned as intended. In a similar manner, the MVME6100 was used with other Motorola VME boards to demonstrate that the MVME6100 can successfully communicate across the VMEbus.

In addition to demonstrating that the product requirements have been met, the test teams conducted performance and stress testing. The performance testing focused on CPU performance, Ethernet performance, PCI-X performance and VMEbus performance. On the VMEbus, the MVME6100 demonstrated 300Mbytes/s memory-to-memory data transfers across a standard five-row backplane using the 2eSST protocol. These results were reported using VMETRO's 2eSST VMEbus analyser, and represent almost eight times improvement in previous non-2eSST VMEbus transfers.

Since some products can fail under stress, it was important to evaluate the operation of the MVME6100 under heavy load when multiple resources were simultaneously saturated. Numerous stress tests were conducted on the MVME6100 in the lab and in the environmental chambers as part of the environmental Design Validation Testing (eDVT). In eDVT, twelve MVME6100 boards were stressed under various environmental conditions for twelve days. On the first day, the boards were subjected to thermal shock and sinusoidal vibration tests. After this 'shake and bake' session, the boards were placed in a chamber for six days with temperature cycling continuously every two hours between -5°C and +60°C. The next phase of environmental testing was a 24-hour power cycle test where every five minutes power was cycled to the boards, while temperature cycled on the hour from -5°C to +55°C. Then external voltages coming into the board were margined and cycled along with temperature (-5°C to +55° C) for 4 days. To push the envelope and find the edges of the design, the voltages and frequencies on some of the boards were adjusted in a four-corners pattern: high frequency/high voltage; high frequency/low voltage; low frequency/high voltage; low frequency/low voltage.

During the stress testing and eDVT sessions the boards were running a VxWorks-based test suite known locally as the 'soak test'. The soak test is a collection of five tests run simultaneously in paired configurations exercising the major subsystems on the board (see Figure 2). The memory crawler exercised the CPU and its L3 cache, the 'north bridge', and memory. The GigE blaster/blastee ran bi-directional Gigabit Ethernet traffic between port 1 on both boards of a pair. The VME blaster/blastee ran bi-directional traffic across the VMEbus between both boards using the VxWorks shared memory feature. The continuous DMA test ran between a Motorola PrPMC880 in PMC slot 1 and main memory on one board, and between a PrPMC880 in PMC slot 2 and main memory on the other board of the pair. Finally, the Tempe DMA test ran continuous bi-directional VME traffic between the two MVME6100 boards, cycling through 2eSST, 2eVME, MBLT, BLT, and SCT VMEbus transfers.

Also used were external beta test customers to operate the MVME6100 in customer environments, running customers' application software. During beta test, customers were asked to use the MVME6100 either as a host controller in a VMEbus system, running either VxWorks or Linux, or as an intelligent PMC carrier, hosting one or two third-party PMC modules, and then performing data transfer (particularly 2eSST) and processing functions, running either VxWorks or Linux. Beta test results were used to confirm that the MVME6100 functioned as specified, met performance and reliability expectations, and was ready for general release.

So the many challenges involved in validating the quality and robustness of a product employing a large number of new technologies were met by creating specialised platforms to test critical sub-systems and facilitate concurrent development of hardware, software, and ASICs. The result will lend confidence to and bolster the success of the VME Renaissance.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.