The Need for Conformance Testing

LXI as a standard continues to support the latest technical developments in LAN products and techniques. This is particularly evident in the adoption of the newest, most convenient discovery mechanisms.

However, changing standards present challenges to companies developing LXI instruments. To help them in two ways—to give guidance on design issues and to officially certify that an instrument meets all the qualifications—the LXI Consortium regularly runs PlugFests where members of an independent testing laboratory examine new products to verify full compliance with the spec.

During PlugFests, the Compliance Working Group plays an advocacy role. Rather than being adversarial, the group makes every attempt to help LXI instrument manufacturers understand what is required and gives them tips on how to reach these goals.

For most of the PlugFests, the Conformance Working Group has seen many of the same problems repeated from product to product. The LXI spec includes a multitude of details, some of which are easy to overlook. Many of these are simply small oversights that can be corrected quickly. If a problem is identified early during a PlugFest and the fix is relatively quick and easy, which often is the case, a manufacturer still has time to get an instrument certified the following day.

Version History

Before looking at specific problems, let's consider the evolution of the specification. In September 2005, the LXI Consortium released Version 1.0 of the LXI standard. Just one year later, Version 1.1 followed with minor corrections and clarifications.

In October 2007, the consortium adopted Version 1.2, and its major focus deals with discovery mechanisms. This means that when you plug an instrument into an LXI system, it automatically is recognized and registered as part of the test system so the user and other instruments can work with it.

Specifically, LXI 1.2 includes enhancements to support plug-and-play identification of LXI devices. This feature, in addition to the existing mandatory identification scheme based on VXI-11, is defined as a future rule and will become a requirement with Version 1.3 scheduled for release this month.

With the new multicast domain name system (mDNS) and DNS Service Directory (DNS-SD), the combination of both better known as the open-source protocol Bonjour from Apple, installation and automatic recognition of LXI devices are as easy as connecting a printer to a PC. The identification information for an LXI device is transferred via an XML file, which provides much more information than the VXI-11 protocol with the *IDN? query command.¹

Currently, the LXI Consortium is working on other essential parts of Version 1.3. Key among them is the incorporation of the 2008 version of IEEE 1588 for synchronizing time among instruments. 1588-2008 was approved by the IEEE last March and is the successor of the 1588-2002 Precision Time Protocol (PTP) standard (see sidebar).

The IEEE 1588-2008 PTP

LXI Class A and Class B devices require the implementation of the IEEE 1588 Precision Time Protocol, which is used to synchronize real-time clocks in modules of a networked distributed system down to the nanosecond range.

The synchronization process consists of two phases:
• First, offset correction is performed. It defines the time difference between master and slave clocks. For that purpose, the master periodically sends a synchronization (Sync) message to all slaves. In parallel, the time at which the message is sent by the master is measured as precisely as possible and preferably with hardware support. The master then sends a second FollowUp message with the exact transmission time of the corresponding Sync message to the slaves. The slaves measure the receive time of these messages, allowing them to calculate the correction value (offset) to the master. The slave clock then corrects the offset to the master clock. If the transmission were to have no delay, then both clocks would be synchronized.
• The second phase of synchronization determines the delay of messages between slave and master. This is done through Request Delay and Delay Response messages in a similar way, and the slave clocks are adjusted accordingly.

Since the introduction of PTP, interest in its capabilities has grown rapidly in many industries. A wide range of applications and different requirements in the areas of process control, telecommunications, power distribution, and test and measurement led to Version 1588-2008, which incorporates two synchronization protocols and additional extensions.

The interaction of the clocks on the network is characterized by 1588 profiles, with 1588 parameters optimized for specific applications. LXI defines such a profile to which all future LXI Class A and Class B devices must comply. Consequently, 1588-2008 is no longer backwards compatible to 1588-2002.

Looking Ahead

Starting with Version 1.3 of the LXI standard, the LXI Consortium will adopt the new 1588 version as quickly as possible so systems using LXI Class A and Class B devices will synchronize with each other based on the 1588-2008 PTP. We expect to see a significant number of new LXI devices conforming to LXI Classes A and B shortly after the release of Version 1.3.

The next really big step will be Version 2.0 that, in addition to the mandatory XML-based discovery, will incorporate even more significant changes. These functionalities include the extension of LXI device Web pages to support instrument configuration and interactive testing of different LXI trigger capabilities such as LAN peer-to-peer messages, IEEE 1588 time events, or the wired trigger bus of LXI Class A devices. Logging of all the events in LXI devices will improve as will the ease of troubleshooting LXI-based test systems.

Resource Management is another major extension that enables management and allocation of LXI devices in a network with more than one controller accessing instruments. Yet another working group is dealing with the standardization of script downloads into LXI devices. Execution of custom scripts can be done without the system controller, simplifying the development of test software and increasing the throughput of test systems.

PlugFests and Certification

The members of the LXI Consortium meet roughly three times a year at PlugFests held around the world. Previous meetings have taken place in California, China, Munich, and Canada, and the next is scheduled for Munich in October.

In closed sessions, future extensions to the LXI standard are discussed during face-to-face meetings in working groups. However, the public is invited to attend tutorials intended for both users and manufacturers interested in joining the LXI ranks.

One important function of a PlugFest is to help LXI instrument manufacturers deal with changes in the specification, test implementations, and certify new products as LXI conformant. An independent testing lab hired by the LXI Consortium to provide its services during the PlugFest is available without charge to all LXI Consortium members.

The last LXI PlugFest was held in Toronto, and a major effort was testing various LXI Class B implementations to the 1588-2008 specification. In addition, the definitions of the LXI 1588 profile were verified and refined.

These PlugFest findings were important for the development of test cases for the certification of Class B devices. A first set of test cases has been defined and will be used by the LXI Conformance Working Group for the extension of the LXI Conformance Test Suite (CoTS), a tool for vendors to test the LXI implementation prior to actual conformance tests. The software, which covers all the required rules and sections of the LXI standard for Class C, B, and A devices, can be downloaded from the Members Area of the LXI Consortium website.

To date, more than 516 instruments from 14 vendors have been certified as LXI compliant. And in the course of conducting its work, the Conformance Working Group has been able to track which issues occur most frequently during testing (Figure 1).

Figure 1. Common LXI Conformance MistakesSource: Wheelwright Enterprises

The Most Common Problem Areas

Interestingly, the leading areas of potential pitfalls have nothing to do with the instrument's technical capabilities but rather are administrative in nature. In addition, as manufacturers become more familiar with the spec and bring their next instruments in for testing, the number of problems has been dropping significantly.

Topping the list are issues that arise because the IVI driver for a potential LXI-compliant instrument must be registered at the IVI Foundation website. These are the drivers that allow users to program instruments using a variety of application-development environments. Some vendors prefer not to register their drivers until they have verified instrument functionality at a PlugFest. In this case, the Conformance Working Group can grant preliminary acceptance of the instrument conditional upon the driver being registered at the website.

Another frequent problem deals with the validation of the Web pages that are part of an LXI instrument's firmware. These Web pages must be in compliance with definitions from the World Wide Web Consortium (W3C). It often happens that manufacturers focus on the instrument and the LXI implementation and then make last-minute changes or additions to their websites that break compliance with W3C requirements.

Frames are another common concern. It's not enough to inspect the content of a Web page; manufacturers sometimes forget to check the frame page itself. Such problems can be easily fixed because these Web pages are easy to test and repair.

Manufacturers can take advantage of the tools on the W3 website to check the compliance of their Web pages. The principle tool used for Web-page conformance can be accessed via an online markup validation service.

DNS hostname issues also are frequent. Routines in instrument firmware make it possible for the device to talk to a net server and have it assign a meaningful name, such as LXI_DMM_1, so it's not necessary to work with IP addresses. The net server must, for instance, verify that a given name still is available and then assign it to the specific instrument. However, sometimes this mechanism doesn't work for new LXI instruments, or the hostname isn't properly displayed on the LXI instrument's home page.

The LXI standard dictates that each instrument must have an LED that indicates LAN status and activity, and it also provides a state diagram of what the LED must display under certain conditions. For example, LED requirements might be slightly different depending on if DNS is performed or if a device is using Auto-IP or manual IP.

In addition, a user must be able to toggle the LED from the LXI instrument Web page to make it easy to identify a specific device sitting in a rack full of devices. In 11% of the cases, the LED functions do not adhere exactly to requirements in the LXI standard.

Another area that frequently needs fixing is a provision for a LAN Reset LAN Configuration Initialization (LCI). A user must be able to push a button to force the unit to return to its default status. In some instruments submitted for compliance testing, this functionality is either missing or doesn't work properly.

Almost as many users have trouble with their IP configurations. The LXI standard recognizes two styles of IP configuration: an Auto/Manual selection, where Auto includes both DHCP and Auto-IP; and three separately enabled selections: DHCP, Auto-IP, and Manual. In the latter case, all three selections must have a check box or equivalent to be conformant. Sometimes, developers forget to include all of the required configuration fields that make for good citizenship in the LAN environment.

A few issues deal with labeling and, although seemingly mundane, are nonetheless important. More times than you might expect, the LXI version displayed on an instrument's Web page does not actually correspond to the version of the LXI specification the instrument complies with.

Or, manufacturers forget to add the mandatory label on the LAN connection that indicates if it does not support automatic medium dependent interface crossover (MDIX) where the instrument can detect if the LAN cable is a standard cable or a crossover cable as needed for peer-to-peer communications. This gets tricky because 1-GB Ethernet in general needs MDIX support where 10-/100-MB Ethernet does not.

Similarly, the LXI specification requires that the LXI logo appear on an instrument's front panel or display, and the instrument's media access control (MAC) address must be clearly visible. These are details that sometimes are forgotten in the heat of a development project with a tight deadline.

PlugFest Coming in October

Following the PlugFest planned for the end of October in Munich, there should be a large number of LXI introductions. And companies that want to get LXI products certified—or simply advice on how to more easily design an LXI instrument—should be sure to attend this or a subsequent PlugFest.

Reference

1. Barendt, N., “LXI 1.2 Improves Discovery and Identification,” LXI ConneXion, February 2008, pp. 12-16.

About the Authors

Jochen Wolle is the head of R&D for software, spectrum, and network analyzers at Rohde & Schwarz. He is a member of the LXI Board of Directors and chairman of the LXI Conformance Working Group. Rohde & Schwarz, P.O. Box 801469, D81671 Munich, Germany, e-mail: [email protected]

Lynn Wheelwright received a B.E.S. and an M.S.E.E. from Brigham Young University and then joined Hewlett-Packard and Agilent Technologies where he worked for 35 years. His honors include five patents relating to instrumentation and measurement techniques. Mr. Wheelwright also has participated in many standards groups and committees including the LXI Consortium and the IVI Foundation Spectrum Analyzer Working Group. He now runs his own consulting company. Wheelwright Enterprises, 3751 Porter Creek Rd., Santa Rosa, CA 95404, e-mail: [email protected]