What do you trust? These days, many people trust the Internet. When they have a question, they often "Google" the answer. Unfortunately, many Web surfers take the results they get as if they came from an infallible oracle. This wouldn't be too bad if the results of most online searches were good. In most cases, though, your surfing will resemble archeology because of the amount of digging (not digg digging) required to find your treasured answers.
A great deal of information on the Internet is valuable and accurate. Lots of it has been researched and edited. But when a simple search yields tens of thousands of results, searching becomes a matter of filtering to reduce the total number of entries or slugging through as many as possible in hopes of finding something useful.
Developers run into this same problem when they use trace tools to debug applications. Trace tools generate a tremendous amount of information that's more accurate than a typical Google search, but most if not all of the information will be totally irrelevant. Often, less than 1% will be meaningful or useful. Like the Googler, developers need to apply filters to reduce the the amount of clutter to find that 1%. Sometimes this information will be all that is necessary. Other times, it's simply a pointer to more interesting or more useful data.
Of course, the big difference between the developer and the Googler is context. Context within the search results is relevant for the developer because each entry depends on the prior entry. In fact, some trace hardware and software will create a log that simply notes non-sequential changes of the program counter register since the default case for an instruction execution is to increment the register, reducing the size of a trace log.
This reduction can be quite significant since no trace delivers 100% of an application's environment. The best it could do is 100% of the electronic environment, and even here, most trace systems only deliver a small fraction of a system's state. The minimalist approach only records the program counter, but even this may be all a developer needs to get enough information to swat a bug.
The big problem with tracing for newbies is the same as it is for most Googlers—the resulting information overload. There is simply too much to wade through, and frustration sets in. It's interesting to note that Googlers tend to have more advanced filtering options than developers have in trace programs, but most Internet search engines don't allow refined filtering on the set of results. Reducing the size of a subset often is an indication of how close you are to useful information.
Alas, few people delve beneath the top layer of filtering when it comes to the Internet and trace tools. Luckily for developers, trace tools like Green Hills Software's Time Machine (see "IDEs of Change") and LynuxWorks SpyKer (see "Tool Debugs Linux Kernels Without Instrumented Code") provide predefined and stored filters. Typically, they can locate situations like blocked or deadlocked threads.
Developers are more prone to use tracing these days for many reasons, but Internet searches have had some impact on getting them used to wading through a sea of information. The techniques aren't identical, but they are similar.
So how much do you trust the Internet? Or your trace program ?