(Thoughts on the Accuracy Limits of Scientific Calculators...)

How willingly we trust our calculators! Yet, like everything else, these ubiquitous tools do have limits. In particular, their accuracy limits are beginning to show in our ever more complex and precise engineering calculations. To illustrate the problem, here's a simple calculation that you can work on your own calculator:

((1 − (cos (3/7))2 − (sin (3/7))2 )2)0.25 = 0 ????

Remembering the trigonometric identity for squared sines and cosines, you know the answer has to be zero. But your 10-digit calculator probably gives an answer on the order of 4 x 10-6, a much larger discrepancy than you would expect from the calculator's claimed 10-digit resolution. This error, of course, comes from accumulated truncation and round-off errors in the transcendental algorithms—the computational version of NOISE.

Another source of squirrelly readings is the low voltage levels used in modern calculators. The MOSFET ICs' inherent analog underpinnings become apparent at low supply voltages, where channel voltages may vary only 10:1 between the 1 and 0 digital states. In essence, each FET is a linear amplifier at both of its digital extremes. As a linear amplifier, it can pick up and amplify strong RF or pulsed fields. For some plastic-cased models, operation near a terminal's horizontal transformer, a radar transmitter, or a medium-frequency broadcast station can jump the display reading without any apparent cause.

The calculator IC's analog roots are, in fact, the reason why you can't buy portable scientific calculators with better than 12-digit precision. It becomes a matter of voltage regulation. The battery's voltage drops about 40% as its energy is drained. Ordinary calculator chips have Power-Supply Rejection Ratios on the order of 180 to 210 dB, allowing 8 to 10 digit calculators to operate without error (10(200/20) dB = 1010 = 10 digits). Leading-edge companies like HP and TI use voltage regulators and factory calibration to gain another 60 dB of suppression, bringing dependable performance to the 13-digit level. But that's about it for portable units.

Although careful analog design keeps PSRR errors below digital significance in scientific calculators, those techniques can't provide the high precision needed in accounting calculators. Consequently, calculating a really large number— such as the bottom line for the second overrun of a military hardware contract—will show some variable errors in the last digits on a 14-digit accounting calculator. In fact, aerospace accountants are rumored to earn their lunch money simply by changing to fresh calculator batteries before the final column addition. Fortunately for them, "calculator faith" hides these small transgressions from the GAO (Government Accounting Office).

The accounting calculators represent a large market, and their accuracy demands will likely fuel research for improved battery and regulator technology in the next decade. Japanese researchers hope to produce accurate 14-digit machines by the turn of the century, spurred by suspicions of low-battery use when converting trade-balance dollars to yen. Despite these incremental improvements, however, some analog experts smugly hint that digital engineers will never design a calculator accurate to one part in 1016.

Analog engineers in the know infer that the Digital Illusion can't be supported beyond 16 digits. At that precision, the digital two-state simplicity collapses and all circuits revert to their basic analog nature. The prima donna digital engineers must then face the dirty "real world" uncertainties that we analog engineers face every day. Just as playtime for digital bus design ended at 20-MHz data rates, the Digital Illusion ends at 16 digits.

The analog engineer—always thoughtful, physically attractive, and suave—doesn't base the Digital Illusion's limit on mere speculation. Nay, this limit has roots deep in theoretical physics, a subject quite familiar to the analog engineer's restless intellect. In fact, it's the famous Heisenberg Uncertainty Principle that imposes an absolute 19-digit limit on digital-calculator accuracy. In 1927, as many experienced analog engineers will recall, Heisenberg recognized that the minimum energy kicked into a system (when making a measurement) is inversely proportional to the measurement time. That is, dE x dt can never be less than Planck's constant, 4.14 x 10-15 electron volts per Hertz (eV/Hz). Consequently, the longer you take to do something, the less disruptive energy is injected, and the more exact the result.

The occurrence of an Uncertainty Error is, of course, probabilistic. It can occur in any calculation, as students of engineering quickly discover, but it is much larger and more likely in calculations where the disruptive energy is large—i.e., where the calculation time is short (or when you are in a big hurry). Consider some facts from your own experience...calculations done on a Cray super-computer at gigaflops rates (earthquake prediction, rainfall prediction, national debt prediction, origins of the universe) are subject to great uncertainties, whereas computations done on a slide rule at deciflops rates (resistor values, the price of 12 op amps) are seldom wrong by more than 5%. These Uncertainty Ratios remain valid even at microflop rates. For example, computer programs that require several man-years of debugging are much more reliable than those that work the first time.

Calculating the 19-digit limit is beyond the scope of this short article, but is recommended as an exercise for the Gentle Reader. Start with Maxwell's equations. Use a 3.6-V lithium battery, 2N3904 transistors, and 99.2 eV (12.648 nm) lithography to establish boundary conditions (savvy analog engineers will use their slide rule's div, curl, and grad scales to make quick work of the vector differential equations). Your answer may differ somewhat from 19, depending on how much you rush the calculations. If you must use a digital computer, analog engineers amicably recommend that you stick with an Apple I, Altair 680, IMSAI 8080, or PDP-8 models to minimize spurious answers.

I hope this peek into calculator theory will dispel the blind trust in calculator and computer results, and in all that complex digital stuff. Now that you've been alerted to a future of continuing digital problems, you will not be surprised when your naive colleague's new 17-digit calculator sinks into Digital Nirvana when trying to pin down the penultimate digit. You will know that calculators, like everything else, face limits. The Digital Illusion can't shelter us indefinitely.

Although the affable, warm-hearted analog engineers have used all of the technology at their disposal to stretch the Illusion over the last 40 years, their numbers dwindle, and the digital engineer's age of innocence must end. A moment of silence, please, for those generous analog folks who have worked so continuously and indiscreetly during these long, hard years....

Now just a couple of final comments from RAP: While I would love to say that I wrote this, I must give all the credit to James A. Kuzdrall, P. E., Chief Engineer at Intrel Service Co., P.O. Box 1247, Nashua, NH 03061; e-mail: [email protected] When I saw Mr. Kuzdrall's draft of this, I knew it was a perfect choice for this guest editorial. He's a man after my own heart.

And, lastly, April Fool!

Comments invited! / RAP

Robert A. Pease / Engineer

Originally published in ELECTRONIC DESIGN, April 2, 1992

RAP's 2000 comments: In addition to the letters from guys who enjoyed this hoax, I also got a couple letters from overseas, asking exactly what formula they should use to compute this 19-digit limit. I had to write a very polite letter, to explain what kind of April Fools joke this was. I asked some friends, "Haven't they heard of April Fools jokes in France, or Japan?" People told me, yes, these are generally well known. But apparently a couple guys were a little too slow (or too serious) to catch on... —rap