The Embedded Microprocessor Benchmark Consortium (EEMBC) has released a new benchmark that targets low power microcontrollers. This is to complement the wide range of benchmarks they already provide to embedded developers. For example, the CoreMark is a very popular benchmark whose results are often quoted by microcontroller vendors when talking about their products.
I got a chance to speak with Markus Levy about the latest benchmark and tools.
Wong: What is ultra-low power? Or I should say, how does EEMBC define ultra-low power, since I suppose this is a relative expression?
Levy: Ultra-low power can best be expressed in terms of the applications it relates to. Ultra-low power represents a growing class of products that could operate on a single coin-cell battery for up to 10+ years. Some of these applications include portable medical devices, security systems, building automation, smart metering, and applications using energy harvesting devices.
Related Articles
- Interview: Markus Levy Highlights The EEMBC Floating Point Benchmark
- Know Your Benchmarks Before You Make Comparisons
Wong: Why did EEMBC decide to target benchmarks for ultra-low power microcontrollers?
Levy: Most, if not all, of the ultra-low power applications contain an 8-, 16-, or 32-bit microcontroller. System designers must be able to choose the best microcontroller to fit their application requirements (including battery life). There hasn’t been an open, industry-standard way to measure microcontroller energy consumption. Furthermore, every microcontroller vendor claims that they have the lowest ultra-low power devices, and the datasheet parameters all vary from one vendor to the next – in other words, there is no ‘easy’ way to compare apples to apples.
This file type includes high resolution graphics and schematics when applicable.
Wong: I understand that EEMBC ULPBench consists of a series of tests?
Levy: That’s correct. You can view ULPBench (Fig. 1) as a family of benchmark tests, where the different versions will account for various aspects of CPU efficiency and energy consumption. To produce a benchmark that we could make available in a reasonable timeframe, initially we decided to produce a test that focuses on the microcontroller core functionality that encompasses the CPU, memory, and real-time clock. We call this the ULPBench-CoreProfile. Subsequent phases of development will include other facets of the microcontroller, such as the analog-to-digital converter, PWM, and SPI. But as you can imagine, the more facets we include in the benchmark, the more complex it becomes to port the benchmark code to the target device, the testing itself also becomes more difficult, so we want to make sure that we do it right and in a way that is representative of real-world applications.
Wong: What exactly is the ULPBench-CoreProfile?
Levy: First of all, I forgot to mention that all the benchmark tests in ULPBench are and will be portable and architecture independent across all microcontrollers and power modes, accommodating 8-, 16-, and 32-bit micros. Specifically, the ULPBench-CoreProfile performs a variety of functions commonly found in ULP applications; among them are memory and math operations, sorting, and GPIO interaction. EEMBC provides the source code; users then build the code with their compiler of choice. This in itself will be interested because it will demonstrate that the compiler will make a difference when it comes to the efficiency of any microcontroller. Using the same source code also ensure that all microcontrollers run the same workload. Just an FYI that today every microcontroller vendor uses a different workload when specifying the power and/or energy values in their datasheets. With the CoreProfile, the microcontroller uses its real-time clock (RTC) to establish a duty cycle, waking up once per second and running the workload. The ULPBench measures the energy consumption for 10 seconds, accounting for both active and low power operating modes.
Wong: You mention that ULPBench measures the energy consumption. Does EEMBC have a standardized methodology?
Levy: Yes. To go along with our family of benchmark tests, EEMBC created a tool called the EnergyMonitor. It’s a small PCB board that attaches to a PC and the device under test. EnergyMonitor includes an integrated GUI for convenient data capture and display.
Wong: There are many tools on the market, why did EEMBC go through the trouble of developing special hardware to support ULPBench?
Levy: When we first conceived of this project, the ULPBench working group members decided that the maximum tolerable price of the equipment required to execute this benchmark should be approximately $50-$100 US. Prices for commercially-available tools, range from about $500 (for a 20MHz oscilloscope) to $20000 (e.g. 1GHz oscilloscope+probes), but for what we want to accomplish, these are overkill and more complex to use. Several microcontroller vendors already include power measuring capabilities on their inexpensive evaluation boards, so it could have been an option to certify and/or standardize their methodologies. However, this would detract from our goal of standardizing on a methodology that anyone could consistently replicate.
Wong: Typically in benchmarking ±5% is not so critical, what is the tolerance range for accuracy with ULPBench?
Levy: While an accuracy of ±5% is not critical for many applications (in terms of power and/or performance), it can have a significant impact on battery life for many ultra-low power applications. The EEMBC EnergyMonitor has a typical accuracy of +/-1% for currents under 1mA, and +/-2% for currents over 1 mA. This high degree of accuracy will be important for vendors in order to substantiate the energy values in their datasheets.
Wong: Drawing from the adage, if you build it they will come – do you really believe that the microcontroller vendors will adopt the ULPBench methodology?
Levy: Good question, because it’s been a lot of work putting together this new benchmark. Fortunately, the EEMBC members have been tremendously supportive in developing ULPBench. We have had very active support from Analog Devices, ARM, Atmel, Cypress, Silicon Labs, Freescale, Microchip, Renesas, Spansion, and STMicro - with TI chairing the working group. Actually, one of the more popular working groups in EEMBC history. It’s been quite impressive how motivated these vendors have been to create a standard for ultra-low power measurement, despite the fact that in benchmarking there can only be one winner. Of course, winning at a benchmark has many permutations which include performance, energy, cost, tool support, on-chip peripherals, and so on.
Wong: How can someone order ULPBench and the EnergyMonitor?
Levy: Until April, the EnergyMonitor is available in limited quantities. Initial cost is $75 US. Contact Markus Levy to place your order. Beyond that, we are working with a distributor who will help us with worldwide sales. One other point worth mentioning is that the EnergyMonitor, while designed to integrate with ULPBench, can be used to measure energy on virtually any application where current consumption ≤28mA with a 3 Volt supply. This makes it useful for various IoT applications and even for measuring sensor energy consumption.
Wong: When will the first ULPBench results be available?
Levy: EEMBC is working on a user interface, similar to what we provide for CoreMark, that will allow users to submit results, as well as other data that is pertinent to reproducing the benchmarking environment (i.e. device information, compiler details, timer configuration, active mode configuration, inactive/sleep mode configuration, operating conditions). Several EEMBC members already have plans for having their devices certified with ULPBench, that will enable them to include the results on their datasheets.