Unless the voice assistant boom driven by Google and Amazon tapers off, microphones could be embedded in everything from thermostats and refrigerators to wearables and headphones over the next few years. Tapping into the push to control almost anything with simple spoken commands, a number of companies are trying to train battery-powered devices to be better listeners.
Syntiant, a semiconductor startup founded last year by former engineering executives from Broadcom, is trying to slash the power required for always-on listening applications such as keyword spotting or speaker identification. The company is building chips based on 40-nanometer NOR flash memory that store machine learning models and perform operations in the same place. It plans to enter volume production in the first half next year.
The Irvine, California-based company is jumping into the market for chips aiming to be more efficient than CPUs, graphics processors (GPUs) and digital signal processors (DSPs). Unlike Nvidia, which dominates the training market, Syntiant is focused on running inference inside devices without shipping workloads to the cloud, potentially improving privacy, lowering latency and saving bandwidth. Like training, inference today usually takes place in data centers.
Syntiant, which has raised $5 million of venture funding from Intel Capital, Seraph Group, Danhua Capital and Embark Ventures to send samples to potential customers, needs more money to offer a complete chip. The company is trying to raise additional cash to ramp up production of its first product. Busch said it would be released in the first half of next year ahead of chips from Mythic, which also uses embedded flash memory to boost machine learning's power efficiency.
“There has been four decades of work done in massively parallel processing, but all that work in CPUs is not really that useful for deep learning,” said chief executive officer Kurt Busch. The company stores neural networks inside the chip, eliminating the power consumed by summoning data from expensive DRAM. By performing operations within flash memory, it cuts power consumption from milliwatts to microwatts—and without giving up the accuracy of the algorithm, said Busch.
Syntiant keeps data movement minimal by running inference in flash memory cells, the same memory cells store trained machine learning models. Hammered into the neural network are weights that determine the difference, for example, between someone saying “tomato” versus “potato.” The time it takes to access these weights in external memory is a major performance bottleneck on machine learning today.
Syntiant’s chip stores the weights as electrical charges, which alter the amount of current that flows through individual memory cells. The current powers multiply and accumulate—more commonly known as MAC—operations used in machine learning. Using analog currents instead of digital signals saves power that traditional processors use to switch voltages. The use of analog computing also enables lower precision.
Today, many machine learning applications use 32-bit precision for both training and inference workloads, Syntiant can support 4-bit and 8-bit precision to save power and handle larger networks. Its architecture, Busch claims, allows the new processor to handle 20 trillion operations per second for every watt. That contrasts with 400 billion operations per second per watt supported by chips based on Nvidia’s Volta architecture, said Busch.
There are differences between Syntiant and Mythic’s technology. Syntiant’s neural decision processor is all analog, while Mythic’s silicon is surrounded by digital circuitry that can be programmed to handle different types of neural networks—more than other chips, chief technology officer Dave Fick told Electronic Design in April. Busch declined to comment on the networks Syntiant's supports or how many weights can be stored inside it.
The companies are eyeing slightly different markets. Syntiant targets ultralow power applications like battery-powered sensors and wearables that can be given commands through the user’s voice. Mythic, which is funded by $55 million from investors including SoftBank Ventures and Lockheed Martin, is targeting image applications in drones and security cameras. Mythic expects to ramp up volume production before the end of next year.
Marketing the technology falls to the Broadcom veterans on its executive team. Pieter Vorenkamp, its former vice president of operations engineering, was hired as chief operating officer of Syntiant. Leading the software team is the former chief technologist of Broadcom’s switch business, Stephen Bailey. Vice president of hardware Dave Garrett is a former distinguished engineer at Broadcom, which is headquartered in San Jose, California.
They will have to outpace rival technology from Intel-owned Movidius—which targets the machine learning algorithms replacing traditional computer vision—and Nvidia—which has open-sourced the architecture inside its car processors. Syntiant is also battling companies like Qualcomm trying to optimize traditional chips for machine learning tasks. It could also compete with Knowles, the world’s largest microphone maker, which sells custom voice control DSPs.
Winning over customers may come down to simplifying the use of its technology. Syntiant is aiming lower the bar for programming its chips, building software tools to support its massively parallel analog computer. Busch said that customers can take networks trained in TensorFlow and load the weights directly into its processor as firmware. “The biggest hindrance to inference on the edge is programming the hardware,” he added.
“Most of the software engineers at semiconductor companies are second-class citizens, and hardware people are mostly non-existent at software companies,” said Busch. He added that simplified programming could trim costs for customers trying out the company’s chips. “We wanted to form a company that was half software engineers working on deep learning and half semiconductor designers.”
Syntiant has partnered with Infineon Technologies, which sells microphones that can be paired with the new chipset. The deal is focused on creating blueprints so that customers can avoid jumping through hoops to add voice control capabilities to new products. In addition to offering an out-of-the-box solution for embedded inference, Busch has previously said that the company would also assist customers looking for semi-custom solutions.