Embedded motherboards are being delivered with a hoard of M.2 and Mini-PCI Express (PCIe) sockets. Usually they’re home to wireless adapters and solid-state drives (SSDs), including SSDs that support NVMe. Now there’s a different module to drop on these motherboards—Hailo’s Hailo-8 (Fig. 1).
The Hailo-8 is a compact chip designed for rugged embedded applications from transportation solutions to mobile devices. The chip delivers 3 TOPS/W with performance extending to 26 TOPS. The chip is built around a structure-defined dataflow architecture. This translates to a 682-frame/s processing speed for the ResNet-50 benchmark with 224- × 224-pixel resolution, 8-bit precision, and a batch size of 1.
Small batch sizes are important when processing streaming video. The chip also foregoes any external DRAM, handling everything on-chip. It’s available in automotive-grade meeting ASIL-B (D) and AEC-Q100 Grade-2 requirements. The chip can operate in a standalone fashion or as a coprocessor.
The new M.2 and Mini-PCIe modules allow the chip to operate as a coprocessor. The M.2 2242 M-Key has a x4 PCIe Gen 3 interface that’s designed to deliver a real-time, low-latency interface for AI/ML inference processing.
The modules have drivers for Linux and Windows. The Hailo dataflow compiler and Hailo-RT run-time support frameworks such as Caffe and TensorFlow in addition to the ONNX data-exchange format.
Hailo partnered with Foxconn, Network Optix, and Socionext to deliver the BOXiedge: Edge VMS Server (Fig. 2). This is essentially a standard embedded, rugged computer with a Hailo-8 M.2 module installed. It can handle up to 20 Ethernet-based camera streams simultaneously. The host processor is a 24-core Arm Cortex-A53 SoC. Furthermore, the system uses less than 30 W of power.
Overall, the Hailo-8 module provides significantly better performance than other modules like those based on Intel’s Myriad-Xand Google Edge TPU.