Inference Engine Squeezes onto M.2 and Mini PCIe Modules

Oct. 21, 2020

Delivering performance of up to 26 TOPS, the Hailo-8 machine-learning accelerator fits easily on M.2 and PCIe modules.

Embedded motherboards are being delivered with a hoard of M.2 and Mini-PCI Express (PCIe) sockets. Usually they’re home to wireless adapters and solid-state drives (SSDs), including SSDs that support NVMe. Now there’s a different module to drop on these motherboards—Hailo’s Hailo-8 (Fig. 1).

1. Hailo’s Hailo-8 machine-learning accelerator chip fits onto an M.2 or Mini-PCIe board.

The Hailo-8 is a compact chip designed for rugged embedded applications from transportation solutions to mobile devices. The chip delivers 3 TOPS/W with performance extending to 26 TOPS. The chip is built around a structure-defined dataflow architecture. This translates to a 682-frame/s processing speed for the ResNet-50 benchmark with 224- × 224-pixel resolution, 8-bit precision, and a batch size of 1.

Small batch sizes are important when processing streaming video. The chip also foregoes any external DRAM, handling everything on-chip. It’s available in automotive-grade meeting ASIL-B (D) and AEC-Q100 Grade-2 requirements. The chip can operate in a standalone fashion or as a coprocessor.

The new M.2 and Mini-PCIe modules allow the chip to operate as a coprocessor. The M.2 2242 M-Key has a x4 PCIe Gen 3 interface that’s designed to deliver a real-time, low-latency interface for AI/ML inference processing.

The modules have drivers for Linux and Windows. The Hailo dataflow compiler and Hailo-RT run-time support frameworks such as Caffe and TensorFlow in addition to the ONNX data-exchange format.

Hailo partnered with Foxconn, Network Optix, and Socionext to deliver the BOXiedge: Edge VMS Server (Fig. 2). This is essentially a standard embedded, rugged computer with a Hailo-8 M.2 module installed. It can handle up to 20 Ethernet-based camera streams simultaneously. The host processor is a 24-core Arm Cortex-A53 SoC. Furthermore, the system uses less than 30 W of power.

2. Hailo, Socionext, and Foxconn worked together to create the BOXiedge: Edge VMS Server that can handle 20 camera streams simultaneously.

Overall, the Hailo-8 module provides significantly better performance than other modules like those based on Intel’s Myriad-Xand Google Edge TPU.

About the Author

William G. Wong | Senior Content Director - Electronic Design and Microwaves & RF

I am Editor of Electronic Design focusing on embedded, software, and systems. As Senior Content Director, I also manage Microwaves & RF and I work with a great team of editors to provide engineers, programmers, developers and technical managers with interesting and useful articles and videos on a regular basis. Check out our free newsletters to see the latest content.

You can send press releases for new products for possible coverage on the website. I am also interested in receiving contributed articles for publishing on our website. Use our template and send to me along with a signed release form.

Check out my blog, AltEmbedded on Electronic Design, as well as his latest articles on this site that are listed below.

You can visit my social media via these links:

I earned a Bachelor of Electrical Engineering at the Georgia Institute of Technology and a Masters in Computer Science from Rutgers University. I still do a bit of programming using everything from C and C++ to Rust and Ada/SPARK. I do a bit of PHP programming for Drupal websites. I have posted a few Drupal modules.

I still get a hand on software and electronic hardware. Some of this can be found on our Kit Close-Up video series. You can also see me on many of our TechXchange Talk videos. I am interested in a range of projects from robotics to artificial intelligence.