Processor and Software Innovations Empower Edge AI Applications

Members can download this article in PDF format.

Generative artificial intelligence (AI) running on powerful cloud-based centralized data centers garners all of the headlines today. But AI is also coming to the edge—the precise location that collects data and initiates a response.

Edge devices incorporating low-power microprocessors that can execute AI algorithms has the potential to enhance our lives in our homes, cars, offices, and factories without compromising cybersecurity. Edge AI can work within a local environment and needn’t exchange information with the cloud, minimizing the risk of data being compromised while bringing real-time intelligence and responsiveness to everyday activities.

Reliable and Secure Pattern Recognition

“AI is about really fast pattern recognition that is reliable and secure,” said Artem Aginskiy, a product-line manager at Texas Instruments. “And if you’re enabling AI at the edge, nothing has to leave the device, so it strongly supports privacy.”

TI envisions a world where an AI-enabled sleep-cycle monitor optimizes your wake-up time; a smart refrigerator helps plan your lunch by scanning its contents for available ingredients and making recommendations in accordance with any dietary restrictions; and your car recognizes you, automatically unlocking, optimizing your seat adjustment, and tuning in to your infotainment preferences. The need for cybersecurity in such a world is clear. For example, you would not want to risk compromising healthcare information such as sleep-cycle data and dietary restrictions.

Security is just one aspect of edge AI—responsiveness is another. Edge AI applications employ advanced analytics to allow decision-making in the moment, boosting productivity and safety. For example, sensors throughout a factory may continuously monitor equipment to predict faults. An edge AI system could monitor and analyze the resulting data and immediately take action to avoid catastrophic failures, avoiding the latencies and other delays inherent in accessing cloud resources.

Edge AI Embedded Processors

With advances in chip design, embedded processors with integrated hardware accelerators are emerging that provide the necessary performance, speed, and energy efficiency to enable affordable edge AI solutions.

TI offers a line of Arm Cortex-based processors that can execute facial recognition, object detection, pose estimation, and other AI functions to bring real-time intelligence to smart-camera systems. Scalable from one to 12 cameras, these processors serve applications such as battery-powered systems (e.g., smart security cameras and vacuum robots); machine-vision systems for industrial automation; professional surveillance systems; and other systems requiring high-performance autonomy, sensor fusion, and functional safety.

Specific processors include TI’s AM62Ax, which targets battery-powered systems and operates at 2 teraoperations per second (TOPS). It offers a resolution of 5 Mpixels and a 4K frame rate of 30 fps. An additional processor, the AM68A, targets real-time retail and factory-automation systems, featuring 8-TOPS performance, 12-Mpixel resolution, and a 60-fps 4K frame rate. Both the AM62Ax and the AM68A include one deep-learning accelerator, one video encode/decode accelerator, and one vision preprocessing accelerator.

Finally, the AM69A targets high-performance sensor-fusion systems such as autonomous mobile robots. It delivers 32-TOPS performance and 12-Mpixel resolution while supporting 60-fps frame rates in two simultaneous streams. The device includes one video encode/decode accelerator, two vision preprocessing accelerators, and four deep-learning accelerators.

To help you get started, TI has an evaluation module for each of these three processors. Figure 1 shows the SK-AM69 evaluation module, which includes connectors for up to 12 cameras.

Software Development

For software development, TI offers Edge AI Studio—a collection of tools for rapidly building edge AI applications on TI embedded devices. The GUI-based tools provide a “low-code” or even “no code” environment that allows you to develop and test AI models even if you lack extensive AI programming experience.

“Texas Instruments is democratizing AI so any developer in the world can develop an intelligent system,” said Aginskiy. “You can use the tool to train and deploy AI models without ever writing a line of code.”

The tools include Model Composer, a fully integrated no-code environment that trains and compiles models for deployment into edge AI applications. It supports a “bring your own data” development approach (Fig. 2), which allows you to retrain models from the TI Model Zoo using your own custom data.

Model Zoo is a collection of deep-learning models for various computer-vision and deep-learning tasks, including image classification, segmentation, and detection. It supports models in several formats, including ONNX from the Open Neural Network Exchange; Caffe, developed by Berkeley AI Research and community contributors; TensorFlow, a free open-source software library for AI and machine learning; and PyTorch, a project of The Linux Foundation.

You can deploy your retrained models from the Model Zoo on your own TI development board and view a live preview of how your trained models perform. Figure 3 illustrates Model Composer’s object-detection and image-classification capabilities. It shows the operation of an object-detection model trained to detect objects commonly found on an engineer’s desk, including launch pads, phones, and power adapters. As each object is placed on the desk, the model displays a box around the detected object and assigns the appropriate classification label.

TI also offers Model Analyzer, a free online service that allows for the evaluation of accelerated deep learning on remotely accessed development boards. With Model Analyzer, you simply login, deploy a custom model, and obtain a variety of performance benchmarks, including information on latency, frames-per-second processing, DDR bandwidth, and accuracy.

Conclusion

High-performance, affordable embedded processors are emerging that let you build and deploy scalable, efficient, and secure edge AI applications. TI offers a line of embedded processors with hardware accelerators that can quickly execute AI algorithms, and it provides software-development tools and evaluation boards that can help you begin exploring edge AI opportunities and bring new edge AI devices to market quickly.