Machine vision has received a major boost from artificial-intelligence (AI) technology, including deep neural networks (DNNs) and machine learning (ML). The challenge in implementing vision applications is that these techniques are performance-hungry. Also, latency in many application areas like automotive advanced driver assistance systems (ADAS) is a critical factor in system design.
Xilinx’s reVISION system (Fig. 1) looks to blend FPGAs with machine vision, putting it in competition with multicore CPUs, GPGPUs, and specialized ML hardware. FPGAs have many advantages in this space because of the parallel nature of many of the vision algorithms.
1. Xilinx’s reVISION looks to blend FPGAs with machine vision for applications from robotics to ADAS.
The difficult thing for developers is the complicated nature of FPGA design. This has been changing, though, as the tools have improved over the years. In fact, many are using FPGAs without dealing with RTL or logic design. Software developers can already take C, C++, and OpenCL code and put it into FPGAs, gaining significant performance benefits compared to executing the code on a CPU or GPU.
Applying these FPGA development methods could achieve a six-fold improvement in images/s/W for ML inferences on an FPGA and a 40X improvement in frames/s/W for computer vision processing. Simultaneously, latency reduces by a factor of 5 compared to CPU and GPU implementations. Of course, these improvements are only available if developers can program the FPGAs.
2. The reVISION tools and stacks are designed to minimize user development for ML and DNN applications.
This is where reVISION comes in. It takes advantage of Xilinx’s SDSoC environment (see “Design Tool Streamlines SoC FPGA Application Creation”) that’s built on the Vivado IDE. It uses a building-block approach and takes advantage of standardized interfaces such as AMBA to link configurable blocks together. These, in turn, can be accessed by on-chip hosts, as with Xilinx’s processor-enhanced Zynq FPGA line, or off-chip hosts. The reVISION stack extends this with DNN and ML tools and blocks (Fig. 2), and allows algorithms to be split between CPU/GPU and FPGA on platforms like the ARM-based Zynq.
The reVISION stack supports algorithm standards such as OpenCV, as well application development support like Berkley’s Caffe and Kronos’ OpenVX vision processing system. For example, Caffe models can be easily incorporated with reVISION.
