FPGAs, AI, and Reconfigurable Vision Systems

April 7, 2017
Vision processing, even with artificial-intelligence support, isn’t an easy task, but it could become simpler with a system that blends FPGAs with machine vision.

Machine vision has received a major boost from artificial-intelligence (AI) technology, including deep neural networks (DNNs) and machine learning (ML). The challenge in implementing vision applications is that these techniques are performance-hungry. Also, latency in many application areas like automotive advanced driver assistance systems (ADAS) is a critical factor in system design.

Xilinx’s reVISION system (Fig. 1) looks to blend FPGAs with machine vision, putting it in competition with multicore CPUs, GPGPUs, and specialized ML hardware. FPGAs have many advantages in this space because of the parallel nature of many of the vision algorithms.

1. Xilinx’s reVISION looks to blend FPGAs with machine vision for applications from robotics to ADAS.

The difficult thing for developers is the complicated nature of FPGA design. This has been changing, though, as the tools have improved over the years. In fact, many are using FPGAs without dealing with RTL or logic design. Software developers can already take C, C++, and OpenCL code and put it into FPGAs, gaining significant performance benefits compared to executing the code on a CPU or GPU.

Applying these FPGA development methods could achieve a six-fold improvement in images/s/W for ML inferences on an FPGA and a 40X improvement in frames/s/W for computer vision processing. Simultaneously, latency reduces by a factor of 5 compared to CPU and GPU implementations. Of course, these improvements are only available if developers can program the FPGAs.

2. The reVISION tools and stacks are designed to minimize user development for ML and DNN applications.

This is where reVISION comes in. It takes advantage of Xilinx’s SDSoC environment (see “Design Tool Streamlines SoC FPGA Application Creation”) that’s built on the Vivado IDE. It uses a building-block approach and takes advantage of standardized interfaces such as AMBA to link configurable blocks together. These, in turn, can be accessed by on-chip hosts, as with Xilinx’s processor-enhanced Zynq FPGA line, or off-chip hosts. The reVISION stack extends this with DNN and ML tools and blocks (Fig. 2), and allows algorithms to be split between CPU/GPU and FPGA on platforms like the ARM-based Zynq.

The reVISION stack supports algorithm standards such as OpenCV, as well application development support like Berkley’s Caffe and Kronos’ OpenVX vision processing system. For example, Caffe models can be easily incorporated with reVISION.

About the Author

William G. Wong | Senior Content Director - Electronic Design and Microwaves & RF

I am Editor of Electronic Design focusing on embedded, software, and systems. As Senior Content Director, I also manage Microwaves & RF and I work with a great team of editors to provide engineers, programmers, developers and technical managers with interesting and useful articles and videos on a regular basis. Check out our free newsletters to see the latest content.

You can send press releases for new products for possible coverage on the website. I am also interested in receiving contributed articles for publishing on our website. Use our template and send to me along with a signed release form. 

Check out my blog, AltEmbedded on Electronic Design, as well as his latest articles on this site that are listed below. 

You can visit my social media via these links:

I earned a Bachelor of Electrical Engineering at the Georgia Institute of Technology and a Masters in Computer Science from Rutgers University. I still do a bit of programming using everything from C and C++ to Rust and Ada/SPARK. I do a bit of PHP programming for Drupal websites. I have posted a few Drupal modules.  

I still get a hand on software and electronic hardware. Some of this can be found on our Kit Close-Up video series. You can also see me on many of our TechXchange Talk videos. I am interested in a range of projects from robotics to artificial intelligence. 

Sponsored Recommendations


To join the conversation, and become an exclusive member of Electronic Design, create an account today!