Chips Accelerate Multi-Camera Automotive Video Processing

June 16, 2011
Multicore processors like Renesas IMAPCAR help deliver real time video processing for automotive applications.

Fig 1. OmniVision’s OV9740 digital camera is finding a home in automotive applications.

Fig 2. Xilinx’s Spartan FPGA (a) and Freescale’s Qorriva (b) provide the processing power needed to blend multiple camera inputs into a 360° bird’s eye view around an automobile in real time.

Video processing sees significant gains with multicore processors. In some instances, custom multicore chips work best. PrimeSense’s chip utilizes multiple cores to support Microsoft’s Kinect (see “Kinecting With The Point Cloud Library,” p. xx). It analyzes reflected images of a projected array of infrared (IR) dots in real time at 60 frames/s (fps).

The Renesas IMAPCAR video recognition processor uses images from standard digital cameras placed around an automobile (see “Four-Wheeled Supercomputers” at electronicdesign.com). Its 16-bit RISC processor handles management, but 64 processing units (PEs) tackle the heavy lifting.

Each PE can execute four-way very long instruction word (VLIW) instructions utilizing the unit’s local memory. IMAPCAR has been used for parking assistance applications, lane change warning, blind corner warning, and dynamic headlight control.

Rerolling Fish Eyes

The OmniVision Technologies OV9740 is just one digital camera finding a home in automotive applications (Fig. 1). It uses OmniVision’s 1.75-µm OmniBSI backside illumination pixel architecture and an image signal processor (ISP) to deliver 720p video at 30 fps. It’s designed for low-light environments with sensitivity of 1300 mV/lux-s.

Chips like Xilinx’s Spartan 6 and Freescale’s Quorriva are taking advantage of camera chips like those from OmniVision (Fig. 2). Four cameras are part of the 360° surround view automotive reference design from Xilinx based on the Spartan 6 FPGA.

Xilinx’s Automotive Targeted Design Platform uses a fisheye lens on each camera to provide a wider field of view (see “FPGA Design Platform Kits Highlight Automotive Convergence” at electronicdesign.com). Of course, this setup distorts the image. Optical correction is possible, but so is mathematical correction if sufficient processing power is available. And this is where an FPGA excels.

The design kit provides a modular design for this application, allowing developers to utilize third-party modules or to enhance or develop their own. The basics include elimination of image distortion and then stitching together multiple images. The data from four cameras also could be used for applications like lane change warnings, but this would require additional computing power.

Freescale’s 32-bit Qorivva MPC5604E microcontroller is used to send compressed video streams over a Fast Ethernet link. It sits next to the digital camera, where it compresses images using low-latency motion JPEG compression providing Internet Protocol (IP) camera support.

This modular approach makes camera placement easier. The two-wire Ethernet network is less cable-intensive than shipping video using other cabling systems. It also makes linking to platforms such as Xilinx’s Spartan 6 easier since many FPGAs like the Spartan 6 support Ethernet.

Real-time video processing is becoming more common in automotive platforms. IP cameras compress video sources for a range of multicore subsystems to analyze so drivers have unprecedented views around the car that hopefully make driving safer.

Freescale
www.freescale.com

Omnivision Technologies
www.ovt.com

Xilinx
www.xilinx.com

About the Author

William G. Wong | Senior Content Director - Electronic Design and Microwaves & RF

I am Editor of Electronic Design focusing on embedded, software, and systems. As Senior Content Director, I also manage Microwaves & RF and I work with a great team of editors to provide engineers, programmers, developers and technical managers with interesting and useful articles and videos on a regular basis. Check out our free newsletters to see the latest content.

You can send press releases for new products for possible coverage on the website. I am also interested in receiving contributed articles for publishing on our website. Use our template and send to me along with a signed release form. 

Check out my blog, AltEmbedded on Electronic Design, as well as his latest articles on this site that are listed below. 

You can visit my social media via these links:

I earned a Bachelor of Electrical Engineering at the Georgia Institute of Technology and a Masters in Computer Science from Rutgers University. I still do a bit of programming using everything from C and C++ to Rust and Ada/SPARK. I do a bit of PHP programming for Drupal websites. I have posted a few Drupal modules.  

I still get a hand on software and electronic hardware. Some of this can be found on our Kit Close-Up video series. You can also see me on many of our TechXchange Talk videos. I am interested in a range of projects from robotics to artificial intelligence. 

Sponsored Recommendations

Comments

To join the conversation, and become an exclusive member of Electronic Design, create an account today!