Tracking Pedestrians With Machine Vision

Machine vision systems have been used for years to automate processes. Applications can range from checking the amount of fluid in bottles on production lines to inspecting the number and alignment of pins on a microchip or detecting flaws in fruit before it is packaged for shipping.

Most of these applications deal with very tangible, defined measurements. For example, in bottle inspection, the camera system is triggered via a proximity sensor when the bottle is in front of the camera. Then edge detection simply looks for a sharp transition in intensities to determine how high the liquid is in the bottle.

Similarly, checking the pins on a chip involves triggering the system when the chip is in front of the camera and performing the same edge detection for each pin. The only change would be the addition of a caliper measurement between each of the pins to ensure none are bent. These calculations aren�t overly demanding on a processor. As long as the system has enough processing power to navigate the array of pixels from the camera fast enough to keep up with the hardware trigger being received, there aren�t many complications. Usually, these inspections end with a pass/fail result that can be fed into a system that rejects the bad parts or into a storage system that maintains a cache of failed inspection images.

But in recent years, engineers and scientists have had more advanced computer components as well as image acquisition algorithms with which to develop machine vision systems for implementing applications that previously weren�t even considered. Video surveillance, for example, often is done with multiple cameras and an individual watching video screens that receive these images. Even with multiple people watching the monitors, this can be a daunting task.

People Tracking
The demand for a smart surveillance system is increasing. At many places, including subway stations, airports, and theme parks, there are far too many cameras installed to implement real-time observation by human operators.

The Institute for Digital Image Processing at Joanneum Research recently designed and implemented high-level computer vision algorithms for object detection, classification, and tracking applications. Using these algorithms, the institute tried to address topics such as how to detect and track pedestrians in video streams.

This was no simple inspection with a pass/fail result. It involved tracking objects of different sizes, velocities, accelerations, directions, and colors. Unlike bottles, microchips, or fruit, each person had a mind of his own, adding another level of difficulty to detecting and tracking people with a computer-based application.

Joanneum Research has a long history of working with vision acquisition and analysis algorithms. While creating an application to sort recycled plastic, it started developing shape-recognition algorithms using Fourier transforms to classify objects.

Fourier features are detectable regardless of rotation, translation, or scale changes of the original pattern. Using Fourier features, certain objects commonly found in recycle sorting could be easily detected, reducing development time and increasing the accuracy of the sorting algorithm.

Joanneum Research developed this system with the National Instruments LabVIEW graphical programming environment and the NI IMAQ Vision library. NI LabVIEW has built-in fast Fourier functions to develop and test the sorting algorithms. The NI IMAQ Vision library contains commonly used machine vision algorithms.

In a later project, Joanneum Research designed a new set of shape analysis functions based on principal components analysis (PCA), a way to map higher dimensional data sets to a lower dimension for analysis. It is known as the optimal linear transformation for keeping the subspace that has largest variance. It also is a system-intensive algorithm and was not possible in real time until faster computer hardware was available.

Another part of this set of algorithms is something called the eigenshape principle, which Joanneum Research used to describe complex shapes with just a few simple parameters. Figure 1 shows a LabVIEW diagram of a routine to compute eigenshapes. The eigenshape principle was an ideal solution for object tracking because vehicles or people could be located regardless of color, speed, direction, or consistency in motion with just a few parameters.