Check out more Kit Close-Up videos.
Recently, I was able to take a quick look at the OAK-D Pro 3D camera from Luxonis. This Series 2 camera utilizes infrared (IR) illumination and dual imagers for active stereo analysis to provide 3D depth information even in no- or low-light environments. The camera also includes a 12-Mpixel RGB camera. Auto focus and fixed focus versions are available. I took a stroll around DepthAI and the company's demo program, too.
The camera has a USB 3 Type-C interface. A typical USB 3 system can power the system; however, here, there's an adapter that allows power to be provided by a separate device if needed. The system includes a hefty heatsink and a standard camera screw mount. The module is available for embedded systems as well.
The system works by using the IR projector to emit thousands of dots that are detected by a pair of IR imagers. The stereoscopic view is analyzed by the camera to generate the 3D depth information. As a result, the system can operate in low- and no-light situations. It's a very useful platform for robotics. Software like OpenCV can utilize this information.
The software to support the OAK-D Pro is called, appropriately, DepthAI. The DepthAI demo program let me get up and running almost immediately. I found installing an image from Github worked better than the Windows install program.
The camera offloads the host with 4 TOPS of processing power for analysis as well as filtering and post-processing. It provides 3D information plus standard RGB content using the RGB camera. Speeds range up to 4K at 30 frames/s and 1080P at 60 frames/s. It supports H.264, H.265, and MJPEG encoding.
The system also incorporates a CEVA BN0086 inertial measurement unit (IMU). It has its own Arm Cortex-M0+ to support the 9-axis MotionEngine sensor, which includes an accelerometer and gyroscope. The positioning data can be utilized by the image-processing system.
The DepthAI SDK with the Robotics Vision Core 2 (RVC2) and demo program, both available on Github, work with all Luxonis 3D cameras. The SDK includes Python and C++ bindings. The demo program can be configured to take advantage of different machine-learning (ML) models. It works on Linux and Windows.
The camera and SDK can be used for Simultaneous Localization And Mapping (SLAM) and future SLAM support may be incorporated into the system. A number of SLAM projects support OAK-D cameras.
Check out more Kit Close-Up videos.