Head-mounted displays (HMDs) for virtual and augmented reality (VR/AR) need sensors to determine head movement and positioning, otherwise the effects provided by the HMD can range from annoying to sickening. Even latency can cause a problem even if the tracking is accurate.
What’s needed is fast and accurate tracking. This is accomplished by using sensors on the HMD, called inside-out tracking, versus external sensors, or outside-in. The former has the advantage of simplifying the overall system, at least from a user’s perspective.
Inside-out 3D Camera Delivers V-SLAM
Two very different solutions have cropped up recently. One, from Intel, is based on its RealSense 3D camera. The RealSense family recently acquired a built-in IMU, although this makes the version of the camera useful for other reasons. The newest addition targets inside-out positioning. The RealSense T265 (Fig. 1) employs Intel’s Movidius Myriad 2 machine-learning (ML) video-processing system.
1. Intel’s T265 RealSense Tracking Camera combines a 3D RealSense camera system with the Movidius Myriad 2 machine-learning, video-processing system.
The T265 implements a visual inertial odometry simultaneous localization and mapping (V-SLAM) system that tracks objects in the real world. It delivers six degrees of freedom (6DOF) of inside-out tracking support. Essentially, it tracks objects and their movement and position with respect to the camera. The RealSense sensors provide the 3D depth information that’s then processed by the Movidius SoC. Fish-eye lenses provide a wide, 170-degree field of view (FOV). Pricing starts at $199.
The T265 can be mounted on an HMD to provide V-SLAM support. It also will deliver video and depth information for other aspects of an AR/VR application. On top of that, the sensor will be useful for robotics and drones, especially in GPS-restricted environments.
Chirping Ultrasonically
Chirp is TDK’s latest acquisition. Instead of infrared, 3D cameras, Chirp’s SonicTrack solution employs tiny ultrasonic transceivers, which can often outperform infrared sensors in a range of applications. They’re able to provide range information by themselves, although not to the degree of a RealSense 3D camera. On the other hand, the CH-101 chip is tiny (Fig. 2).
2. TDK’s Chirp CH-101 ultrasonic transceiver is tiny and power-efficient.
Ultrasonic sensors have been used for range sensing applications in a number of areas like proximity sensors on cars. Things get a bit more interesting when multiple sensors are combined with the kind of intelligence Intel added to its RealSense cameras.
One application that TDK is working on delivers inside-out positioning information about the 3D hand controllers for AR/VR applications; in particular, HTC’s Vive Focus HMD (Fig. 3). Three Chirp sensors are mounted on each controller and additional sensors are on the HMD. These are used cooperatively to provide accurate relative positioning information via triangulation. Thus, the controllers can be used without initial calibration and setup.
3. TDK is working with HTC and its Vive Focus HMD to provide inside-out tracking of the hand controllers with the VR HMD.
The CH-101 is an omnidirectional sensor. A single sensor is only able to provide range information, not positional information. Likewise, the sensor can be used as a transmitter, receiver, or both albeit one mode at a time.
TDK’s InvenSense Fusion software can combine fuse sensor data from a number of devices, including the new ICM42688 and ICM42686 6-axis motion sensors. The software is designed to run on standard microcontrollers.
