Three Dimensional HDR (High Dynamic Range) Veillance for 3D Cameras Such as Kinect Sensors

Raymond Lo - University of Toronto, Canada                    
Jason Huang - University of Toronto, Canada              
Valmiki Rampersad - University of Toronto, Canada     
Steve Mann - University of Toronto, Canada

This paper presents the idea of 3D (Three Dimensional) HDR (High Dynamic Range) sensing, along with examples. We propose a method of 3D HDR veillance (sensing, computer vision, video capture, or the like) by integrating tonal and spatial information obtained from multiple HDR exposures for use in conjunction with one or more 3D cameras.

In one embodiment, as a proof-of-concept, we construct a 3D HDR camera from multiple 3D cameras such as Kinect sensors. In this embodiment the 3D cameras are arranged in a fixed array, such that the geometric relationships between them remain constant over time. Only a single camera calibration step is required at the initial time of assembling and fixing the cameras into the array. Preferably the cameras either view from the same position through beam splitters, or are fixed close to one another, so that they capture approximately the same subject matter. The cameras are arranged so they each capture a differently exposed image or video of approximately the same subject matter. In one embodiment, two Kinect cameras are attached together facing in the same direction, with an ND (Neutral Density) filter over one of them, so as to obtain a darker exposure. The dark and light exposures are combined to obtain more accurate 3D sensing in high contrast scenes.

The 3D HDR design ideas might, more generally, be incorporated into existing 3D cameras, resulting in a new kind of 3D sensor that can work in nearly any environment, including high contrast scenes such as outdoor scenes, or scenes where a bright light is shining directly into the sensor.

High Dynamic Range
Video Image Processing
Intelligent Image Processing
Wearable Computing
HDR Video