Using Specular Highlights as Pose Invariant Features for 2D-3D Pose Estimation


Authors

Aaron Netz
Margarita Osadchy

CVPR 2011 Paper: PDF, BibTeX

Abstract

We address the problem of 2D-3D pose estimation in difficult viewing conditions, such as low illumination, cluttered background, and large highlights and shadows that appear on the object of interest. In such challenging conditions conventional features used for establishing correspondence are unreliable. We show that under the assumption of a dominant light source, specular highlights produced by a known object can be used to establish correspondence between its image and the 3D model, and to verify the hypothesized pose. These ideas are incorporated in an efficient method for pose estimation from a monocular image of an object using only highlights produced by the object as its input. The proposed method uses no knowledge of lighting direction and no calibration object for estimating the lighting in the scene. The evaluation of the method shows good accuracy on numerous synthetic images and good robustness on real images of complex, shiny objects, with shadows and difficult backgrounds.

3D model set

The original 3D model files can be found here in .OBJ format.

Synthetic image set

These are 3D renderings of the original models, that were used for the synthetic tests in this work. Each image was rendered with a randomly generated pose and light direction. The entire set can be found here. Some sample images:
buddha_2 cane_1 duck_11 kitten_3

Outdoor image set

These are pictures of the manufactured models, that were taken outdoors. The entire set can be found here. Some sample images:
cow08 fertility09 gargoyle21 torso03

Indoor image set

These are pictures of the manufactured models, that were taken indoors. The entire set can be found here. Some sample images:
frog-1 mouse-2 fertility-10 cow-9