Abstract
We present a novel approach to pose estimation and model-based recognition of specular objects in difficult viewing conditions, such as low illumination, cluttered background, large highlights and shadows that appear on the object of interest. In such challenging conditions conventional features are unreliable. We show that under the assumption of a dominant light source, specular highlights produced by a known object can be used to establish correspondence between its image and the 3D model, and to verify the hypothesized pose and the identity of the object. Previous methods that use highlights for recognition make limiting assumptions such as known pose, scene-dependent calibration, simple shape, etc. The proposed method can efficiently recognize free form specular objects in arbitrary pose and under unknown lighting direction. It uses only a single image of the object as its input and outputs object identity and the full pose. We have performed extensive experiments for both recognition and pose estimation accuracy on synthetic images and on real indoor and outdoor images.
3D model set
The original 3D model files can be found here in .OBJ format.
- fertiity.obj is based upon model provided courtesy of the AIM@SHAPE Shape Repository.
- kitten.obj, frog.obj are based upon models provided courtesy of Frank_ter_Haar by the AIM@SHAPE Shape Repository.
- cane.obj, buddha.obj, gargoyle.obj are based upon models provided courtesy of VCG-ISTI by the AIM@SHAPE Shape Repository.
- duck.obj, mouse.obj are based upon models provided courtesy of IMATI by the AIM@SHAPE Shape Repository.
- torso.obj, horse.obj are based upon models provided courtesy of INRIA by the AIM@SHAPE Shape Repository.
- cow.obj is based upon model provided courtesy of MPII by the AIM@SHAPE Shape Repository.
Synthetic image set
These are 3D renderings of the original models, that were used for the synthetic tests in this work.
Each image was rendered with a randomly generated pose and light direction.
The entire set can be found here.
Some sample images:
Outdoor image set
These are pictures of the objects, that were taken outdoors.
Images against black background can be found here, images against cluttered background can be found here.
Some sample images:
Indoor image set
These are pictures of the objects, that were taken indoors.
The entire set can be found here.
Some sample images:
