Tracking three-dimensional rigid objects with direct image alignment and local appearance based feature matching

Pradit Mittrapiyanuruk, Purdue University

Abstract

We present a new model based method for 3D tracking of a rigid object. The proposed method consists of the pose initialization module and the pose refinement module. These two modules work cooperatively to accomplish the task of tracking a 3D rigid object. Our approach for pose initialization is based on the notion of local appearance based feature matching and is similar to the work of Gordon and Lowe, Rothganger et al., and Lepetit and Fua. For pose refinement, we present a new method based on direct image alignment that extends the work of Cobzas et al. by using the image alignment framework as proposed by Baker et al. In our new method, an object model is represented by a set of 3D planar patches located on the object surface. Associated with each patch is its appearance model that consists of a collection of eigenimage sets. Each eigenimage set is obtained from a keyframe image and images recorded from nearby viewpoints. A keyframe is the image corresponding to a viewpoint that is significantly different from the viewpoints used for the other keyframes. (Our work includes an automated procedure for keyframe extraction from a training video sequence.) Using such an object model, the problem of 3D pose estimation is recast as the direct image alignment based minimization problem. Given an initial 3D pose, the inverse compositional method is applied to solve the minimization by iteratively refining the 3D pose until convergence is achieved. This contribution also presents two methods for semi-automatically constructing the object models for pose initialization and the pose refinement.

Degree

Ph.D.

Advisors

Kak, Purdue University.

Subject Area

Electrical engineering

COinS