Structure from Motion

This research is to investigate the use of robust methods in automatic recovery of 3D structure information and camera positions from image sequences taken with an uncalibrated camera. By analysing how noise affects the estimation process, the reliability of the partial results and implicitely the final result can be increased by using statistically justified estimators.
As a testbed I have implemented a modular system for automatic reconstruction starting from a given image sequence. It includes low level feature extraction such as corners, edges, regions; feature matching across frames; projective structure estimation modules such as trifocal tensor estimation, triangulation, camera resectioning, projective alignment of coordinate systems; autocalibration and metric structure estimation.

Example: MOVI sequence (118 frames).

Some frames
metric reconstruction
texture mapped	Model (vrml), Movie (avi, 3.6Mb)

Feature Extraction

Traditional gradient based edge detection is a three steps procedure: gradient filtering, nonmaxima suppression and hysteresis thresholding. By looking at the gradient filtering as a projection in a linear vector space, an independent edge confidence measure is derived from data that sits in the orthogonal complement of the gradient subspace (this data being "invizible" to the gradient operator). The nonmaxima suppression and hysteresis thresholding steps are modified to include the confidence, in this case the decision is made in a 2D space using thresholding curves. The edge detector is able to find sharp edges with small amplitude without the expense of clutter.

Check out the EDISON system which implements the confidence based edge detection and mean shift based segmentation (original and synergistic).

Here is an example of what one might get by applying the gradient filter (enlarged images):

	-- gradient filter -->
	-- gradient filter -->

Some examples:

original	confidence based	Canny

Point Matching under Large Image Transformations

Interest point correspondences are determined by combining traditional optical flow with matching color distributions computed with oriented kernels. Tracking while maintaining also the object orientation is possible using the same formulation.
Examples to come.