Optimal structure from motion: Local ambiguities and global estimates

Structure From Motion (SFM) refers to the problem of estimating spatial properties of a three-dimensional scene from the motion of its projection onto a two-dimensional surface, such as the retina. We present an analysis of SFM which results in algorithms that are provably convergent and provably optimal with respect to a chosen norm. In particular, we cast SFM as the minimization of a high-dimensional quadratic cost function, and show how it is possible to reduce it to the minimization of a two-dimensional function whose stationary points are in one-to-one correspondence with those of the original cost function. As a consequence, we can plot the reduced cost function and characterize the configurations of structure and motion that result in local minima. As an example, we discuss two local minima that are associated with well-known visual illusions. Knowledge of the topology of the residual in the presence of such local minima allows us to formulate minimization algorithms that, in addition to provably converge to stationary points of the original cost function, can switch between different local extrema in order to converge to the global minimum, under suitable conditions. We also offer an experimental study of the distribution of the estimation error in the presence of noise in the measurements, and characterize the sensitivity of the algorithm using the structure of Fisher's Information matrix.