Rank Constraints for Homographies over Two Views: Revisiting the Rank Four Constraint

It is well known that one can collect the coefficients of five (or more) homographies between two views into a large, rank deficient matrix. In principle, this implies that one can refine the accuracy of the estimates of the homography coefficients by exploiting the rank constraint. However, the standard rank-projection approach is impractical for two different reasons: it requires many homographies to even score a modest gain; and, secondly, correlations between the errors in the coefficients will lead to poor estimates. In this paper we study these problems and provide solutions to each. Firstly, we show that the matrices of the homography coefficients can be recast into two parts, each consistent with ranks of only one. This immediately establishes the prospect of realistically (that is, with as few as only three or four homographies) exploiting the redundancies of the homographies over two views. We also tackle the remaining issue: correlated coefficients. We compare our approach with the "gold standard"; that is, non-linear bundle adjustment (initialized from the ground truth estimate-the ideal initialization). The results confirm our theory and show one can implement rank-constrained projection and come close to the gold standard in effectiveness. Indeed, our algorithm (by itself), or our algorithm further refined by a bundle adjustment stage; may be a practical algorithm: providing generally better results than the "standard" DLT (direct linear transformation) algorithm, and even better than the bundle adjustment result with the DLT result as the starting point. Our unoptimized version has roughly the same cost as bundle adjustment and yet can generally produce close to the "gold standard" estimate (as illustrated by comparison with bundle adjustment initialized from the ground truth). Independent of the merits or otherwise of our algorithm, we have illuminated why the naive approach of direct rank-projection is relatively doomed to failure. Moreover, in revealing that there are further rank constraints, not previously known; we have added to the understanding of these issues, and this may pave the way for further improvements.