THE SINGULAR-VALUE DECOMPOSITION AND LONG AND SHORT SPACES OF NOISY MATRICES

Using geometrical, algebraic, and statistical arguments, it is clarified why and when the singular value decomposition is successful in so-called subspace methods. First we introduce the concepts of long and short spaces. We discuss a fundamental asymmetry in the consistency properties of the estimates: The model, which is associated with the short space, can be estimated consistently but the estimates of the original data, which follow from the long space, are always inconsistent. We find an expression for the asymptotic bias in terms of canonical angles, which can be estimated from the data. This allows us to describe all equivalent reconstructions of the original signals as a matrix ball, the center of which is the minimum variance estimate. Remarkably, the canonical angles also appear in the optimal weighting that is used in weighted subspace fitting approaches. The results are illustrated with a numerical simulation. Examples that are discussed include total linear least squares, the direction-of-arrival estimation algorithm ESPRIT, a biomedical signal processing application to separate the fetal ECG from that of the mother, and the identification of linear state space models from noisy input-output data.