ESTIMATION OF THE NUMBER OF SIGNALS IN THE PRESENCE OF UNKNOWN CORRELATED SENSOR NOISE

A method for estimating the number of transmitted signals in the presence of spatially correlated sensor noise is proposed. The procedure is developed under the assumption that the unknown noise covariance matrix is a band matrix. In practice, it is quite robust with respect to this finite correlation length assumption. In sensor array processing, this amounts to assuming that the noise field is locally correlated spatially. Since spatial stationarity of the noise is not necessary, it also applies, for instance, to white noise with different power along the array. Simulations indicate that the asymptotic analysis holds for quite small sample sizes.