Estimating the number of signals in the presence of white noise

In signal processing, high-resolution signal parameter estimation is a significant problem. In particular, estimation of the direction of the narrow-band signals emitted by multiple sources has many applications in signal processing. Quite a number of papers appeared in the last 25 years regarding estimation of the parameters related to the direction in which signals arrive, but not that much attention has been given to estimating the number of signals. In this paper we develop a method using the penalty function technique. Instead of using any fixed penalty function like AIC or MDL, a class of penalty functions satisfying some special properties is used. We prove that any penalty function from that particular class will produce consistent estimates under the assumptions that the random disturbances are independent and identically distributed with mean zero and finite variance. We also obtain the probabilities of wrong detection for any particular penalty function and estimate it using the matrix perturbation technique. It sheds light on how to choose the proper penalty function for any particular model. Simulations are performed to verify the usefulness of the analysis and to compare our method with the existing ones. (C) 2000 Elsevier Science B.V. All rights reserved. MSC: 62H12; 62H15; 62F12.