Stochastic versions of the EM algorithm: An experimental study in the mixture case

We compare three different stochastic versions of the EM algorithm: The Stochastic EM algorithm (SEM), the ''Simulated Annealing'' EM algorithm (SAEM) and the Monte Carlo EM algorithm (MCEM). We focus particularly on the mixture of distributions problem. In this context, we investigate the practical behaviour of these algorithms through intensive Monte Carlo numerical simulations and a real data study. We show that, for some particular mixture situations, the SEM algorithm is almost always preferable to the EM and ''simulated annealing'' versions SAEM and MCEM. For some severely overlapping mixtures, however, none of these algorithms can be confidently used. Then, SEM can be used as an efficient data exploratory tool for locating significant maxima of the likelihood function. In the real data case, we show that the SEM stationary distribution provides a contrasted view of the loglikelihood by emphasizing sensible maxima.