A component-wise EM algorithm for mixtures

被引:53
作者
Celeux, G
Chrétien, S
Forbes, F
Mkhadri, A
机构
[1] INRIA Rhone Alpes, ZIRST, F-38334 Saint Ismier, France
[2] Univ Franche Comte, Dept Math, F-25030 Besancon, France
[3] Univ Cadi Ayyad Marrahesh, Fac Sci Semlalie, Dept Math, Marrakech, Morocco
关键词
EM algorithm; missing data space reduction; mixture estimation; proximal point algorithm; SAGE algorithm;
D O I
10.1198/106186001317243403
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
Maximum likelihood estimation in finite mixture distributions is typically approached as an incomplete data problem to allow application of the expectation-maximization (EM) algorithm. In its general formulation, the EM algorithm involves the notion of a complete data space, in which the observed measurements and incomplete data are embedded. An advantage is that many difficult estimation problems are facilitated when viewed in this way. One drawback is that the simultaneous update used by standard EM requires overly informative complete data spaces, which leads to slow convergence in some situations. In the incomplete data context, it has been shown that the use of less informative complete data spaces, or equivalently smaller missing data spaces, can lead to faster convergence without sacrifying simplicity. However, in the mixture case, little progress has been made in speeding up EM. In this article we propose a component-wise EM for mixtures. It uses, at each iteration, the smallest admissible missing data space by intrinsically decoupling the parameter updates. Monotonicity is maintained, although the estimated proportions may not sum to one during the course of the iteration. However, we prove that the mixing proportions will satisfy this constraint upon convergence. Our proof of convergence relies on the interpretation of our procedure as a proximal point algorithm. For performance comparison, we consider standard EM as well as two other algorithms based on missing data space reduction, namely the SAGE and AECME algorithms. We provide adaptations of these general procedures to the mixture case. We also consider the ECME algorithm, which is not a data augmentation scheme but still aims at accelerating EM. Our numerical experiments illustrate the advantages of the component-wise EM algorithm relative to these other methods.
引用
收藏
页码:697 / 712
页数:16
相关论文
共 32 条
[1]  
Celeux G., 1999, 3746 INRIA
[2]   Kullback proximal algorithms for maximum-likelihood estimation [J].
Chrétien, S ;
Hero, AO .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2000, 46 (05) :1800-1810
[3]  
CHRETIEN S, 1998, 313 CSPL U MICH
[4]  
CHRETIEN S, 1998, IEEE INT S INF THEOR
[5]  
CIARLET P.G, 1988, INTRO NUMERICAL LINE
[6]   MAXIMUM LIKELIHOOD FROM INCOMPLETE DATA VIA EM ALGORITHM [J].
DEMPSTER, AP ;
LAIRD, NM ;
RUBIN, DB .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL, 1977, 39 (01) :1-38
[7]   PENALIZED MAXIMUM-LIKELIHOOD IMAGE-RECONSTRUCTION USING SPACE-ALTERNATING GENERALIZED EM ALGORITHMS [J].
FESSLER, JA ;
HERO, AO .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 1995, 4 (10) :1417-1429
[8]   SPACE-ALTERNATING GENERALIZED EXPECTATION-MAXIMIZATION ALGORITHM [J].
FESSLER, JA ;
HERO, AO .
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1994, 42 (10) :2664-2677
[9]   ANOTHER INTERPRETATION OF THE EM ALGORITHM FOR MIXTURE DISTRIBUTIONS [J].
HATHAWAY, RJ .
STATISTICS & PROBABILITY LETTERS, 1986, 4 (02) :53-56
[10]  
HERO AO, 1995, STAT SINICA, V5, P41