On the slow convergence of EM and VBEM in low-noise linear models

被引：20

作者：

Petersen, KB ^{[1
]}

Winther, O ^{[1
]}

Hansen, LK ^{[1
]}

机构：

[1] Tech Univ Denmark, DK-2300 Copenhagen, Denmark

来源：

NEURAL COMPUTATION | 2005年 / 17卷 / 09期

关键词：

D O I：

10.1162/0899766054322991

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We analyze convergence of the expectation maximization (EM) and variational Bayes EM (VBEM) schemes for parameter estimation in noisy linear models. The analysis shows that both schemes are inefficient in the low-noise limit. The linear model with additive noise includes as special cases independent component analysis, probabilistic principal component analysis, factor analysis, and Kalman filtering. Hence, the results are relevant for many practical applications.

引用

页码：1921 / 1926

页数：6

共 12 条

[1]

ATTIAS H, 1999, P 15 C UNC ART INT U

[2]

Beal MJ, 2003, BAYESIAN STATISTICS 7, P453

[3] AN INFORMATION MAXIMIZATION APPROACH TO BLIND SEPARATION AND BLIND DECONVOLUTION [J].

BELL, AJ ;

SEJNOWSKI, TJ .

NEURAL COMPUTATION, 1995, 7 (06) :1129-1159

[4]

BERMOND O, 1999, P 1 INT WORKSH IND C

[5] MAXIMUM LIKELIHOOD FROM INCOMPLETE DATA VIA EM ALGORITHM [J].

DEMPSTER, AP ;

LAIRD, NM ;

RUBIN, DB .

JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL, 1977, 39 (01) :1-38

[6] Mean-field approaches to independent component analysis [J].

Hojen-Sorensen, PADFR ;

Winther, O ;

Hansen, LK .

NEURAL COMPUTATION, 2002, 14 (04) :889-918

[7]

McLachlan G. J., 1997, EM ALGORITHM EXTENSI

[8]

OLSSON RK, 2005, UNPUB STATE SPACE MO

[9]

PETERSEN KB, 2005, IEEE INT C AC SPEECH

[10]

PETERSEN KB, 2005, 20052 TU DENMARK

← 1 2 →