Learning and inference in hierarchical models with singularities

被引：7

作者：

Amari, Shun-Ichi ^{[1
]}

Ozeki, Tomoko ^{[1
]}

Park, Hyeyoung ^{[1
]}

机构：

[1] Lab. for Mathematical Neuroscience, RIKEN BSI

来源：

Systems and Computers in Japan | 2003年 / 34卷 / 07期

关键词：

Bayesian predictive distribution; Gaussian random field; Information geometry; Maximum likelihood estimate; Singularity;

D O I：

10.1002/scj.10353

中图分类号：

学科分类号：

摘要：

When we infer the underlying rule which generates a large amount of data, we assume a family of hierarchical statistical models and estimate an appropriate model and its parameters. In this case, the parameter space of the model usually includes singularities, and interesting phenomena, different from those appearing in conventional inference theory, are observed. In this paper, we review the studies of singular models in learning and inference which are being extensively developed in Japan, and elucidate the mechanisms of strange behavior by using simple models. © 2003 Wiley Periodicals, Inc.

引用

页码：34 / 42

页数：8

共 25 条

[1] Amari S., Natural gradient works efficiently in learning, Neural Computation, 10, pp. 251-276, (1998)
[2] Amari S., Nagaoka H., Information Geometry, (2000)
[3] Amari S., Ozeki T., Differential and algebraic geometry of multilayer perceptrons, IEICE Trans, E84-A, pp. 31-38, (2001)
[4] Amari S., Park H., Fukumizu K., Adaptive method of realizing natural gradient learning for multilayer perceptrons, Neural Computation, 12, pp. 1399-1409, (2000)
[5] Brockett R.W., Some geometric questions in the theory of linear systems, IEEE Trans Automatic Control, 21, pp. 449-455, (1976)
[6] Chernoff H., On the distribution of the likelihood ratio, Ann Math Stat, 25, pp. 573-578, (1954)
[7] Chernoff H., Lander E., Asymptotic distribution of the likelihood ratio test that a mixture of two binomials is a single binomial, J Stat Planning Inference, 43, pp. 19-40, (1995)
[8] Dacunha-Castelle D., Gassiat E., Testing in locally conic models, and application to mixture models, Probability Stat, 1, pp. 285-317, (1997)
[9] Fukumizu K., Likelihood Ratio of Unidentifiable Models and Multilayer Neural Networks, (2001)
[10] Fukumizu K., Generalization error of a linear neural network with a singular Fisher information matrix, Tech Rep IEICE, NC96-3, pp. 17-24, (1996)

← 1 2 3 →