ANALYSIS OF AN IDENTIFICATION ALGORITHM ARISING IN THE ADAPTIVE ESTIMATION OF MARKOV-CHAINS

被引：30

作者：

ARAPOSTATHIS, A ^{[1
]}

MARCUS, SI ^{[1
]}

机构：

[1] UNIV TEXAS,DEPT ELECT & COMP ENGN,AUSTIN,TX 78712

来源：

MATHEMATICS OF CONTROL SIGNALS AND SYSTEMS | 1990年 / 3卷 / 01期

关键词：

Adaptive estimation; Markov chains; Stochastic control;

D O I：

10.1007/BF02551353

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

We investigate an algorithm applied to the adaptive estimation of partially observed finite-state Markov chains. The algorithm utilizes the recursive equation characterizing the conditional distribution of the state of the Markov chain, given the past observations. We show that the process "driving" the algorithm has a unique invariant measure for each fixed value of the parameter, and following the ordinary differential equation method for stochastic approximations, establish almost sure convergence of the parameter estimates to the solutions of an associated differential equation. The performance of the adaptive estimation scheme is analyzed by examining the induced controlled Markov process with respect to a long-run average cost criterion. © 1990 Springer-Verlag New York Inc.

引用

页码：1 / 29

页数：29

共 29 条

[11] LIMIT THEOREM FOR PARTIALLY OBSERVED MARKOV CHAINS
KAIJSER, T
[J]. ANNALS OF PROBABILITY, 1975, 3 (04) : 677 - 696
[12] LIMIT-THEOREM FOR MARKOV-CHAINS IN COMPACT METRIC SPACES WITH APPLICATIONS TO PRODUCTS OF RANDOM MATRICES
KAIJSER, T
[J]. DUKE MATHEMATICAL JOURNAL, 1978, 45 (02) : 311 - 349
[13] IDENTIFICATION OF A TRANSITION MATRIX OF A MARKOV CHAIN FROM NOISY MEASUREMENTS OF STATE
KASHYAP, RL
[J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 1970, 16 (02) : 161 - +
[14] Kolonko M., 1982, Mathematische Operationsforschung und Statistik, Series Optimization, V13, P567, DOI 10.1080/02331938208842822
[15] A SURVEY OF SOME RESULTS IN STOCHASTIC ADAPTIVE-CONTROL
KUMAR, PR
[J]. SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 1985, 23 (03) : 329 - 380
[16] KUSHNER H., 1984, APPROXIMATION WEAK C
[17] Kushner H. J., 1978, Stochastic approximation methods for constrained and unconstrained systems
[18] AN INVARIANT MEASURE APPROACH TO THE CONVERGENCE OF STOCHASTIC APPROXIMATIONS WITH STATE DEPENDENT NOISE
KUSHNER, HJ
SHWARTZ, A
[J]. SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 1984, 22 (01) : 13 - 27
[19] KUSHNER HJ, 1981, J MATH ANAL APPL, V82, P527
[20] KUSHNER HJ, 1983, RECENT ADV STATISTIC, P211

← 1 2 3 →