On algorithms for restricted maximum likelihood estimation

This work proposes a globally convergent algorithm, based on gradient projections, for maximum likelihood estimation under linear equality and inequality restrictions (constraints) on parameters. The proposed algorithm has wide applicability, and as an important special case its application to restricted expectation-maximization (EM) problems is described. Often, a class of algorithms that we call expectation-restricted-maximization (ERM) is used to deal with constraints in the EM setting. We describe two such ERM algorithms that handle linear equality constraints, and discuss their convergence. As we explain, the assumptions for global convergence of one of the algorithms may be practically too restrictive, and as such we suggest a modification. We provide an example where the second algorithm fails. In general we argue that the gradient projection (GP) algorithm is superior to ERM algorithms in terms of simplicity of implementation and time to converge. We give an example of application of GP to parameter estimation of mixtures of normal densities where linear inequality constraints are imposed, and compare CPU times required for the algorithms discussed. (C) 2002 Elsevier B.V. All rights reserved.