A REPARAMETERIZATION TO IMPROVE NUMERICAL OPTIMIZATION IN MULTIVARIATE REML (CO)VARIANCE COMPONENT ESTIMATION

Multivariate restricted maximum likelihood (REML) (co)variance component estimation using numerical optimization on the basis of Downhill-Simplex (DS) or quasi-Newton (QN) procedures suffers from the problem of undefined 'covariance matrices' as are produced by the optimizers. So far, this problem has been dealt with by assigning 'bad' function values. For this procedure to work, it is implied that the information this 'bad' function Value conveys is sufficient to avoid going in the same direction in the following optimization step. To a limited degree DS can cope with this situation. On the other hand QN usually breaks down if this situation occurs too frequently. This contribution analyzes the problem and proposes a reparameterization of the covariance matrices to solve it. As a result., faster converging QN optimizers can be used, as they no longer suffer from lack of robustness. Four real data sets were analyzed using a multivariate model estimating between 17 and 30 (co)variance components simultaneously. Optimizing on the Cholesky factor instead of on the (co)variance components themselves reduced the computing time by a factor of 2.5 to more than 250, when comparing the robust modified DS optimizer operating on the original covariance matrices to a QN optimizer using reparameterized covariance matrices.