Bayesian parameter estimation with informative priors for nonlinear systems

The estimation of parameters for nonlinear process models is often accomplished through optimization routines that search for a global optimum with respect to a least squares or weighted least squares criterion. While such an approach is often reasonable, it fails to account for all of the information that is available from the data and practitioner. Here we focus on the inclusion of prior knowledge in the estimation of parameters in nonlinear dynamic systems. We use the Bayesian paradigm to define the probability distribution over process model parameters, called the Bayesian posterior. The quantities associated with this posterior distribution (e.g., credible regions, means, modes) are estimated via Markov Chain Monte Carlo (MCMC) integration. We first give a short introduction to Bayesian parameter estimation and the role of MCMC in evaluating arbitrary probability distributions. Bayesian parameter estimation (via MCMC) is then applied to three case studies. The first case study shows the basic methodology of assigning and evaluating a Bayesian posterior to a simple problem that consists of estimating the mean and variance of a sample. The second case study uses prior information that specifies a preference for a particular type of reaction mechanism over another for a simulated fermentation system; the inclusion of such prior information is shown to improve the estimated values of the model parameters in situations where data are sparse or noisy (compared to the more common weighted least squares approach). The third case study develops a hybrid semi-parametric neural network (NN) model to predict time-dependent observed state variables (cell and protein concentration) in Escherichia coli fermentations. An integral step in the development of this hybrid model is parameter estimation of a nonlinear dynamic model. A hybrid model developed from the Bayesian parameter estimates is shown to outperform a hybrid model developed from the weighted least squares parameter estimates for predicting the final protein yield of a test set.