Bayesian hypothesis testing: a reference approach

For any probability model M equivalent to {p(x\theta, omega), theta is an element of Theta, omega is an element of Omega} assumed to describe the probabilistic behaviour of data x is an element of X; it is argued that testing whether or not the available data are compatible with the hypothesis H-0 equivalent to (theta = theta(0)) is best considered as a formal decision problem on whether to use(a(0)), or not to use (a(1)), the simpler probability model (or null model) M-0 equivalent to {p(x\theta(0), omega), omega is an element of Omega}, where the loss difference L(a(0), theta, omega) - L(a(1), theta, omega) is proportional to the amount of information delta(theta(0), theta, omega) which would be lost if the simplified model MO were used as a proxy for the assumed model M. For any prior distribution pi(theta, omega), the appropriate normative solution is obtained by rejecting the null model MO whenever the corresponding posterior expectation integral integral delta(theta(0), theta, omega) pi(theta, omega \ x)dtheta domega is sufficiently, large. Specification of a subjective prior is always difficult, and often polemical, in scientific communication Information theory may be used to specify a prior, the reference prior, which only depends on the assumed model M, and mathematically describes a situation where no prior information is available about the quantity of interest. The reference posterior expectation, d(theta(0), x) = integral delta pi(delta \ x)ddelta, of the amount of information delta(theta(0), theta, omega) which could be lost if the null model were used, provides an attractive non-negative test function, the intrinsic statistic, which is invariant under reparametrization. The intrinsic statistic d(theta(0), x) is measured in units of information, and it is easily calibrated (for any sample size and any dimensionality) in terms of some average long-likelihood ratios.. The corresponding Bayes decision rule the Bayesian reference criterion (BRC), indicates that the null model M-0 should only be rejected if the posterior expected loss of information from using the simplified model M-0 is too large or, equivalently, if the associated expected average log-likelihood ratio is large enough. The BRC criterion provides a general reference Bayesian solution to hypothesis testing which does not assume a probability mass concentrated on M-0 and, hence, it is immune to Lindley's paradox. The theory is illustrated within the context of multivariate normal data, where it is shown to avoid Rao's paradox on the inconsistency between univariate and multivariate frequentist hypothesis testing.