STATISTICAL-MECHANICS OF HYPOTHESIS EVALUATION

Following ideas of Gull, Skilling and MacKay, we develop and explore a statistical-mechanics framework through which one may assign values to the parameters of a model for a 'rule' (instanced, here, by the noisy linear perceptron), on the basis of data instancing the rule. The 'evidence' which the data offers in support of a given assignment, is likened to the free energy of a system with quenched variables (the data): the most probable (MAP) assignments of parameters are those which minimize this free-energy; tracking the free-energy minimum may lead to 'phase transitions' in the preferred assignments. We explore the extent to which the MAP assignments lead to optimal performance.