OPTIMAL LEARNING WITH A NEURAL NETWORK

We introduce optimal learning with a neural network, which we define as minimising the expectation generalisation error. We find that the optimally-trained spherical perceptron may learn a linearly-separable rule as well as any possible network. We sketch an algorithm to generate optimal learning, and simulation results support our conclusions. Optimal learning of a well-known, significant unlearnable problem, the <<mismatched weight>> problem, gives better asymptotic learning than conventional techniques, and may be simulated enormously more easily. Unlike many other learning schemes, optimal learning extends to more general networks learning more complex rules.