STATISTICAL-MECHANICS OF A MULTILAYERED NEURAL NETWORK

Statistical mechanics is applied to estimate the maximal information capacity per synapse (c) of a multilayered feedforward neural network, functioning as a parity machine. For a large number of hidden units, K, the replica-symmetric solution overestimates dramatically the capacity,K2. However, a one-step replica-symmetry breaking gives lnK/ln2, which coincides with a theoretical upper bound. It is suggested that this asymptotic behavior is exact. Results for finite K are also discussed. © 1990 The American Physical Society.