Training MLPs via the expectation maximization algorithm

Two factors that inhibit the use of multi layer perceptrons (MLP) by the pattern recognition and statistical modelling community are the long training times and a poorly developed theory of network internals. The paper proposes a solution to these problems by introducing a framework in which the hidden nodes of a multi layer perceptron are models of a binary stochastic process governed by the Bernoulli probability density function. Within this framework, we present an expectation maximisation (EM) algorithm for training the network. The algorithm iteratively finds the most probable internal representation given the network inputs and desired outputs. The internal representation forms target values for the hidden nodes so that the desired output of all nodes in the network is specified. This removes the necessity of using back propagation of errors to modify hidden layer weights, and effectively decomposes the MLP into two single layer perceptrons, the parameters of which can be efficiently optimised. We demonstrate the use of our algorithm on a number of binary and multiway classification tasks