On the decision regions of multilayer perceptrons

The capabilities of two-layer perceptrons are examined with respect to the geometric properties of the decision regions they are able to form. It is known that two-layer perceptrons can form decision regions which are nonconvex and even disconnected, though the extent of their capabilities in comparison to three-layer structures is not well understood. By relating the geometry of arrangements of hyperplanes to combinatorial properties of subsets hypercube vertices, certain facts concerning the decision regions of two-layer perceptrons are deduced, and examples of decision regions which can be realized by three-layer perceptrons but not by a two-layer form are constructed. The results indicate that the graduation in ability between two- and three-layer architectures is strict. The examples of nonconvex and disconnected decision regions illustrate that the two-layer perceptron is a more capable structure than was once supposed