Visualizations of 2-D hidden unit space

For the visualizations, the backpropagation learning procedure was applied to strictly layered feedforward networks with one hidden layer that contained just two units. Values on the input units were binary (0, 1). The squashing function on the output units was the standard sigmoid with upper and lower bounds at 0 and 1. An expanded range, (-1, 1) was used for the hidden units to enhance learning speed and enhance the separation of patterns in the HUAP visualization technique. The resulting images reveal several properties of the hidden unit representations achieved by backpropagation. These include (1) that the normal solution to XOR collapses the pattern space to a one-dimensional manifold and (2) the high symmetry of the hidden unit patterns achieved in the N-2-N encoder task