On the weight sparsity of multilayer perceptrons

Approximating and representing a process, a function, or a system with an adaptive parametric model constitutes a major part of current machine learning research. An important characteristic of these models is parameter sparsity, an indicator of how succintly a model can codify fundamental properties of the approximated function. This paper investigates the sparsity patterns of a multilayer perceptron netwrok trained to mount a man-on-the-middle attack on the DES symmetric cryptosystem. The notions of absolute and effective synaptic weight sparsity are introduced and their importance to network learning procedure is explained. Finally, the results from the training of the actual multilayer perceptron are outlined and discussed. In order to promote reproducible research, the MATLAB network implementation has been posted in GitHub.