Using information entropy bounds to design VLSI friendly neural networks

This paper presents a method for calculating the minimal size of a VLSI optimal network for a given problem. A VLSI optimal network is a network using integer weights in a given range [-p,p] and units with a small constant fan-in. The value p is a small integer which is calculated from the problem parameters such that the problem is guaranteed to have a solution. It is shown that the number of weights can be lower bounded in the worst case by the expression: m(n+1)[nlog( _dmin/^dmx)+(n-1)log(_dmin/¹ +1)+½log(n-1)-_12.(n-1)^loge+c] where d _min is the minimum distance between patterns of opposite classes, d_max is the maximum distance between any patterns, m is the number of patterns in the largest class, n is the number of dimensions and c is a constant. The methodology is tested on various problems using a limited precision integer weights constructive algorithm