Electronic neural net algorithm for maximum entropy solutions to ill-posed problems. II. Multiply connected electronic circuit implementation

For pt.I see ibid., vol.36, no.2, p.288-94 (1989). Using standard electronic components, the authors have constructed a multiply connected analog circuit that minimizes a specific cost function of its external inputs, interconnects, and node characteristics. The circuit is designed so that the cost function contains an informational entropy regularizer to give maximum-entropy solutions to ill-posed problems. The circuit performs a gradient search to achieve the minimization in a time defined by a circuit RC time constant rather than the complexity of the problem. The circuit reaches a stable equilibrium condition irrespective of the initial values of its outputs. For the circuit considered, once a partial compensation was made for the input offset voltages of the operational amplifiers used, the solution generated by the circuit was close to the 0.1% estimated for an ideal circuit. Settling times of about 15 ms were observed. The nature of the particular problem chosen required extremely high gains on the constraint nodes in the circuit. The resulting stability problems were overcome by slowing the response of the signal nodes in the circuit