An analytical approach to hardware-friendly adaptive learning rate neural networks

In this paper hardware implementation of adaptive learning rate neural networks is studied. Some design guidelines are presented to improve integration of learning algorithm into the hardware. By using them, it is possible to design high performance neural networks, which are capable of handling a rapidly-conversing learning algorithm in analog chips. The analytical approach developed in this work provides more insight towards tuning of a reliable design. Our experimental results prove that this approach performs above the conventional fixed learning rate approach, and is almost comparable to the ideal gradient based adaptive approach.