A hardware efficient implementation of a boxes reinforcement learning system

This paper presents two modifications to the Boxes-ASE/ACE reinforcement learning algorithm to improve implementation efficiency and performance. A state history queue (SHQ) replaces the decay computations associated with each control state, decoupling the dependence of computational demand from the number of control states. A dynamic link table implements CMAC state association to decrease training time, yet minimize the number of control states. Simulations of the link table demonstrated its potential for minimizing control states for unoptimized state-space quantization. Simulations coupling the link table to CMAC state association show a 3-fold reduction in learning time. A hardware implementation of the pole-cart balancer shows the SHQ modification to reduce computation time 12-fold