Vector Quantization for State-Action Map Compression - Comparison with Coarse Discretization Techniques and Efficiency Enhancement

We have proposed to use vector quantization (VQ) for compressing a decision making rule (a policy) on a multi-dimensional memory array. Our VQ method can reduce the amount of memory for recording a policy. In this paper, we compare this method with other techniques that economize the amount of memory in order to measure the ability of this method more rigidly than ever. One of the techniques is tile coding, which is frequently used for reinforcement learning. The other is the mere reduction of the resolution of a policy. Moreover, we try applying VQ to already compressed policies. As a result, vector quantized policies and double vector quantized policies could mark better performance than the others.