Basis iteration for reward based dimensionality reduction

We propose a linear dimensionality reduction algorithm that selectively preserves task relevant state data for control problems modeled as Markov decision processes. The algorithm works by alternating value function estimation with basis vector adaptation. The approach is demonstrated on two tasks: a toy task designed to illustrate the key concepts, and a more complex three dimensional navigation task.