Past-future Information Bottleneck for linear feedback systems

We present an information-theoretically motivated method for dimensionality reduction and realization of linear dynamical systems. Given a linear system with Gaussian inputs, we apply the Information Bottleneck principle to construct a continuous range of reduced-order systems, each satisfying an optimal trade-off between the compression rate of past inputs to, and the prediction accuracy of future outputs from the original system. We apply a certain variant of the Ho-Kalman algorithm to obtain realizations of the reduced-order systems and show that they lie near the optimal Information Curve. We explore the behavior of a set of systems realized using the method and discuss an extension of the method to closed-loop linear systems.