System Identification with Unbounded Loss Functions Under Algorithmic Deficiency

We describe and analyze a comprehensive learning model to address issues such as consistency, convergence rate, and sample complexity in the general context of system identification. The learning model is based on unbounded loss functions, and it incorporates a measure of algorithmic deficiency. We define and use a novel formulation of algorithmic solution that is an extension of the empirical risk minimization method in the sense that it uses a generic notion of side information as opposed to the commonly used input/output observation of a system. Sufficient conditions for consistency as well as closed form expressions for exponential convergence rate and sample complexity of the identification algorithm are derived