From wiener filtering to recent advances on complexity-based system identification and state estimation

Wiener filters laid a foundation for optimal signal estimation under stochastic noises and have influenced development of filtering technologies in signal processing, estimation, identification, and stochastic control of partially observed systems over 50 years. Recent advances in computer and communication technologies have ushered in a new era of identification and estimation in which complexity issues play a central role. This paper summarizes recent development on system identification and state estimation under quantized observations and irregular sampling, and presents new results on decision-based identification in which optimal resource allocation is sought. Adaptive resource allocation algorithms are introduced. The algorithms are shown to converge to the optimal resource allocation by employing the ODE approach in stochastic approximation methodologies. Convergence and convergence rates are established.