Direct Updating of Intertemporal Criterion Functions for a Class of Adaptive Control Problems

Previous papers demonstrate the feasibility of direct criterion function updating for a class of adaptive control problems with sequential myopic expected cost objectives. In contrast, the present paper demonstrates the feasibility ofdirect criterion function updating for a class of adaptive control problems with intertemporal expected cost objectives. Specifically, the proposed criterion filter sequentially updates an initial estimate for expected cost plus cost-to-go in a dynamic programming framework. The control law generated by the filtered expected cost plus cost-to-go estimates is tested against the optimal control law for a linear-quadratic system with random state coefficients. Computer simulation results indicate that the total costs realized under the filter control law are approximately on par with the total costs realized under the optimal control law for the tested range of time horizons, cost function coefficients, and mean and standard deviation values for the random state coefficients.