Controller design (from scratch) using approximate dynamic programming

Selected adaptive critic (AC) methods are known to implement approximate dynamic programming for determining (approximately) optimal control policies for nonlinear plants. The present research focuses on an AC method known as dual heuristic programming (DHP). There are many issues related to stability that are yet to be addressed for the case of online training of controllers. Nevertheless, now that the operational aspects of the DHP method are becoming refined and better understood it is instructive to carry out empirical research with the method, to inform theoretical research being carried out in parallel. In particular it is seen as useful to explore the important correspondences between the form of a utility function and the resulting controllers designed by the DHP method. The task of designing a steering controller for a 2-axle terrestrial autonomous vehicle is the basis of the empirical work reported here. Three different utility functions are crafted each incorporating additional control objectives. The controllers designed by the DHP method are correspondingly different-but in each case, the resulting controllers have responses that `look and feel´ like ones a human designer might have designed