On the adaptive control of a partially observable Markov decision process

The study represents the initial stages of a program to address the adaptive control of partially observable Markov decision processes (POMDP) with finite state, action, and observation spaces. The authors review the results on the control of POMPD with known parameters and, in particular, the results on the control of quality control/machine replacement models. They study the adaptive control of a problem with simple structure: the two-state binary replacement problem. An adaptive control algorithm is defined, and initial results in the direction of using the ODE method are obtained