Adaptive sensor-planning algorithm with Q-learning

In this paper, we propose an adaptive learning technique for "sensor-planning", that is how to control PTZ (pan-tilt-zoom) camera sensor for human finding without prior environment information. Our idea is based on Q-learning with positive and negative rewards as optical-flow result and camera motion respectively. Additionally, an adaptive learning rate mechanism enables us to change the control sequence quickly to adapt situation change. This planning method has advantages of real-time processing, online learning and quick adaptation. We show a simulation experiment in pseudo room environment to compare our method with traditional methods. Our algorithm can detect moving object 21 percent faster than probabilistic method