Hierarchical preference learning for light control from user feedback

We propose a system for optimized light control in smart homes considering both energy efficiency and user preference. The method is based on learning the user preferences online and under different states (time, location, activity). To achieve adaptive and interactive learning of user preferences, we propose to use hierarchical reinforcement learning (HRL) to adapt the user model dynamically from user feedback. The input to HRL is user´s activity obtained from a two-level vision analysis from a camera network. The input includes the user´s position and fine-level activities such reading, eating and cutting. HRL learns user´s preferences when the user gives feedback to the system through changing the offered light setting. The strength of HRL compared to regular reinforcement learning is that due to state abstraction the number of routines is significantly smaller than the number of actual states, therefore the convergence can be significantly expedited. As more feedback is given by the user, HRL refines the preferences for individual states within the routines. The optimal light intensity level is determined as a balance between user satisfaction and energy cost.