A practical comparison of three robot learning from demonstration algorithms

Research on robot learning from demonstration has seen significant growth in recent years, but existing evaluations have focused exclusively on algorithmic performance and not on usability factors, especially with respect to naïve users. Here we present findings from a comparative user study in which we asked non-experts to evaluate three distinctively different robot learning from demonstration algorithms - Behavior Networks, Interactive Reinforcement Learning, and Confidence Based Autonomy. Participants in the study showed a preference for interfaces where they controlled the robot directly (teleoperation and guidance) instead of providing retroactive feedback for past actions (reward and correction). Our results show that the best policy performance in most metrics was achieved using the Confidence Based Autonomy algorithm.