Augmenting physical books towards education enhancement

This paper presents an interactive desk that augments physical books that are placed upon its surface with multimedia content and interactive applications. Such content is dynamically displayed in augmentation to the currently open page of the book, that is, aligned in realtime with its 2D orientation upon the desk. The rendered applications are controlled by the users with the use of a stylus, both through contact with the book or desk, as well as, through a small vocabulary of gestures performed with the stylus. The evaluation of the accuracy, robustness, and performance of the proposed computer vision modules supporting this interaction are reported through quantitative experiments. In addition, the system usability was validated and the suitability of educational applications was explored through pilot applications, which include music and digital animation on pages, content-based multimedia presentation, context-based online search, as well as, note-taking through handwriting.