POLI: Interactive multimedia authoring and retrieval on mobile phone by leveraging its audio channel and coded light

Connecting digital information with physical objects can enrich their content and make them interactive. In this paper, we present POLI, which allows a user to author digital content to physical objects using a mobile authoring app and another user to explore the authored content using a mobile client app. The mobile apps discover their user´s location by decoding a gray code sequence received from a light sensor which is plugged in the phone´s audio jack. To make the authoring process natural and easy, POLI provides three novel gesture based interactive approaches, which follow people´s intuitive understandings of an object´s boundary.