Real-time clothes comparison based on multi-view vision

In this paper, we present a clothing recognition system that augments clothes recommendation and fashion exploration using the intelligent multi-view vision technology of the responsive mirror, an implicitly controlled human-computer interaction system for clothes fitting rooms. The responsive mirror provides shoppers with real-time ldquoselfrdquo and ldquosocialrdquo clothes comparisons. The system recommends clothing that is ldquosimilarrdquo and ldquodifferentrdquo than the clothing that the person is trying on in the mirror. The goal of the research in this paper is to create a recommendation system that uses a definition of ldquosimilarrdquo and ldquodifferentrdquo that matches human perception. We address the social nature of the recognition problem by conducting a user study to identify the salient clothes factors that people use to determine clothes similarity. We describe the computer vision and machine learning techniques employed to recognize the factors that human eyes perceive in term of clothing similarity from frontal-view outfit images. We describe the key components of the motion-tracking and clothes-recognition systems and evaluate their performance by user study and experiments on a simulated clothes fitting image dataset. The approach and results presented here will benefit designers and developers of similar applications in the future.