Investigation of Japanese Onomatopoeias as Features for SHITSUKAN-Based Image Retrieval

The term "shitsukan" refers to the perception of materials and surface qualities of natural and manmade objects. The main goal of our study is to extend the current content-based image retrieval (CBIR) systems to develop a shitsukan-based image retrieval (SBIR) system. This paper focuses on Japanese onomatopoeias as a feature for SBIR and verifies their suitability based on psychophysical experiments. In this study, we conducted two different experiments. In the first experiment, subjects assigned suitable onomatopoeias to 50 test images. In the second experiment, they selected perceptually similar images for each test image from the other 49 test images. Then, we investigated the relationship between the assigned onomatopoeias and the selected similar images. The results indicate that perceptually similar images were assigned to the same onomatopoeias with correlation, and that onomatopoeias were effective as a feature for SBIR.