Extracting deep social relationships from photos

Hidden within the relative location of tags in images is a relational model that can identify how close two individuals are, or, the affinity of a person to an object or a brand. Based on this model we can 1) better understand the relationship between users/tags, 2) find photos where a user is pictured but not tagged in, and 3) enable searching “inside” images by clicking on any location within an image to start a search. This paper proposes a method of modeling the relationship between objects based on their spatial arrangement in a set of tagged images. Based on the relative coordinates of each object tag, we compute a joint relativity between each tag pair, generate a social relationship graph and propose an efficient image search method using the joint Relativity graph. We evaluated our approach with real world data from Facebook, showing a direct relationship between the number of tagged photos and the amount of information obtained from these photos, and an average correlation coefficient of 0.8 between user-generated relativity scores and those obtained by our algorithm.