Tattoo based identification: Sketch to image matching

Tattoos on human body provide important clue to the identity of a suspect. While a tattoo is not an unique identifier, it narrows down the list of identities for the suspect. For these reasons, law enforcement agencies have been collecting tattoo images of the suspects at the time of booking. A few successful attempts have been made to design an automatic system to search a tattoo database to identify near-duplicate images of a query tattoo image. However, in many scenarios, the surveillance image of the crime scene is not available, so the query is in the form of a sketch of a tattoo (as opposed to an image of a tattoo) drawn based on the description provided by an eye-witness. In this paper, we extend the capability of tattoo image-to-image matching by proposing a method to match tattoo sketches to tattoo images using local invariant features. Specifically, tattoo shape is first extracted from both tattoo sketch and tattoo image using Canny edge detector. Local patterns are then extracted from tattoo shape as well as tattoo image (appearance) using SIFT A local feature based sparse representation classification scheme is then used for matching. Experimental results on matching 100 tattoo sketches against a gallery set with 10,100 tattoo images show that the proposed method achieves significant improvement (rank-200 accuracy of 57%) compared to a state-of-the-art tattoo image-to-image matching system (rank-200 accuracy of 19%).