A Framework for Making Face Detection Benchmark Databases

The images in face detection benchmark databases are mostly taken by consumer cameras, and thus are constrained by popular preferences, including a frontal pose and balanced lighting conditions. A good face detector should consider beyond such constraints and work well for other types of images, for example, those captured by a surveillance camera. To overcome such constraints, a framework is proposed to transform a mother database, originally made for benchmarking face recognition, to daughter datasets that are good for benchmarking face detection. The daughter datasets can be customized to meet the requirements of various performance criteria; therefore, a face detector can be better evaluated on desired datasets. The framework is composed of two phases: 1) intrinsic parametrization and 2) extrinsic parametrization. The former parametrizes the intrinsic variables that affect the appearance of a face, and the latter parametrizes the extrinsic variables that determine how faces appear on an image. Experiments reveal that the proposed framework can generate not just data that are similar to those available from popular benchmark databases, but also those that are hardly available from existing databases. The datasets generated by the proposed framework offer the following advantages: 1) they can define the performance specification of a face detector in terms of the detection rates on variables with different variation scopes; 2) they can benchmark the performance on one single or multiple variables, which can be difficult to collect; and 3) their ground truth is available when the datasets are generated, avoiding the time-consuming manual annotation.