SPOAN: Load Balancing Replica Placement Strategy for Large Scale Biometric Identification Service

Large-scale identification system is a distributed data processing system that requires both high throughput and high reliability. In large-scale identification systems, not only the volume of computing is extremely high but also the I/O volume per CPU second is extremely large. Therefore, how one places the data replicas in the computing nodes can have a significant impact on the performance of identification and on the availability of the biometric data sets. However, the traditional replica placement method used in the distributed processing frameworks and distributed storage technologies are unable to rebalance loads among nodes when the specific nodes are down and the throughput of identification slows. To address this problem, we propose a replica placement strategy, called SPOAN, that enables us to rebalance the loads among computing nodes and maintain data more reliably. We also developed the architecture of the large scale identification systems using SPOAN. As the result of evaluations involving emulation of biometric database containing about 1.2 billion individuals, we found that we could rebalance the loads even when specific computing nodes were slowed down unlike traditional placing systems of replicas and improve throughput by up to 46%.