The study on large scale image processing architecture based on Hadoop2.0 clusters

This paper describes the DeepView platform which is a pilot system to classify large scaled images collected from a VMS server based on Hadoop 2.0 clusters. Multiple DeepView Classifier tasks analyze images simultaneously to detect objects in the images. Classifier task is implemented as a direct YARN task instead of MapReduce task to avoid intensive disk I/O and limited input format. Moreover, the small data access problem of Hadoop can be avoided because Application Master controls YARN tasks to access only local blocks of image files before image classification starts.