An approach for MapReduce based log analysis using Hadoop

Log is the main source of system operation status, user behavior, system´s actions etc. Log analysis system needs not only the massive and stable data processing ability but also the adaptation to a variety of scenarios under the requirement of efficiency and performance, which can´t be achieved from available standalone analysis tools or even single computing framework. Hence we propose a log analyzer with the combination of Hadoop and Map-Reduce paradigm. The joint of Hadoop and MapReduce programming tools makes it possible to provide batch analysis in minimum response time and in-memory computing capacity in order to process log in a high available, efficient and stable way.