Research on the Algorithm of Hadoop-Based Spatial-Temporal Outlier Detection

A spatial-temporal outlier is an object whose non-spatial attribute value is significantly different from those of other objects in its spatial and temporal neighbors. Identifying or detecting spatial-temporal outliers will help us find some unexpected, interesting and useful knowledge in many application fields, for example: financial fraud detection, fault diagnosis, network intrusion detection and so on. However, the existing spatial-temporal outlier detection algorithms can´t efficiently deal with big dataset. In this paper, a Hadoop-based spatial-temporal outlier detection algorithm is proposed. This approach takes the spatial autocorrelation into consideration. Therefore, the weight is introduced in the approach. However, the calculation involved in calculating weight is significantly large. Besides, the big dataset needs to be processed in this approach. Therefore, Hadoop is used to improve it´s performance. The Ningbo sea tide dataset is used to validate the effectiveness and scalability of this approach.