Vessel route anomaly detection with Hadoop MapReduce

Author

Xiaoguang Wang ; Xuan Liu ; Bo Liu ; de Souza, Erico N. ; Matwin, S.

Author_Institution

Fac. of Comput. Sci., Dalhousie Univ., Halifax, NS, Canada

fYear

2014

fDate

27-30 Oct. 2014

Firstpage

25

Lastpage

30

Abstract

We present a two-level approach to detect abnormal activities for vessels´ routes. The data is obtained from the Automatic Identification System (AIS) which is required to be installed on vessels over specific gross tonnage. In the first level, we develope a Clustering algorithm: Density-based Spatial Clustering of Applications with Noise considering Speed and Direction (DBSCAN_SD). This algorithm is applied to pre-cluster the data points. Using domain knowledge in maritime, experts adjust the results produced by DBSCAN_SD with extra features. In this way, we get the optimal labeling result about whether a data point is normal or abnormal. In the second level, we use the labeled data generated in the first level to train the Parallel Meta-Learning (PML) algorithm on Hadoop. The results show that both accuracy and time complexity results are improved when we increase the number of nodes in a cluster.

Keywords

data handling; learning (artificial intelligence); marine engineering; marine vehicles; parallel processing; pattern clustering; security of data; AIS; DBSCAN_SD; Hadoop MapReduce; PML algorithm; abnormal activities detection; automatic identification system; clustering algorithm; data points preclustering; density-based spatial clustering of applications with noise; domain knowledge; optimal labeling; parallel meta-learning; speed and direction; two-level approach; vessel route anomaly detection; Classification algorithms; Clustering algorithms; Computational modeling; Machine learning algorithms; Noise; Training; Trajectory; AIS data; Anomaly detection; Hadoop; MapReduce;

fLanguage

English

Publisher

ieee

Conference_Titel

Big Data (Big Data), 2014 IEEE International Conference on

Conference_Location

Washington, DC

Type

conf

DOI

10.1109/BigData.2014.7004464

Filename

7004464