Nearest neighbour based algorithm for data reduction and fault diagnosis

Dimensionality reduction is one of the prime concerns when analyzing process historical data for plant-wide monitoring, because this can significantly reduce computational load during statistical model building. Most research has been concerned with reducing the dimension along the variable space, i.e. reducing the number of columns. However, no efforts are made to reduce dimensions along the sample (row) space. In this paper, an algorithm based on nearest neighbor is presented here that exploits the principle of distributional equivalence (PDE) property of the correspondence analysis (CA) algorithm to achieve data reduction along the sample space without significantly affecting the diagnostic performance. The data reduction algorithm presented here is unsupervised and can achieve significant data reduction when used in conjunction with CA. The data reduction ability of the proposed methodology is demonstrated using the benchmark Tennessee Eastman process simulation case study.