Recurrent Clustering for Unsupervised Feature Extraction with Application to Sequence Detection

In many unsupervised learning applications both spatial and temporal regularities in the data need to be represented. Traditional clustering algorithms, which are commonly employed by unsupervised learning engines, lack the ability to naturally capture temporal dependencies. In supervised learning methods, temporal features are often learned through the use of a feedback (or recurrent) signal. Drawing inspiration from the Elman recurrent neural network, we introduce a winner-take-all based recurrent clustering algorithm that is able to identify temporal regularities in an unsupervised manner. We explore the potential pitfalls that result from adding feedback to an incremental clustering algorithm and apply the proposed technique to several time series inference problems in the context of semi-supervised learning. The results clearly indicate that the framework can be broadly applied with particular relevance to scalable deep machine learning architectures.