Multistage committees of deep feedforward convolutional sparse denoise autoencoder for object recognition

Deep learning and unsupervised feature learning systems are known to achieve good performance in benchmarks by using extremely large architectures with many features at each layer. However, we found that the number of features´ contribution to performance is very small when it is more than the threshold. Meanwhile, the size of pooling layer has an important influence on performance. In this paper, we present an unsupervised method to improve the classification result by going deep and combining multistage classifiers in a committee with a small amount of features at each layer. The network is trained layer-wise via denoise autoencoder (dA) with L-BFGS to optimize convolutional kernels and no backpropagation is used. In addition, we regularize the dA encouraging representations to fit sparse for each coding layer. We apply it on the STL-10 dataset which has very few training examples and a large amount of unlabeled data. Experimental results show that our method presents higher performance than the existing ones on the condition via individual network.