Using recurrent networks for non-temporal classification tasks

In recent years, deep neural networks have led to considerable advances in the performance of neural network architectures. However, deep architectures tend to have a large numbers of parameters, leading to long training times and the need for huge amounts of training data and regularization. In addition, biological neural networks make extensive use of recurrent and feedback connections, which are absent for most commonly used deep architectures. In this paper, we investigate the use of recurrent neural networks as an alternative to deep architectures. The approach replaces depth with recurrent computations through time. It can also be seen as a deep architecture with parameter tying. We show that for a comparable numbers of parameters or complexity, replacing depth with recurrency can result in improved performance.