Bagging of duo output neural networks for single output regression problem

This paper presents an approach to the single output regression problem using ensemble of duo output neural networks based on bagging technique. Each component in the ensemble consists of a pair of duo output neural networks. The first neural network is trained to provide duo outputs which are a pair of truth and falsity values whereas the second neural network provides a pair of falsity and truth values. The target outputs used to train the second network are organized in reverse order of the first network. For the former neural network, the truth and non-falsity outputs are used to created the average truth output. For the later neural network, the falsity and non-truth outputs are used to provide the average falsity output. In order to combine outputs from components in the ensemble, the simple averaging and the dynamic weighted averaging techniques are used. The weight is created based on the difference between the truth and non-falsity values. The proposed approach has been tested with three benchmarking UCI data sets, which are housing, concrete compressive strength, and computer hardware. The proposed ensemble methods improves the performance as compared to the traditional ensemble of neural networks, the ensemble of complementary neural networks, and the ensemble of support vector machine with linear, polynomial, and radial basis function kernels.