On learning analysis of neural fuzzy systems

Self-constructing neural fuzzy inference network (SONFIN) is a neural fuzzy system and owing to its structure learning capability, SONFIN has been demonstrated to have excellent learning performance. However, various parameters must be selected in the implementation of SONFIN. In this paper, the learning behavior of SONFIN is studied. First, the SONFIN system with different thresholds and variances are considered. Different selections will result in different rule numbers and membership function widths. Our experiment results indicate that when there are possibilities of overfitting, more rules may not always come up with better performance. Secondly, two different learning algorithms are considered; the backpropagation (BP) learning algorithm and the recursive least square (RLS) algorithm. It can found that the learning of using RLS is much faster than that of using BP as expected. However, it can be found that when overfitting may occur, BP can have better learning performance in terms of testing errors. Finally, the use of reset for the covariance matrix in the RLS algorithm is investigated. From this primitive study, it can be found that the learning algorithms or some parameter selections may have both good effects in the testing performance and the training performance before the learning does not have significant overfitting. However, when the learning crosses this point, any selection is good for learning may bring bad effects on the testing performance.