Interpreting and improving multi-layered networks by free energy-based competitive learning

In this paper, we propose a new algorithm to interpret and improve multi-layered neural networks. The new method is used to simplify and interpret final representations obtained by conventional learning methods such as BP or RBF. In addition, the method is used to retrain networks so as to produce networks with better performance in terms of generalization errors. Neural networks have so far produced effective internal representations for many problems. However, because obtained information is extremely distributed over many elements, it is difficult to interpret the meaning of representations. For this problem, we use competitive learning by which we can focus upon some elements in networks. In addition, because the number of effective elements in neural networks can significantly be reduced, we can expect improved performance in terms of generalization. Competitive processes are considered to be a process of entropy minimization in which a small number of units acquires the majority of information on input patterns. For simplifying computation, we use a free energy in which entropy and training errors are simultaneously used. We applied the methods to an artificial data and a cabinet approval rating estimation problem. In both problems, we succeeded in extracting the main features in input patterns and improved generalization performance could be obtained.