Combining Hard and Soft Competition in Information-Theoretic Learning

In this paper, we try to combine conventional competitive learning with information-theoretic methods to improve competitive performance. We have so far proposed a new type of information-theoretic method to simulate competitive processes. Though the information-theoretic method solves the dead neuron problem and shows the soft-type competition, the method is sometime slow in convergence. To solve this problem, we combine standard learning with information-theoretic learning. By this combination, we can shorten a learning process considerably