Data-smoothing regularization, normalization regularization, and competition-penalty mechanism for statistical learning and multi-agents

This paper provides an overview on advances of two new learning regularization approaches, both are developed in the past several years from the studies of Bayesian Ying Yang learning (BYY). The first is data smoothing regularization, which was firstly proposed in (Xu, 1997a) for parameter learning in a way similar to Tikhonov regularization but with an easy solution to the difficulty of determining an appropriate hyperparameter. The second is normalization regularization firstly proposed in (Xu, 2001b) which regularizes parameter learning via de-learning of conscience or penalizing type and has a close relation to the rival penalized competitive learning (RPCL) (Xu, Krzyzak, & Oja, 1993). Also, the algorithms for the two types of regularized learning versus the algorithms for maximum likelihood learning and the RPCL learning are presented in a unified learning procedure. Moreover, studies on the competition-penalty mechanism are further elaborated, and this mechanism, especially RPCL mechanism, is suggested to monitoring the performances of multi-agents.