Bayesian ying-yang theory for empirical learning, regularization and model selection: general formulation

The Bayesian ying-yang learning system and theory developed by the present author (1995, 1996) is further elaborated in a general formulation, focusing on systematically introducing the key points of the theory for empirical learning, data smoothing based regularization, structural regularization, and model selection. Moreover, discussions have been made on the relationship and difference between this theory and the existing approaches, especially the Helmholtz machine, information theory as well as information geometry theory