On the closure of the set of functions that can be realized by a given multilayer perceptron

Author

Gori, Marco ; Scarselli, Franco ; Tsoi, Ah Chung

Author_Institution

Dept. d´´Ingegneria dell´´Inf., Siena Univ., Italy

Volume

9

Issue

6

fYear

1998

fDate

11/1/1998 12:00:00 AM

Firstpage

1086

Lastpage

1098

Abstract

Given a multilayer perceptron (MLP) with a fixed architecture, there are functions that can be approximated up to any degree of accuracy, without having to increase the number of the hidden nodes. Those functions belong to the closure F¯ of the set F¯ of the maps realizable by the MLP. In this paper, we give a list of maps with this property. In particular, it is proven that: 1) rational functions belongs to F¯ for networks with inverse tangent activation function; and 2) products of polynomials and exponentials belongs to F¯ for networks with sigmoid activation function. Moreover, for a restricted class of MLPs, we prove that the list is complete and give an analytic definition of F¯

Keywords

function approximation; multilayer perceptrons; polynomial approximation; rational functions; function approximation; inverse tangent activation function; multilayer perceptron; neural nets; polynomials; rational functions; sigmoid activation function; Australia; Computer networks; Informatics; Logistics; Multi-layer neural network; Multilayer perceptrons; Neural networks; Neurons; Polynomials; Vectors;

fLanguage

English

Journal_Title

Neural Networks, IEEE Transactions on

Publisher

ieee

ISSN

1045-9227

Type

jour

DOI

10.1109/72.728354

Filename

728354