Variable binding in a neural network using a distributed representation

A rigorous definition of the notion of distribution used in distributed connectionist models was put forward by Van Gelder (1992). According to this, the representations formed in a standard feedforward network are distributed as they are context-dependent compositions of the input constituents. It has been argued that distributed representations cannot display compositionality, but some connectionist models demonstrate a form of functional compositionality when performing structure-sensitive tasks. For example, a model using distributed representations was used to perform the passivisation of English sentences. This demonstrates that functionally compositional distributed representations can perform structure sensitive operations without extracting the individual constituents, and provides evidence that distributed connectionist systems can be both compositional and systematic