Reassessing Combinatorial Productivity Exhibited by Simple Recurrent Networks in Language Acquisition

It has long been criticized that connectionist models are inappropriate models for language acquisition since one of the important properties, the property of generalization beyond the training space, cannot be exhibited by the networks. Recently van der Velde et al. have discussed the issue of the combinatorial productivity, arguing that simple recurrent networks (SRNs) fail in this regard. They have attempted to show that performance of SRNs on generalization is limited to word-word association. In this paper, we report our follow-up study with two simulations demonstrating that (i) bi-gram does not play the dominant role as claimed (ii) SRNs are indeed able to exhibit combinatorial productivity when appropriately trained.