The coding of literary form: Data mining and the information structure of historical texts

This working paper argues that many data-mining projects in the humanities limit themselves by choosing words as their default unit of analysis. Some authors, problems, and forms are better illuminated by analysis of individual textual symbols, others by examination of multiword constructions. Insights about the nature of code from mathematical information theory, long but perhaps prematurely rejected by humanists on theoretical grounds, may give researchers less subjective and more powerful tools by which to measure the information characteristics of texts and the innovations of specific historical writers.