Being Literate with Large Document Collections: Observational Studies and Cost Structure Tradeoffs

How do people work with large document collections? We studied the effects of different kinds of analysis tools on the behavior of people doing rapid large-volume data assessment, analysis and organization. We analyzed the micro-structure details of using automated clustering techniques, the use of standard timeline and cluster visualization methods, alongside desktop paper sorting and piling. We find that the "natural" methods people use (with piles of paper documents) are in fact very sophisticated and have a subtlety that is lacking in current computer interfaces. This analysis shows that the lack of expressiveness and responsiveness in current interface designs dramatically limits human performance, suggesting ways in which the next generation of analytic tools must evolve in order to support literate use of large volume/complex document collections.