Analytical usability evaluation for digital libraries: a case study

There are two main kinds of approach to considering usability of any system: empirical and analytical. Empirical techniques involve testing systems with users, whereas analytical techniques involve usability personnel assessing systems using established theories and methods. We report here on a set of studies in which four different techniques were applied to various digital libraries, focusing on the strengths, limitations and scope of each approach. Two of the techniques, heuristic evaluation and cognitive walkthrough, were applied in text-book fashion, because there was no obvious way to contextualize them to the digital libraries (DL) domain. For the third, claims analysis, it was possible to develop a set of reusable scenarios and personas that relate the approach specifically to DL development. The fourth technique, CASSM, relates explicitly to the DL domain by combining empirical data with an analytical approach. We have found that heuristic evaluation and cognitive walkthrough only address superficial aspects of interface design (but are good for that), whereas claims analysis and CASSM can help identify deeper conceptual difficulties (but demand greater skill of the analyst). However, none fit seamlessly with existing digital library development practices, highlighting an important area for further work to support improved usability.