Quality vs. quantity: comparing evaluation methods in a usability-focused software architecture modification task

A controlled experiment was performed to assess the usefulness of portions of a usability-supporting architectural pattern (USAP) in modifying the design of software architectures to support a specific usability concern. Results showed that participants using a complete USAP produced modified designs of significantly higher quality than participants using only a usability scenario. Comparison of solution quality ratings with a quantitative measure of responsibilities considered in the solution showed positive correlation between the measures. Implications for software development are that usability concerns can be included at architecture design time, and that USAPs can significantly help software architects to produce better designs to address usability concerns. Implications for empirical software engineering are that validated quantitative measures of software architecture quality may potentially be substituted for costly and often elusive expert assessment.