Assessing identification of compliance requirements from privacy policies

In the United States, organizations can be held liable by the Federal Trade Commission for the statements they make in their privacy policies. Thus, organizations must include their privacy policies as a source of requirements in order to build systems that are policy-compliant. In this paper, we describe an empirical user study in which we measure the ability of requirements engineers to effectively extract compliance requirements from a privacy policy using one of three analysis approaches-CPR (commitment, privilege, and right) analysis, goal-based analysis, and non-method-assisted (control) analysis. The results of these three approaches were then compared to an expert-produced set of expected compliance requirements. The requirements extracted by the CPR subjects reflected a higher percentage of requirements that were expected compliance requirements as well as a higher percentage of the total expected compliance requirements. In contrast, the goal-based and control subjects produced a higher number of synthesized requirements, or requirements not directly derived from the policy than the CPR subjects. This larger number of synthesized requirements may be attributed to the fact that these two subject groups employed more inquiry-driven approaches than the CPR subjects who relied primarily on focused and direct extraction of compliance requirements.