Toward benchmarks to assess advancement in legal requirements modeling

As software engineers create and evolve information systems to support business practices, these engineers need to address constraints imposed by laws, regulations and policies that govern those business practices. Requirements modeling can be used to extract important legal constraints from laws, and decide how, and evaluate if an information system design complies to applicable laws. To advance research on evaluating requirements modeling formalisms for the representation of legal information, we propose several benchmarks that we believe represent important challenges in modeling laws and requirements governing information systems, and evaluating the compliance of these requirements with laws. While incomplete, the proposed set of benchmarks covers a range of challenges in modeling laws and requirements that we observed in privacy and security law: from the possibility to trace model fragments to law fragments, to the ability to distinguish modalities in law, and to model relations between requirements and law fragments, needed when evaluating compliance. Benchmarks can be used as a checklist when designing and discussing requirements formalisms that support legal requirements modeling. Each benchmark is motivated by related work, a brief legal excerpt, and our experience in modeling regulations.