Evaluating ontologies: Towards a cognitive measure of quality

Ontologies are formal specifications of shared conceptualizations of a domain. Important applications of ontologies include distributed knowledge-based systems, such as the semantic web, and the evaluation of modelling languages, e.g. for business process or conceptual modelling. These applications require formal ontologies of good quality. The quality of a formal ontology requires both a good conceptualization of a domain and a good specification of the conceptualization. In this paper, we focus on the latter aspect, and present a method to test how well a specification of a formal ontology corresponds to a conceptualization of a domain held by ontology users. Our experimental method is based on principles of cognitive psychology. We present two experiments to demonstrate our method using upper-level ontologies.