Performance Evaluation of Cognitive Radios: Metrics, Utility Functions, and Methodology

Performance evaluation of cognitive radio (CR) networks is an important problem but has received relatively limited attention from the CR community. Unlike traditional radios, a cognitive radio may change its objectives as radio scenarios vary. Because of the dynamic pairing of objectives and contexts, it is imperative for cognitive radio network designers to have a firm understanding of the interrelationships among goals, performance metrics, utility functions, link/network performance, and operating environments. In this paper, we first overview various performance metrics at the node, network, and application levels. From a game-theoretic viewpoint, we then show that the performance evaluation of cognitive radio networks exhibits the interdependent nature of actions, goals, decisions, observations, and context. We discuss the interrelationships among metrics, utility functions, cognitive engine algorithms, and achieved performance, as well as various testing scenarios. We propose the radio environment map-based scenario-driven testing (REM-SDT) for thorough performance evaluation of cognitive radios. An IEEE 802.22 WRAN cognitive engine testbed is presented to provide further insights into this important problem area.