Effect of intrusion detection on secure group communications in hierarchically structured group architectures

We develop a class of adaptive security protocols with designs to allow group communication systems (GCSs) in mobile ad hoc networks (MANETs) to dynamically adjust operational settings to best satisfy application-imposed performance and security requirements, leveraging the inherent tradeoff between security and performance properties of the system. These adaptive security protocols include an intrusion detection protocol for dealing with insider attacks and a scalable region-based hierarchical group key management protocol for dealing with outsider attacks. Our design settings include the time interval over which intrusion detection should be performed, and the regional area size for the region-based hierarchical group key management protocol for group key management. When given a set of parameter values characterizing operational and environmental conditions of a GCS, we identify optimal design settings to be used by the system dynamically to maximize the mean time to security failure of the system while minimizing the total group communication cost incurred for GCSs in MANET environments.