Data Mining and Service Rating in Service-Oriented Architectures to Improve Information Sharing

The evolution of Web services and service oriented architectures (SOA) has enabled removal of many technological barriers that have historically limited information sharing across the United States intelligence community (IC). During the past decade, Internet and Web technologies have facilitated the exponential growth of data and information available creating information overload and forcing the need for tools to manage information. Just as the Internet brought about information overload, the growth in development and deployment of services will lead to service overload in the coming decade. To address the challenge, this paper proposes an extended SOA infrastructure to facilitate service discovery and automate service orchestration by using data mining and service rating technologies. Data mining, the process of extracting patterns and knowledge hidden from large volumes of raw data, includes techniques such as classification, clustering, association rules, and sequential patterns. Service rating, on the other hand, involves techniques of capturing and incorporating user review and ratings on service performance, effectiveness, and reliability. By applying these technologies to an analytical environment within the IC, this paper demonstrates how they can help with service categorization, service discovery, automated service composition and construction of dynamic communities of interest (COIs), which empowers analysts to collaboratively solve complex problems