Massively Scalable Web Service Discovery

The increasing popularity of Web services (WS) has exemplified the need for scalable and robust discovery mechanisms. Although decentralized solutions for discovering WS promise to fulfill these needs, most make limiting assumptions concerning the number of nodes and the topology of the network and rely on having information on the data a-priori (e.g. categorizations or popularity distributions). In addition, most systems are tested via simulations using artificial datasets. In this paper we introduce a lightweight, scalable and robust WSDL discovery mechanism based on real-time calculation of term popularity. In order to evaluate this mechanism, we have collected and analyzed real data from deployed WS and performed a large-scale emulation on the DAS-3 distributed supercomputer. Results show that we can achieve Web-scale service discovery based on term search and we also sketch an extension of this mechanism to support a fully-fledged WS query language.