MDP and learning based approach for ubiquitous services composition

The issue of services composition to offer seamless access to a variety of high level services has received widespread attention in recent years. The proposed approaches have been mainly issued from the research undertaken jointly on Workflow and AI-based planning techniques. These methods are based in general on strong assumptions: static environment, deterministic invocation of the services. However, the number of service invocation failures increase considerably due to the dynamic and uncertain nature of the ubiquitous environment. These failures can cause a degradation of the quality of the provided services and affect their continuity. Moreover, to select the best adapted service to the user´s context among functionally equivalent services, a quality based selection mechanism is needed. To bring a solution to these problems, we propose in this paper a quality-based services selection approach, coupled with a learning mechanism for ensuring a flexible and failure-tolerant service composition. Our approach is based on the Markov Decision Processes (MDPs) and the Bayesian learning models. The results obtained in the different tests of simulation, show clearly the feasibility of the adopted method and the convergence of the learning mechanism.