VDEP: VM Dependency Discovery in Multi-tier Cloud Applications

The automatic discovery of dependencies in distributed Cloud applications is very useful for large scale deployments. Dependencies can be used to identify the anomalies due to errors, failures or the performance bottleneck in applications. Although existing dependency models can be useful, we believe more comprehensive dependency model would improve anomaly detection in large scale distributed applications. We propose a VM dependency discovery system and introduce dependency primitives that incorporate complex application behavior/interaction patterns. We also formulate response time characteristics for each dependency primitive. Using the component dependencies and traffic monitoring, we develop a stochastic model to estimate the response time probability distribution for components and overall application. We evaluate and validate our system with various production applications. Experiments show that we can accurately discover application dependencies and also predict not only the average response time but the 95th percentile response time within 8% of the actual response time.