Demystifying Cloud Benchmarking Paradigm - An in Depth View

Industry has been boomeranging with "Cloud" in every forte and IaaS (Infrastructure as a Service) layer has always been the backbone of any cloud computing service. The infrastructure layer of any cloud service consists of an array of physical processors along with storage and network components with a virtualization layer on the top that can expose the combined processing power and storage of the array as a set of virtual instances. To ensure that a particular software system has the right virtual infrastructure attributes, it becomes imperative that the cloud infrastructure in question be audited or measured by the software designer before deploying the system on a particular cloud. Whether you are a cloud company that provides this virtual infrastructure or you are a software architect in charge of choosing the right infrastructure provider, you need to have the right set of tools for measuring the performance of the infrastructure. In other words, you need to benchmark the cloud service in specific areas to find out how the service is performing. This is called Cloud Benchmarking. At a conceptual level, cloud benchmarking is not any different from regular computing system benchmarking. However, there is a vast difference between the two when it comes to implementation of the benchmarking methodology and measurement of the critical QOS parameters. This research paper discusses the relevant aspects of cloud benchmarking such as the cloud characteristics to benchmark, the parameters to measure, the role of automation in cloud benchmarking and finally the performance-cost normalization of the benchmarking results so that the cloud provider offering the best value for money can be determined.