Research on HDFS-based web server cluster

With the increasing popularity of the internet and ecommerce, all kinds of Web-based applications become the focus of software development for its convenient, fast and easy operation, simultaneously, most of these web applications have a large number access and concurrent access volume. This requires that web applications have higher performance, especially higher concurrent performance. One main approach to improve web application performance is the clustering technology. Web server cluster is a system consisting of several servers which run the same web application server, the system acts as one server for clients. Web cluster can dispatch the requests to different nodes to improve the effectiveness and stability of the entire application. The traditional web cluster has obtained some achievements, their form of organization can be classified as two types: one is based on the DNS, the other based on distributor (Dispatcher), but all run on the application server level, this paper presents one web cluster which built on a distributed file system HDFS, this kind of cluster can take advantage of the schedule and load balancing algorithm of the HDFS to achieve its scheduling and load balance purpose, which is a innovative way to build the web cluster.