Title :
Monitoring Hadoop by Using IEEE1888 in Implementing Energy-Aware Thread Scheduling
Author :
Hiroaki Takasaki;Samih M. Mostafa;Shigeru Kusakabe
Author_Institution :
Grad. Sch. of Inf. Sci. &
Abstract :
Hadoop is a software framework for writing applications that process huge amount of data in parallel typically on clusters of commodity computing nodes. In implementing our energy-aware scheduling framework on a Hadoop system consisting of commodity nodes, we use IEEE1888, an open network protocol, Ubiquitous Green Community Control Network Protocol. Our scheduling framework is originally developed for single-node commodity platforms and consists of a kernel level thread scheduler, API, a monitor and a controller for the behavior of the scheduler. In order to implement this framework on a Hadoop cluster, we monitor and control the slave nodes over the network by using IEEE1888. Our framework monitors performance metrics and adjusts the behavior of the scheduler through the API. The framework can effectively reduce energy consumption for memory-intensive workload on thread-enabled Hadoop.
Conference_Titel :
Ubiquitous Intelligence and Computing, 2014 IEEE 11th Intl Conf on and IEEE 11th Intl Conf on and Autonomic and Trusted Computing, and IEEE 14th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UTC-ATC-ScalCom)
DOI :
10.1109/UIC-ATC-ScalCom.2014.55