Title :
Experiments on Networking of Hadoop
Author :
Navaz, Abdul ; Velusam, Gandhimathi ; Gurkan, Deniz
Abstract :
Hadoop is a popular application to analyze and process big data problems in a networked distributed system of computers. Investigations of performance for application-aware networking have been of interest with the software-defined networking paradigm through on-demand and dynamic policy enforcements. Network usage characterization of Hadoop jobs can further help understand what policy enforcements may be needed during application use cases. At scale experimentation of Hadoop jobs will help facilitate such characterizations. We report how Hadoop networking usage can be characterized in an experimentation environment using the GENI (Global Environment for Network Innovation). Furthermore, we report a distributed switch framework that may help alleviate the fault tolerance schemes in Hadoop application in the forwarding plane. Delay in recovery from failures has been reduced by almost 99% through such a distributed switch architecture deployed on the GENI experimentation environment.
Keywords :
data handling; fault tolerance; parallel processing; software defined networking; GENI experimentation environment; Hadoop jobs; Hadoop networking; application aware networking; big data problems; distributed switch architecture; distributed switch framework; dynamic policy enforcements; fault tolerance; global environment; network innovation; network usage characterization; networked distributed system; software defined networking paradigm; Artificial neural networks; Computer architecture; Fault tolerance; Fault tolerant systems; Heart beat; Switches; Switching systems; Big Data; GENI; Hadoop; Software Defined Networks (SDN);
Conference_Titel :
Network Protocols (ICNP), 2014 IEEE 22nd International Conference on
Conference_Location :
Raleigh, NC
Print_ISBN :
978-1-4799-6203-7
DOI :
10.1109/ICNP.2014.87
