Title :
Distributed data access and resource management in the D0 SAM system
Author :
Terekhov, I. ; Pordes, R. ; White, V. ; Lueking, L. ; Carpenter, L. ; Schellman, H. ; Trumbo, J. ; Veseli, S. ; Vranicar, M. ; White, S.
Author_Institution :
D0 Collaboration, Fermi Nat. Accel. Lab., Batavia, IL, USA
Abstract :
SAM (Sequential Access through Meta-data) is the data access and job management system for the D0 high energy physics experiment at Fermilab. The SAM system is being developed and used to handle the Petabyte-scale experiment data, accessed by hundreds of D0 collaborators scattered around the world. In this paper, we present solutions to some of the distributed data processing problems from the perspective of real experience dealing with mission-critical data. We concentrate on the distributed disk caching, resource management and job control. The system has elements of Grid computing and has features applicable to data-intensive computing in general
Keywords :
cache storage; distributed databases; distributed programming; high energy physics instrumentation computing; information retrieval; meta data; operating systems (computers); resource allocation; D0 SAM system; D0 collaborators; D0 high-energy physics experiment; Fermilab; Grid computing; Petabyte-scale experiment data; data-intensive computing; distributed data access; distributed data processing; distributed disk caching; distributed job control; distributed resource management; job management system; meta-data; mission-critical data; sequential access; Catalogs; Collaboration; Data processing; Energy management; Grid computing; History; Laboratories; Mission critical systems; Particle scattering; Resource management;
Conference_Titel :
High Performance Distributed Computing, 2001. Proceedings. 10th IEEE International Symposium on
Conference_Location :
San Francisco, CA
Print_ISBN :
0-7695-1296-8
DOI :
10.1109/HPDC.2001.945179
