Distributed data access and resource management in the D0 SAM system

Author

Terekhov, I. ; Pordes, R. ; White, V. ; Lueking, L. ; Carpenter, L. ; Schellman, H. ; Trumbo, J. ; Veseli, S. ; Vranicar, M. ; White, S.

Author_Institution

D0 Collaboration, Fermi Nat. Accel. Lab., Batavia, IL, USA

fYear

2001

fDate

2001

Firstpage

87

Lastpage

92

Abstract

SAM (Sequential Access through Meta-data) is the data access and job management system for the D0 high energy physics experiment at Fermilab. The SAM system is being developed and used to handle the Petabyte-scale experiment data, accessed by hundreds of D0 collaborators scattered around the world. In this paper, we present solutions to some of the distributed data processing problems from the perspective of real experience dealing with mission-critical data. We concentrate on the distributed disk caching, resource management and job control. The system has elements of Grid computing and has features applicable to data-intensive computing in general

Keywords

cache storage; distributed databases; distributed programming; high energy physics instrumentation computing; information retrieval; meta data; operating systems (computers); resource allocation; D0 SAM system; D0 collaborators; D0 high-energy physics experiment; Fermilab; Grid computing; Petabyte-scale experiment data; data-intensive computing; distributed data access; distributed data processing; distributed disk caching; distributed job control; distributed resource management; job management system; meta-data; mission-critical data; sequential access; Catalogs; Collaboration; Data processing; Energy management; Grid computing; History; Laboratories; Mission critical systems; Particle scattering; Resource management;

fLanguage

English

Publisher

ieee

Conference_Titel

High Performance Distributed Computing, 2001. Proceedings. 10th IEEE International Symposium on

Conference_Location

San Francisco, CA

ISSN

1082-8907

Print_ISBN

0-7695-1296-8

Type

conf

DOI

10.1109/HPDC.2001.945179

Filename

945179