Thumbnail
Access Restriction
Open

Author Potekhin, M. ♦ Wenaus, T. ♦ Ito, H.
Source CERN Document Server
Content type Text
File Format PDF
Date Created 2012-05-08
Language English
Subject Domain (in DDC) Natural sciences & mathematics ♦ Physics ♦ Modern physics ♦ Technology ♦ Engineering & allied operations ♦ Applied physics
Subject Keyword database ♦ Computing and Computers ♦ noSQL ♦ Django ♦ Cassandra
Abstract For several years the PanDA Workload Management System has been the basis for distributed production and analysis for the ATLAS experiment at the LHC. Since the start of data taking PanDA usage has ramped up steadily, typically exceeding 500k completed jobs/day by June 2011. The associated monitoring data volume has been rising as well, to levels that present a new set of challenges in the areas of database scalability and monitoring system performance and efficiency. These challenges are being met with an R&D effort aimed at implementing a scalable and efficient monitoring data storage based on a noSQL solution (Cassandra). We present our motivations for using this technology, as well as data design and the techniques used for efficient indexing of the data. We also discuss the hardware requirements as they were determined by testing with actual data and realistic rate of queries. In conclusion, we present our experience with operating a Cassandra cluster over an extended period of time and with data load adequate for planned application.
Description Presented at: J. Phys.: Conf. Ser. 396 (2012) 052041 Computing in High Energy and Nuclear Physics 2012, New York, NY, USA, 21 - 25 May 2012, pp.052041
Collaboration with: for the ATLAS
Learning Resource Type Article
Publisher Date 2012-01-01
Rights License Preprint: (License: CC-BY-4.0)
Page Count 6