Beyond the mechanics of batch control for thousands of jobs submitted each day to the WLCG HEP, the process of carrying out the science on experimental data also requires an open-source approach. All of the collaborators within the experiment have a responsibility to
Beyond the mechanics of batch control for thousands of jobs submitted each day to the WLCG HEP, the process of carrying out the science on experimental data also requires an open-source approach. All of the collaborators within the experiment have a responsibility to make sure the software is constantly improving in its ability to spot significant events in the noise of a billion particles.
It's not impossible to be a physicist without an understanding of C++ and how to optimise code through the compiler, but it is a class taught as part of an undergraduate degree at Sussex and other HEP institutions.
"Our community has been open source since before it was really called open source," says Bird. "We use commercial software where we need to and where we can, but all of our own data analysis programs are written in-house, because there's nothing that exists to do that job."
The size and complexity of ATLAS and CMS means there are hundreds of software professionals employed to create and maintain code, but since many sites will collaborate on different parts of the experiment, tools such as SVN are vital for managing the number of contributions to each algorithm.
"You wouldn't be able to do it on a different operating system," Dr Salvatore says. "It's becoming easier to work with tools like SVN from within the operating system, and this is definitely one of the things you exploit, that you can go as close as possible to the operating system; while with proprietary systems the OS is hidden by several layers."
Managing the input of thousands of HEP academics is a challenge which calls for more than mere version control, though. "There's a bit of sociology, too," says Calafiura, "With over 3,000 collaborators, there's quite a bit of formality to decide who is a member of ATLAS, and who gets to sign the Higgs paper. So there's a system of credits. Traditionally, this work is either in the detector or contributing to the software. If someone has a good algorithm for tracking a particle's path through the detector, they spend a couple of years perfecting that algorithm and contribute it to the common software repository, and it becomes part of the official instructions.
"Up until 2010, the majority of contributions to the software came from physicists, except for the core area where professionals work. Then everything changed the moment real data started flowing in. In the last couple of years, physicists are