% ---------------------------------------------------
% CHAPTER SUMMARY MESSAGE: < WRITE THIS BEFORE ANY EDITING>
% ---------------------------------------------------

%Contributors: Igor Lakomov (ALICE), Konstantin Kanishchev (AMS-02), Konstantin Skazytkin
%(ALICE), Jonas Eschle (LHCb), Alexander Kurepin (ALICE), Andrey Ustyuzhanin (Yandex,
%LHCb), Michela Paganini (ATLAS), Gabriel Perdue (MINERvA), Ryan Reece (ATLAS),
%Sean-Jiun Wang (CMS), Sergei Gleyzer (CMS), Meghan Kane (Industry - SoundCloud), Steven Schramm (ATLAS), Eduardo Rodrigues (LHCb), Martin Erdmann, Patrick Koppenburg (Nikhef, LHCb)

\subsection{Introduction}


Discovery science provides a challenge that attracts brilliant minds eager to push the boundaries of scientific understanding of nature. Particle physics has a rich problem domain that offers avenues for intellectual reward. The goal is to achieve a vibrant collaboration between the data science and high-energy physics communities by finding a common language and working together to further science.
%Before building collaborations, it is important to have a coherent strategy and vision of how to engage other communities.
Both communities can benefit from such collaboration. The HEP community can explore new research directions and applications of machine learning, novel algorithms, and direct collaboration on HEP challenges. The ML community can benefit from a diverse set particle physics problems with unique challenges in scale and complexity, and a large community of researchers that can expand the machine learning horizon by contributing to solving problems relevant to both communities. For example, the treatment of systematic uncertainties is an important topic for the HEP and ML communities. By working together on common challenges, the two fields can further progress in solving such problems.\\

There are a number of existing examples of collaboration between HEP and ML that have produced fruitful results through mostly local connections (e.g.~\cite{Likhomanenko:2016tgu,Paganini:2017hrr,darkmachines} ). The HEP community should continue such collaborations and look for additional collaborations with the ML community.\\

The HEP community needs to define its challenges in a language that the ML community can understand. This may involve only retaining necessary information with clear and concise explanations as to its relevance. Machine learning likewise has a significant amount of domain knowledge. Ideas and solutions provided by both communities should be presented in an understandable way for scientists without in-depth knowledge.

\subsection{Academic Outreach and Engagement}

Direct collaboration between HEP researchers and computer scientists working in the area of machine learning is an important possible driver of innovation in the area of machine learning applications described in Section~\ref{sec:applications}.
It is important for high-energy physics to engage and collaborate with the academic community focused on new machine learning algorithms and applications as they will naturally be interested in applying new ideas to interesting and complex data provided by HEP.\\

Conferences and workshops are a core aspect of the academic ML community, and organizing or contributing to key conferences is a means of gaining interest.
Organizing sessions or mini-workshops within major ML conferences, such as NIPS, would increase the familiarity of HEP within the ML community and jump-start future collaborations.
This has been explored in single cases~\cite{NIPS:2015:ALEPH} but is not an established, regular workshop series.
At the same time inviting ML experts to HEP workshops, as done at~\cite{FlavourDataMining} and the DS@HEP series~\cite{DSatHEP2015, DSatHEP2016, DSatHEP2017}, can foster greater long-term collaboration.
There should be coordinated efforts to:
\begin{itemize}
 \item Organize workshops and conferences open to external collaborators to discuss applications, algorithms and tools
 \item Organize thematic workshops around topics relevant to HEP
       %\item Engage machine learning experts by creating prestigious fellowships and data-science related initiatives
\end{itemize}

%\subsection{Science outreach}
HEP should reach out to other scientific communities with similar challenges, for example astrophysics/cosmology, medium energy nuclear physics and computational biology. This can lead to more active partnerships to better collaborate on ideas, techniques, and algorithms.

%[ A comment : I see that wrt the version I printed (June) the scope has been reduced to reflect what is actually described in Sec.6: collaboration between HEP and ML communities. I find that a bit sad as one could (should?) think of other fields of science that use ML a lot. Astrophysics is mentioned above, but social sciences and genomics face similar challenges. A quick search for ML in bioRxiv is enlightening. Could this section be expanded a bit?  ]

\subsection{Machine Learning Challenges}

To engage the wider ML community, challenges such as the Higgs Boson Challenge (2014) or the Flavor Physics Challenge~\cite{NIPS:2015:ALEPH,FlavourDataMining} have been organized on Kaggle.
These types of challenges draw considerable attention from the machine learning community and additional similar challenges should be organized in the future.\\

Organization of a challenge requires a well documented dataset, a starting-kit and an evaluation metric to rank the solutions.
This forces the organizers to simplify the problem as much as possible, while retaining its intrinsic complexity.
The drawback of challenges is that once they are launched, participants priority is winning the challenge and not eventual collaboration with HEP.
It is important to foresee upfront a way to integrate incoming solutions, for example via forums and post-challenge workshops (like~\cite{FlavourDataMining}) where a diversity of competitive algorithms can be presented.
The challenge dataset and evaluation metric should be released publicly so that further developments can continue.

\subsection{Collaborative Benchmark Datasets}\label{subsec:benchmark}
There is a strong incentive for HEP to develop public benchmark datasets, beyond just challenges. Access to a dataset makes the discussion much more concrete and productive. Within the HEP community, common datasets enable comparisons of algorithms with much better accuracy, which is very useful for research and development. The same benchmark datasets can also be used for teaching, tutorials and training.\\

These benchmark datasets could be built based on public simulation engines or released by experiments within the bounds of their data access policy. Even a small subset of an experiment's simulated data can be the base of a very valuable benchmark dataset. For example, the CMS experiment has released a significant amount of its simulated and collected data via the CERN Open Data Portal~\cite{opendata}.\\

To be maximally useful, the subsequent guidelines should be followed when designing a benchmark dataset:
\begin{itemize}
 \item Simplify the dataset as much as possible while conserving all methodical difficulties relevant to solving the problem
 \item Document the dataset to make it understandable by a non-HEP expert
 \item Create methodologies and metrics for the evaluation of proposed solutions, and document them
 \item Prepare an integration plan for incoming ideas and solutions
 \item Feedback results of successful applications
\end{itemize}

%\subsection{Collaborative Benchmark Datasets}

%As discussed in Section~\ref{subsec:benchmark}, collaborative benchmark datasets can be useful for developing ML challenges and for collaboration with the ML community.

The HEP community should organize and curate a variety of such benchmark datasets covering its current physics drivers and make them publicly available. To improve the reproducibility of results and algorithm comparisons, some of the data used for evaluation of the solutions should be kept private.
%an effort known as TrackingML is underway to create datasets and establish a challenge to elicit solutions from the broader ML community, Move to bridges.
Additionally, after investing heavily into producing highly-detailed and realistic simulations, the HEP community can provide the machine learning community with labeled datasets with high statistical power to test algorithms and develop novel ideas.

%\subsection{Communication and Outreach}
%A multi-prong strategy for engaging the ML community is presented, targeting different areas: academia, industry and the ``general public''.

%\subsection{Inter-experimental Collaboration}
%to be added

\subsection{Industry Engagement}
Industry has been focused on the development and adoption of machine learning techniques. In addition to algorithm and software development, one of the promising areas is the adoption of dedicated specialized hardware and high performance co-processors. GPUs, FPGAs, and high core count co-processors all have the potential to dramatically increase performance of machine learning applications relevant to the HEP community.
One of the challenges is gaining the human expertise for development and implementation. Industry interactions bring specific technology opportunities and access to specialized expertise that can be difficult to hire and support internally.\\

There are specific areas of development where industry has expressed interest in collaborating with HEP.  Automated resource provisioning, data placement, and scheduling are similar to industrial applications to improve efficiency. Applications such as data quality monitoring, detector health monitoring and preventative maintenance can be automated using techniques developed for other industrial quality control applications. There are two more forward looking areas that coincide with HEP physics drivers, namely computer vision techniques for object identification and real-time event classification. These present a challenge to industry due to its complexity and benefit outside of HEP.\\

%		\item Bridge to industry (partnerships in applying externally developed techniques and tools to HEP problems anomaly detection against fraud transactions, spam filtering etc)

%\subsubsection{CERN OpenLab and research-industry collaborative initiatives}
CERN OpenLab is a public-private partnership that accelerates the development of cutting-edge solutions for the LHC community and wider scientific research. CERN OpenLab has established the infrastructure to maintain non-disclosure agreements, to arrange ownership of intellectual property, and to provide an interface between CERN and industry. As part of its upcoming phases, OpenLab plans to explore machine learning applications for the benefit of LHC experiments computing and the HL-LHC. Such initiatives and industry partnerships should be supported in the future.

%Through CERN openlab, CERN collaborates with leading ICT companies and research institutes. Within the CERN openlab framework, CERN provides access to its complex ICT infrastructure and its engineering experience — in some cases even extended to collaborating institutes worldwide. Testing in CERN's demanding environment provides the collaborating companies with valuable feedback on their products, while enabling CERN to assess the merits of new technologies in their early stages of development for possible future use.

%CERN openlab is continuing its work to support CERN’s research community, with a particular focus on the upgrades to the LHC and the detectors that will be carried out during LS2 and LS3. With the data rates from the experiments set to increase significantly, efforts have been focused on supporting the work to overhaul and modernise their data-acquisition systems, while also ensuring that the maximum benefits are gained from the hardware available to CERN’s teams by making sure the software running on it has been fully optimised. Efforts have now begun to identify the ICT challenges that will be tackled in CERN openlab’s sixth phase (2018 to 2020).

%One of the most significant technical challenges facing CERN is how to provision computing for the LHC experiments and the accelerator for the HL-LHC.
%In order to tackle said challenge, CERN openlab Phase VI will choose projects that are intended to help close the gap and deliver the evolutionary and revolutionary changes needed for the success of the program. The exploration of new applications of Machine Learning techniques aligns with the goals of industry and the next phase of CERN openlab.

%The work areas are divided into three sections that will be expanded below: research and development (R\&D) for data centre technologies and infrastructure, R\&D for software and computing performance, and R\&D for machine learning and data analytics. In particular, for its Phase VI, CERN openlab is in the process of investigating the opportunity of establishing a number of key projects in machine learning for data taking, triggering, simulation, reconstruction, detector and data quality monitoring with leading companies in deep learning technologies.

\subsection{Machine Learning Community-at-large Outreach}
Another form of engagement is using the communications mediums to broadcast our challenges and attract interested collaborators. There are a variety of channels which can be leveraged to increase the visibility of our problems and research opportunities in the ML community. These can be popular forums such as reddit, personal or official blogs, social media, or direct contact with influential personalities.\\

%\subsubsection{Podcasts, Blog Posts and Meetups}
Podcasts have shown to be a great vehicle for reaching a large audience. Listeners are keen to consume material that is outside of their immediate problem domain in a way that is easy to digest. There is an abundance of machine learning podcasts with a large base of listeners that can be targeted for outreach:
\begin{itemize}
 \item \href{http://lineardigressions.com/}{\textcolor{blue}{Linear Digressions}} (co-hosted by former ATLAS Ph.D. Katie Malone)
 \item \href{http://partiallyderivative.com/}{\textcolor{blue}{Partially Derivative}}
 \item \href{http://www.thetalkingmachines.com/}{\textcolor{blue}{Talking Machines}}
 \item \href{https://dataskeptic.com/}{\textcolor{blue}{Data Skeptic}}
 \item \href{http://becomingadatascientist.com/}{\textcolor{blue}{Becoming a Data Scientist Podcast}}
 \item \href{https://itunes.apple.com/us/podcast/not-so-standard-deviations/id1040614570?mt=2}{\textcolor{blue}{Not So Standard Deviations}}
 \item \href{https://twimlai.com/}{\textcolor{blue}{This Week in ML \& AI}}
\end{itemize}

%Some examples are (Partially Derivative Episode: \href{http://partiallyderivative.com/podcast/2017/02/28/michael-kagan}{``Particle Physics and Machine Learning at CERN with Michael Kagan''}) as well as applications of programming and technology in physics (Talk Python To Me Episode \#29: ``\href{https://talkpython.fm/episodes/show/29/python-at-the-large-hadron-collider-and-cern}{Python at the Large Hadron Collider and CERN}'' with Kyle Cranmer).


%Bringing together tangentially related fields is commonplace in ML podcasts. For example, podcasts such as This Week in Machine Learning (TWiML) features machine learning experts across a wide variety of problem domains, spanning topics from a  \href{https://twimlai.com/twiml-talk-12-brendan-frey-reprogramming-human-genome-w-ai/}{\textcolor{blue}{genomics}} to \href{https://twimlai.com/twiml-talk-027-intelligent-autonomous-robots-ilia-baranov/}{\textcolor{blue}{robotics}} to \href{https://twimlai.com/twiml-talk-5-joshua-bloom-machine-learning-stars-productizing-ai/}{\textcolor{blue}{astrophysics}}.Podcasts typically consist of a casual conversation from 30 minutes to an hour. Therefore, there is a high yield in the form of listeners versus time investment required to release a podcast.
%\subsubsection{Blog Posts}

Another form of engagement is through outreach-style blog posts to explain HEP challenges in a way that is easy to understand by the public.\\

Another outreach opportunity is to make HEP related presentations at machine learning Meetups across the world to generate awareness, engage community, foster cross pollination of ideas between HEP and industry. Some popular ML meetups are:
\begin{itemize}
 \item NYC: \url{https://www.meetup.com/NYC-Machine-Learning/}
 \item Berlin: \url{https://www.meetup.com/Advanced-Machine-Learning-Study-Group/}
 \item SF: \url{https://www.meetup.com/SF-Bayarea-Machine-Learning/}
\end{itemize}

In conclusion, existing outreach efforts should be expanded to attract greater collaboration between the HEP and ML communities. By understanding and speaking the same language, the two communities can better collaborate and find solutions to present and future challenges.

%Section outline:
%\begin{itemize}
%	\item Describe the need for a bridge and its benefits
%	\item Explain community differences
%	\item ML rapid experimentation
%	\item ML and HEP Domain knowledge differs (need to be able to communicate)
%	\item Roadmap to creating a bridge
%	\item Define the problem
%	\item Discuss a common approach within HEP for reaching out to ML
%	\item Reach out to ML
%	\item Become more involved members of the ML community
%	\item Support new collaborations between HEP-ML
%	\item Make benefits of collaborations accessible to the community
%\end{itemize}
