PeakNet: An Autonomous Bragg Peak Finder with Deep Neural Networks
Serial crystallography at X-ray free electron laser (XFEL) and synchrotron
facilities has experienced tremendous progress in recent times enabling novel
scientific investigations into macromolecular structures and molecular
processes. However, these experiments generate a significant amount of data
posing computational challenges in data reduction and real-time feedback. Bragg
peak finding algorithm is used to identify useful images and also provide
real-time feedback about hit-rate and resolution. Shot-to-shot intensity
fluctuations and strong background scattering from buffer solution, injection
nozzle and other shielding materials make this a time-consuming optimization
problem. Here, we present PeakNet, an autonomous Bragg peak finder that
utilizes deep neural networks. The development of this system 1) eliminates the
need for manual algorithm parameter tuning, 2) reduces false-positive peaks by
adjusting to shot-to-shot variations in strong background scattering in
real-time, 3) eliminates the laborious task of manually creating bad pixel
masks and the need to store these masks per event since these can be
regenerated on demand. PeakNet also exhibits exceptional runtime efficiency,
processing a 1920-by-1920 pixel image around 90 ms on an NVIDIA 1080 Ti GPU,
with the potential for further enhancements through parallelized analysis or
GPU stream processing. PeakNet is well-suited for expert-level real-time serial
crystallography data analysis at high data rates.
Cong Wang, Po-Nan Li, Jana Thayer, Chun Hong Yoon
# PeakNet: Bragg peak finding in X-ray crystallography experiments with U-Net

###### Abstract

Serial crystallography at X-ray free electron laser (XFEL) sources has experienced tremendous progress in achieving high data rate in recent times. While this development offers potential to enable novel scientific investigations, such as imaging molecular events at logarithmic timescales, it also poses challenges in regards to real-time data analysis, which involves some degree of data reduction to only save those features or images pertaining to the science on disks. If data reduction is not effective, it could directly result in a substantial increase in facility budgetary requirements, or even hinder the utilization of ultra-high repetition imaging techniques making data analysis unwieldy. Furthermore, an additional challenge involves providing real-time feedback to users derived from real-time data analysis. In the context of serial crystallography, the initial and critical step in real-time data analysis is finding X-ray Bragg peaks from diffraction images. To tackle this challenge, we present PeakNet, a Bragg peak finder that utilizes neural networks and runs about four times faster than Foscake peak finder, while delivering significantly better indexing rates and comparable number of indexed events. We formulated the task of peak finding into a semantic segmentation problem, which is implemented as a classical U-Net architecture. A key advantage of PeakNet is its ability to scale linearly with respect to data volume, making it well-suited for real-time serial crystallography data analysis at high data rates.

## I Introduction

Serial femtosecond crystallography (SFX) with X-ray free electron lasers (XFEL) has enabled structural determination of macromolecules by outrunning radiation damage at room temperature, an approach commonly known as _diffraction-before-destruction_[1, 15, 14]. Time-resolved serial femtosecond crystallography (TR-SFX) at XFEL facilities, like the Linac Coherent Light Source (LCLS), allows the investigation of biochemical reactions with picosecond time resolution which has led to many significant structural studies [11, 13, 12, 14, 15, 16, 17, 18] since the first published work at LCLS [1]. In the meantime, the data rate at XFEL facilities have also gradually increased from hundreds of Hz to the MHz range, with the European X-ray Free Electron Laser (EuXFEL) being the first MHz facility available to users and LCLS-II set to launch in 2023.

At the MHz data rate, novel real-time data analysis approaches are needed to tackle two major challenges that arise in SFX experiments, namely data reduction and real time feedback. Firstly, data reduction aims to veto, extract and compress scientific data by identifying important events, commonly referred to as "hits", and vetoing irrelevant ones from terabyte per second data streams. One typical approach for SFX hit finding utilizes existing Bragg peak finders, such as the Cheetah peak finder [1], the robust peak finder [1], DIALS peak finder [15] or the Poscake peak finder [14]. Additionally, a neural network-based SFX hit finder was reported to achieved a processing rate of 1.3 kHz on a single GPU [17]. ORB feature extractor combined with a multilayer perceptron (MLP) hit classifier showed promising results for SFX data [10]. Secondly, real-time feedback in SFX experiments provides users with information on the experimental conditions and allows them to make real-time decisions that can optimize the data acquisition process. For example, it is critical to fine-tune the hit rate to maximize the usage of collected data; it is also important to know the current diffraction limit, crystalline mosaicity, reciprocal space coverage, and other relevant factors over time. Therefore, hit finding alone is insufficient to provide the necessary insights, making real-time Bragg peak finding crucial in delivering fast and comprehensive feedback. The previously mentioned peak finder in Poscake runs at roughly 5 Hz on a data center CPU, which may result in substantial cost when attempting to scale up the process.

In this work, we present PeakNet that addresses the problem of high data rate peak finding by converting it into the task of peak segmentation using neural networks. To accomplish this goal, we propose to train a neural network model to classify each individual pixel as either belonging to a peak or not belonging to a peak. This is followed by converting a connected area of pixels as a single peak, a task that can be readily accomplished through connected component
analysis [22]. Subsequently, the peak coordinates will be determined by computing the center of mass of all the pixels that belong to the identified peak. It is worth noting that the entire peak finding process takes place on GPUs, offering a fully parallelizable approach for achieving scalability. Concretely, our contributions can be summarized as follows:

* We introduce a deep neural network-based Bragg peak finder, named PeakNet, which can effectively and efficiently segment hundreds or even thousands of Bragg peaks with various peak profiles in detector images of any dimension. Despite being trained on peaks labeled by an existing peak finder, we highlight that PeakNet's capability is not limited by the peak finder used in labeling.
* We demonstrate the efficacy of PeakNet by evaluating its performance on large multi-panel (\(1739\times 1748\)) and single-panel (\(1920\times 1920\)) detectors. It significantly outperformed existing methods in terms of indexing rate, while delivering comparable number of indexed events. We also fine-tuned the underlying U-Net architecture of PeakNet to strike an optimal balance between model efficacy and runtime performance, leading to a four times out-of-the-box processing speed improvement over Posocake peak finder.

## II Related work

Reliable and automatic peak finding algorithms have undergone multiple iterations of development. An early approach involved utilizing template matching [23] based on libraries of peak shapes, but this method was found to be slow and inefficient during runtime, especially when dealing with peaks with low signal-to-noise ratios (SNR). The main obstacle lies in the localization of peaks and the tuning of user parameters for pattern matching, compounded by the difficult task of developing a comprehensive library of peak shapes. Another influential approach employed region-growing techniques [1, 1] to detect pixel areas with high skewness. In the context of serial crystallography with XFEL sources, the efficacy of this method was diminished due to the lack of sufficient data to provide reliable statistics, rendering it susceptible to outliers and less adept in analyzing weak peaks at higher scattering angles, which are essential for achieving atomic resolution in structure determination.

While template matching and region-growing techniques require users to provide prior information like threshold values, Robust Peak Finder (RPF) [10] has demonstrated proficient peak finding capabilities with minimal user input. This technique utilizes the Modified Selective Statistical Estimator (MSSE) method to segment between background pixels and actual peaks. Furthermore, the RPF method can also parallelize the processing of diffraction patterns, making it suitable for real-time data analysis.
of feature channels doubles with respect to its previous contracting layer. Likewise, a single expanding path contains a stride-two \(2\times 2\) transposed convolutional layers concatenated with the corresponding feature map from the contracting path. This is followed by two stride-one \(3\times 3\) convolutional layers connected to a ReLU activation function. It is worth noting that the feature map cropping may be necessary during the concatenation between feature maps from two paths. In the last layer, all 8 channels will be mapped to a single channel through a \(1\times 1\) convolution.

### _The loss function: focal loss_

A main technical issue in training U-Net for Bragg peak detection is the extreme peak-background class imbalance (e.g., \(>10^{3}:1\)), resulting in reduced prediction accuracy. This may not pose as a problem in previous peak finders like BraggNet, which operates on a much smaller area of \(32\times 32\), whereas PeakNet is trained on larger multi-panel detectors, such as the Cornell-SLAC pixel array detector (CSPAD) [1] with a dimension of \(194\times 185\) for one of its panels. Focal loss was introduced to address the extreme class imbalance problem for object detectors [10]. It is defined as:

\[\text{FL}(p)=-\alpha\cdot(1-p)^{\gamma}\cdot\log p. \tag{1}\]

Here, \(p\) is the probabilities of a pixel \(x\) being a peak. \(\gamma\) controls a modulating factor \((1-p)\) and \(\alpha\) is an additional modulating factor. We used \(\gamma=2.0\) and \(\alpha=1.2\) in PeakNet. In practice, numerical instability will often emerge in the calculation of \(p=e^{-\text{logit}}\), where \(\text{logit}=\log{(1+e^{-x})}\). To bypass this issue, logit needs to be rewritten as \(\text{logit}=a+\log{(e^{-a}+e^{-x-a})}\), where \(a=max(0,-x)\). As a result, if we denote the label of a pixel \(x\) as \(y\), the focal loss becomes

\[\text{FL}(p)=\alpha\cdot y\cdot(1-p)^{\gamma}+\text{logit}+(1-y)\cdot p^{ \gamma}\cdot x+p^{\gamma}\cdot(1-y)\cdot\text{logit} \tag{2}\]

## IV Experiments

### _PeakNet on multi-panel detectors: Trained on panels but predicting on assembled images._

We demonstrated the possibility of PeakNet making prediction on assembled images from a multi-panel detector despite being trained on individual panels. To facilitate this investigation, we utilized a multi-panel detector CSPAD, which is widely used in serial crystallography experiments at LCLS. Specifically, we used crystallographic data of streptavidin crystals from LCLS experiment _cxic0415_ run _101_ at the Coherent X-ray Imaging (CXI) instrument, where the CSPAD deployed consists of 32 panels. An individual diffraction image recorded on CSPAD are conventionally saved in a stack of 64 mini-panels (half of a physical CSPAD panel). For data labeling, we randomly selected 268 mini-panels, which were labeled using the Posocake peak finder with the default settings. We then split the labeled data into 60% training set and 20% validation set. The remaining 20% data were initially set aside for testing purposes, but were not used due to the lack of an effective metric, which will be discussed later in the evaluation of our model's efficacy. Our test set included the first 5000 events of the run, which facilitated a faster feedback loop compared with going through all events in the run. Notably, we did not apply any image preprocessing, and thus PeakNet

Fig. 1: Peak finding process in PeakNet.
directly takes in a diffraction image of its original size and outputs a segmentation map.

Fig. 3 (a) displays labeled samples, with the input mini-panel image on the left and pixel-wise labeled peaks on the right. The overall peak label is a binary mask that shares the same size as the input, with a value of one indicating a peak pixel and zero otherwise. Consequently, each input consists of a \(185\times 194\) mini-panel from the CSPAD and a binary mask of the same size. Additionally, Fig. 3 (b) shows a sample model prediction on an entire CSPAD image with a size of \(1739\times 1748\).

### _Examining PeakNet's generalizability beyound the peak finder used in labeling_

The current strategy for data labeling involves using an existing peak finder, and it is important to evaluate whether any limitations of the peak finder used in data labeling will affect the peak finding capability of PeakNet. One specific limitation to investigate is the size dependency of the peak finder. Notably, the peak size range is a user-defined input requested by the Poscake peak finder. This works well when diffraction spots have consistent sizes. However, detectors such as the Rayonix have a large point spread function for intense peaks, which may result in large peaks being ignored by the Poscake peak finder. Consequently, there is a concern about whether PeakNet will inherit the same limitations since it is trained on Poscake peak labels.

Fig. 4 presents a side-by-side comparison of peak detection on a Rayonix image, with the PeakNet results displayed on the left and the Poscake results on the right. The diffraction data were collected from crystals of SARS-CoV-2 proteases using the Macromolecular Femtosecond Crystallography (MFX) instrument [15] at LCLS. Upon visual inspection of Fig. 4 (c) and (d), both PeakNet and Poscake were observed to detect a considerable number of peaks, suggesting a potentially comparable performance between the two methods. However, upon closer inspection of the view near the area slightly below the beamstop position, one obvious, reasonably sized Bragg peak was observed but not detected by the Poscake peak finder, whereas PeakNet successfully located this peak. This finding provides direct evidence that PeakNet's peak finding capability is not strictly limited by the peak finder used in the data labeling process. In fact, PeakNet appears to possess its own understanding of Bragg peaks, as evidenced by the exclusion of a weak single Bragg peak-like spot near the upper edge of the image, as shown in Fig. 4 (a) and (b). Therefore, it is important to note that PeakNet is not merely a neural network-based replica of the Poscake peak finder despite being trained on labels supplied by it.

### _Model efficacy and runtime performance_

We consider model efficacy of a peak finder in two metrics: (1) indexing rate and (2) the number of indexed events. Table I provides a comparative analysis of two distinct datasets, CXI and MFX, showcasing the outcomes obtained when processed using two peak finders, Poscake and PeakNet. The detectors employed for the respective datasets are CSPAD and Rayonix. The analyzed events for the CXI dataset span from 0 to 5000, whereas the MFX dataset encompasses a range of 0 to 8998. Poscake identified 2107 events in the CXI dataset and 465 in the MFX dataset, yielding indexing rates of 71.2% and 84.5%, respectively. In contrast, PeakNet discovered 1991 events in the CXI dataset and 430 in the MFX dataset, exhibiting superior indexing rates of 78.6%

Fig. 2: The U-Net architecture in PeakNet.
and 90.2%, respectively. Although both peak finders exhibit varying levels of effectiveness, our evaluation has revealed that PeakNet achieves superior indexing rates across both datasets, while also yielding comparable numbers of indexed events. These results suggest that PeakNet may represent a more reliable and efficient solution for the analysis of such data.

To provide further context and justification for this efficacy measure, it is important to note that conventional metrics like accuracy, precision and F-1 scores are not always well-defined in the context of peak finding, due to the lack of actual "ground truth". This is because the peak labels used for training and evaluation are derived from another peak finder, rather than being uniquely determined in some fashion. As a result, it is necessary to consider alternative measures of model performance and reliability, such as the indexing rate and the number of indexed events. However, conventional metrics may remain a viable option when applied to synthetic data generated by a simulator. For clarification purposes, we used the _indexamajig_ tool within the _CrystFEL_ data analysis software [17] to perform indexing. We employed multiple indexing methods, including _mosflm_, _xds_ and _xgandalf_, with an additional setting of _-int-radius=4,5,6_.

Fig. 3: (a) PeakNet input images and peak labels. (b) PeakNet prediction on an entire CSPAD image. (c) A magnified view of a small region of the detector.
Furthermore, while the runtime performance of PeakNet is excellent as compared with other crystallography data analysis methods in Table II, there is a caveat to consider. Although PeakNet is designed as a nearly end-to-end solution, with its neural networks returning a segmentation map that represents the pixel-wise probability of being a peak from input diffraction images, we must employ connected component analysis to convert the segmentation maps into peak coordinates. While the neural network runtime is exceptionally fast, capable of processing 11 Rayonix images (\(1920\times 1920\)) in parallel on a 1080 Ti GPU in just 4ms (equivalent to 0.36 ms per image), the additional time required for connected component analysis is considerably longer and thus a significant runtime bottleneck in the current version of PeakNet. In fact, this bottleneck processing time scales nearly linearly with respect to the number of images, and it adds an additional 30ms of processing time for each image despite running on the same

Fig. 4: Peak finding on a Rayonix image using (c) PeakNet and (d) Psocake. (a), (b), (c) and (d) are magnified views of some regions.
GPU. Practically speaking, the current version of PeakNet runs at 49ms per Rayonix event on 1080 Ti, which is actually 10 \(\times\) faster than some diffraction image classification algorithms.

## V Conclusions

In this paper, we introduced PeakNet, a deep learning-based peak finder that efficiently tackles the Bragg peak finding problem as a semantic segmentation task. The U-Net architecture of PeakNet has been optimized to achieve an ideal balance between efficacy and runtime performance, resulting in a processing speed four times faster than the Posocake peak finder while maintaining superior indexing rates and a comparable number of indexed events. A notable bottleneck in the current version of PeakNet is the connected component analysis, which scales linearly with input data volume, whereas the neural network portion scales beyond linear capability on GPUs, limited largely by GPU memory. It is worth mentioning that the current data reduction concept primarily focuses on processing single events as quickly as possible, centering the design principle on CPUs. However, with neural networks, we propose a new direction towards batch image processing on GPUs, exemplified by PeakNet in the context of serial crystallography.

## Acknowledgment

Earlier versions of the U-Net models were built and tested by P.L under the supervision of C.H.Y. The experiments were designed by C.W with inputs from J.B.T and C.H.Y. C.W prepared the datasets, labeled experimental data, and trained and evaluated the refined neural network model. The manuscript was written by C.W and C.H.Y with input from all authors. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences under Award Number FWP-100643. Use of the Linac Coherent Light Source (LCLS), SLAC National Accelerator Laboratory, is supported by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences under Contract No. DE-AC02-76SF00515.
