Neural networks trained on synthetically generated crystals can extract
  structural information from ICSD powder X-ray diffractograms
Machine learning techniques have successfully been used to extract structural
information such as the crystal space group from powder X-ray diffractograms.
However, training directly on simulated diffractograms from databases such as
the ICSD is challenging due to its limited size, class-inhomogeneity, and bias
toward certain structure types. We propose an alternative approach of
generating synthetic crystals with random coordinates by using the symmetry
operations of each space group. Based on this approach, we demonstrate online
training of deep ResNet-like models on up to a few million unique on-the-fly
generated synthetic diffractograms per hour. For our chosen task of space group
classification, we achieved a test accuracy of 79.9% on unseen ICSD structure
types from most space groups. This surpasses the 56.1% accuracy of the current
state-of-the-art approach of training on ICSD crystals directly. Our results
demonstrate that synthetically generated crystals can be used to extract
structural information from ICSD powder diffractograms, which makes it possible
to apply very large state-of-the-art machine learning models in the area of
powder X-ray diffraction. We further show first steps toward applying our
methodology to experimental data, where automated XRD data analysis is crucial,
especially in high-throughput settings. While we focused on the prediction of
the space group, our approach has the potential to be extended to related tasks
in the future.
Henrik Schopmans, Patrick Reiser, Pascal Friederich
Neural networks trained on synthetically generated crystals can extract structural information from ICSD powder X-ray diffractograms

###### Abstract

Machine learning techniques have successfully been used to extract structural information such as the crystal space group from powder X-ray diffractograms. However, training directly on simulated diffractograms from databases such as the ICSD is challenging due to its limited size, class-inhomogeneity, and bias toward certain structure types. We propose an alternative approach of generating synthetic crystals with random coordinates by using the symmetry operations of each space group. Based on this approach, we demonstrate online training of deep ResNet-like models on up to a few million unique on-the-fly generated synthetic diffractograms per hour. For our chosen task of space group classification, we achieved a test accuracy of 79.9% on unseen ICSD structure types from most space groups. This surpasses the 56.1% accuracy of the current state-of-the-art approach of training on ICSD crystals directly. Our results demonstrate that synthetically generated crystals can be used to extract structural information from ICSD powder diffractograms, which makes it possible to apply very large state-of-the-art machine learning models in the area of powder X-ray diffraction. We further show first steps toward applying our methodology to experimental data, where automated XRD data analysis is crucial, especially in high-throughput settings. While we focused on the prediction of the space group, our approach has the potential to be extended to related tasks in the future.

## 1 Introduction

Machine learning techniques have emerged as a powerful tool in the toolkit of materials scientists. While they are often used to make predictions on the properties of materials or find new materials with certain properties, an increasingly interesting domain is the automated analysis of raw experimental measurements guided by machine learning [1].

With the advent of high-throughput experiments, the amount of gathered data is vast and the analysis often becomes a bottleneck in the processing pipeline [2]. Powder X-ray diffraction (XRD) is an important measurement technique used to obtain structural information from polycrystalline samples [3]. The diffractograms are an information-dense fingerprint of the structure of the material. However, analyzing these diffractograms is not an easy task [4]. Full structure solutions and Rietveld refinement take time and require expert knowledge, both about the analysis technique and the materials class at hand. This is not feasible in high-throughput experiments on a larger scale. Therefore, the question arises whether it is possible to automatically analyze powder diffractograms with machine learning models trained on large amounts of data, making it possible to run inference almost instantaneously.

During the last few years, there have been several studies tackling this objective by applying machine learning models to various tasks concerning the analysis of powder diffractograms such as phase classification [5, 6, 7, 8, 9], phase fraction determination [10], space group classification [11, 12, 13, 14, 15, 16], machine-learning-guided Rietveld refinement [17, 18], extraction of lattice parameters [19, 20, 21, 16] and crystallite
sizes [16, 19], and also novelty detection based on unsupervised techniques [22]. Since an abundant source of experimental diffractograms is hard to come by, most applications train their models on simulated diffractograms from the Inorganic Crystal Structure Database (ICSD) [23], which contains a total of 272 260 structures (October 2022).

Lee _et al._ used a deep convolutional neural network (CNN) trained on a large dataset of multiphase compositions from the quaternary Sr-Li-Al-O pool to classify present phases in the diffractogram [5]. In a follow-up study, they further showed good results for phase fraction inference in the quaternary Li-La-Zr-O pool [10]. Schuetzke _et al._ performed phase classification on iron ores and cement compounds and used data augmentation with respect to lattice parameters, crystallite sizes, and preferred orientation [7]. They showed that especially the lattice parameter variations enhance the classification accuracy significantly.

Instead of the analysis of phase composition, Dong _et al._ performed regression of scale factors, lattice parameters, and crystallite sizes in a five-phase catalytic materials system [19]. In contrast to supervised tasks, Banko _et al._ used a variational autoencoder to visualize variations in space group, preferred orientation, crystallite size, and peak shifts [22]. Park _et al._ used a deep CNN to classify space groups of single-phase diffractograms, reaching a test accuracy of 81.14% on simulated diffractograms. However, as we will show later in this paper, this accuracy is highly overestimated and drops to 56.1% when test splits are designed in a way to reduce data leakage in non-IID datasets such as the ICSD. Vecsei _et al._ developed a similar approach and applied their classifier to experimental diffractograms from the RRUFF mineral database [24], reaching an experimental test accuracy of 54%.

While the ICSD contains a large number of structures spanning many different classes of materials, it still falls short in size, distribution, and generality compared to the datasets used to train very large state-of-the-art models of other fields such as computer vision. Furthermore, the ICSD database is highly imbalanced with respect to space groups, as can be seen in the histogram in Figure 1. This makes the classification of space groups more difficult, as shown and discussed by Zaloga _et al._[13]. The ICSD also contains a limited number of different structure types that may not adequately represent the crystal structures analyzed in future experiments.

To overcome these shortcomings, we propose to train machine learning models on diffractograms simulated from synthetic crystal structures randomly generated based on the symmetry operations of the space groups. This makes it possible to train on structures with new structure types not present in the ICSD. We used the crystals from the ICSD only to determine vague statistics guiding the random generation and for calculating the test accuracy. Our approach goes one step further than classical data augmentation by fully detaching itself from the individual entries in the ICSD database. It is our hypothesis that when trying to solve more general tasks concerning the extraction of structural details from powder diffractograms, the crystals from which the training dataset is simulated do not need to be stable or chemically viable. We propose viewing the problem simply as a mathematical task of getting back some of the real-space information leading to given powder X-ray diffractograms.

Here, we applied this approach to the classification of the crystal symmetry, namely the space group. The space group is usually one of the first structural pieces of information needed after synthesizing a new material. This task is well-suited to showcase the strengths of using a synthetic dataset and to benchmark it. We further show the results of using our methodology to infer space group labels of an experimental dataset.

We embedded our synthetic generation algorithm in a framework with distributed computing capabilities to generate and simulate diffractograms on multiple nodes in parallel using the _Python_ library _Ray_[25]. In contrast to the traditional approach of generating a simulated dataset before training, we used this distributed computing architecture to build an infinite stream of randomly generated and simulated diffractograms to perform batch-wise online learning. This increases the generalization performance, eliminates the problem of overfitting, and allows very large models to be trained.

## 2 Methods

### Generating synthetic crystals

To generate synthetic crystals, we randomly place atoms on the Wyckoff positions of a given space group and then apply the respective symmetry operations. The algorithm is explained in the
following (see also Figure 1(a) for a flow diagram of the algorithm). We only explain the most important steps, details can be found in Section S1 of the SI.

1. Random selection of a space group. We follow the space group distribution of the ICSD to allow comparison with previous work.
2. Sample unique elements of the crystal and their number of repetitions in the asymmetric unit.
3. Place atoms onto the Wyckoff positions and draw uniform coordinates for each.
4. Draw lattice parameters from a KDE based on the ICSD.
5. Apply space group symmetry operations.

Parts of this algorithm were inspired by the generation algorithm of the _Python_ library _PyX-tal_[26]. We only keep generated crystals for training if the conventional unit cell volume is below \(7000\,\mathrm{\SIUnitSymbolAngstrom}^{3}\) and if there are less than 100 atoms in the asymmetric unit. We did not employ any form of distance checks on the coordinates, as we found this to have no meaningful impact on space group classification accuracy. We only prevented the algorithm from placing more than one atom onto a Wyckoff position that does not have a degree of freedom. We also did not use partial occupancies. We chose this algorithm for its simplicity and its capability to reproduce many important characteristics of ICSD crystals adequately (see Section 3.1).

For some space groups, there are not enough crystals in the ICSD to form a representative KDE for the volume or to calculate suitable occupation probabilities for individual Wyckoff positions. Therefore, we chose to only perform the classification on space groups with 50 or more crystals available in the statistics dataset we used to extract the probabilities (see Section 2.4), resulting in the exclusion of 85 space groups (see Figure 1). A classifier trained directly on ICSD data of all space groups will likely not be able to properly identify these space groups containing very few samples.

If a similar performance for all space groups is desired, a uniform distribution of space groups in the training dataset is needed. This is trivially possible with our synthetic approach, in contrast to training directly on the ICSD, where weighting, over-, or undersampling methods are needed [27]. To allow a direct and fair comparison between our approach and the original approach of training directly on ICSD entries, we still followed the same distribution of space groups of the ICSD in our synthetic training dataset. This eliminates the problem that the effective number of total space groups is smaller when training on a highly imbalanced dataset, making it easier to reach high accuracies.

Figure 1: Distribution (logarithmic scale in blue, linear scale in red) of space groups in the ICSD. Space groups are sorted by count. The population of the space groups varies by multiple orders of magnitude, showing that the ICSD is a highly imbalanced dataset regarding space groups. The space groups excluded due to insufficient statistics are visualized with black stripes. Since the exclusion is based on the statistics dataset and this histogram shows the full ICSD, the excluded space groups are not exactly the first 85 counted from the left.
### Simulating diffractograms

To simulate powder X-ray diffractograms based on the generated crystals, we used the implementation found in the _Python_ library _Pymatgen_[28]. We optimized the simulation code using the LLVM just-in-time compiler _Numba_[29]. This increases the performance of the main loop over the reciprocal lattice vectors of the crystal significantly and makes the continuous simulation while training (discussed in the next section) possible.

We used the wavelength \(1.5406\,\mathrm{\SIUnitSymbolAngstrom}\) (Cu \(\mathrm{K}_{\alpha}\) line) to simulate all diffractograms. The obtained peaks were further broadened with a Gaussian peak profile to form the full diffractogram. To obtain the peak widths, we used the Scherrer equation[30]

\[\beta=\frac{K\lambda}{L\cos\theta}, \tag{1}\]

where \(\beta\) is the line broadening at half maximum intensity (on the \(2\theta\)-scale), \(K\) is a shape factor, \(\lambda\) is the wavelength, and \(L\) is the (average) thickness of crystallites. We drew crystallite sizes from the range \([20,100]\) nm and used \(K=0.9\).

Diffractograms were generated in the range \(2\theta\in[5,90]\,\mathrm{\SIUnitSymbolAngstrom}\) with step size \(0.01\,\mathrm{\SIUnitSymbolAngstrom}\). After generating each diffractogram, it was rescaled to fit in the intensity range \([0,1]\).

### Continuous generation of training data

Typically, machine learning models are trained with a fixed dataset predefined at the beginning of training. Sometimes, data augmentation is applied to further increase the effective size of this dataset. In contrast to that, we generated our dataset on-the-fly, parallel to model training. The main advantage of using this approach compared to a fixed-size dataset is the eliminated possibility to overfit to individual diffractograms since every diffractogram is only used once. Furthermore, not having to pre-simulate a dataset before training makes this approach more flexible when changing simulation parameters.

We used a distributed architecture on multiple nodes using the _Python_ framework _Ray_[25], which enabled the training on 1-2 GPUs and simultaneous generation of training data on more than 200 CPU cores (see Figure 2b and SI Section S2.2). Depending on the model size and corresponding training speed, this setup allows training with up to millions of unique diffractograms per hour.

### Dataset split

The ICSD database contains many structures that are very similar with slightly different lattice parameters and coordinates. For example, there are 25 entries for NaCl (October 2022). Furthermore, there are 3898 entries that have the same structure type as NaCl and thus also similar powder diffractograms. If some of them appear in the training dataset and some in the test dataset,

Figure 2: a) Flowchart of how the generation algorithm produces synthetic crystals. Atoms are independently placed on the Wyckoff positions and random coordinates are drawn. b) Overview of the distributed computing system implemented using the _Python_ library _Ray_[25]. Two compute nodes (that generate and simulate diffractograms) are connected to the _Ray_ head node using a _Ray queue_ object.
the classification will be simplified to recognizing the structure type or structure. In that case, the test set accuracy will not represent the true generalization performance of the neural network. To quantify the true generalization performance, we split the dataset in such a way that the same structure type appears either only in the training or in the test dataset. We used the structure type definitions provided by the ICSD. The obtained accuracy on the test dataset reflects the accuracy of our network when being used on a novel sample with a structure type not yet present in the ICSD database.

We want to emphasize that the used test split is very important for the task of space group classification and not a trivial choice. The ICSD contains many subtypes of structure types (for example, subtypes of perovskites), which we regarded as separate structure types in our split. Considering the subtypes as the same structure type may also be a viable option when performing the split. A combination of a split based on structure type and sum formula or similar approaches are also possible. However, we think that our choice of splitting the dataset is able to effectively measure the generalization performance while still being relatively simple.

We divided the ICSD (database version 2021, June 15) in a 70:30 split. For our synthetic crystal approach, the 70% part (called statistics or training dataset) was only used to create the KDEs and to calculate the Wyckoff occupation probabilities needed for the generation algorithm. Since we can judge the performance of the synthetic generation algorithm by comparing the training accuracy (on synthetic crystals) with the accuracy tested on diffractograms simulated directly from the statistics dataset, an additional validation dataset was not needed. For comparison with the original approach by Park _et al._[11] of directly training on ICSD crystals, we simulated crystals directly from the statistics dataset and trained on them.

Analogous to the synthetic generation, we only used crystals with a conventional unit cell volume below \(7000\,\mathrm{\SIUnitSymbolAngstrom}^{3}\) and with less than 100 atoms in the asymmetric unit for the statistics and test dataset. This covers \(\approx 94\%\) of the ICSD crystals.

### Models and experiments

#### Models

We will briefly introduce the models we used for the classification of space groups. A more detailed description can be found in the SI Section S2.1.

As a baseline, we first used the CNN models introduced by Park _et al._[11]. They used three models, one for the classification of crystal systems ("parkCNN small"), one for extinction groups ("parkCNN medium"), and one for space groups ("parkCNN big"). All models have three convolution layers with two hidden fully connected layers and one output layer. The three models differ in the number of neurons in the hidden fully connected layers, increasing the number of model parameters with the number of target labels. Here, we only used the models "parkCNN medium" and "parkCNN big" and applied both to the classification of space groups. When using ICSD crystals to train the "parkCNN" models, dropout was used, while the training of the "parkCNN" models on synthetic crystals did not use dropout.

Since the approach of using an infinite stream of generated training data eliminates the problem of overfitting, we further used deeper models with a higher number of model parameters. For this, we used the deep convolutional neural networks ResNet-10, ResNet-50, and ResNet-101, which were introduced by He _et al._[31] in 2015.

Details of the machine learning setup can be found in the SI Section 2.2. Overall, our setup allowed us the training of models over up to 2000 epochs with more than \(100\,000\) unique, newly generated crystals and corresponding diffractograms in each epoch.

#### Experiments

We performed two sets of experiments to evaluate our new dataset split as well as our synthetic crystal generation approach and compare it to state-of-the-art models in literature: Firstly, we trained and tested models on ICSD crystals only, and secondly, we trained on synthetic crystals and tested on ICSD crystals.

In particular, we first performed an experiment with the "parkCNN medium" model trained directly on the diffractograms simulated from the ICSD statistics dataset (similar to Park _et al._[11]) with a fully random train-test split, instead of splitting by the structure type of the crystals. This experiment makes a comparison of the two different methods of train-test split possible. We then
trained the "parkCNN big" model using the structure type-based split, again directly on ICSD diffractograms. We further repeated the same experiment using the smaller model "parkCNN medium" to resolve potential overfitting to the ICSD diffractograms.

For the experiments performed on our continuously generated dataset based on synthetic crystals, we used the structure type- based split. As discussed in Section 2.4, the training / statistics dataset was only used to extract more general statistics, such as the element distribution. First, we trained the "parkCNN big" model. For each batch, we generated 435 random crystals and simulated two diffractograms with different crystallite sizes for each of them. This resulted in the batch size of 870. Since our synthetic crystal generation algorithm yields an infinite stream of unique diffractograms to train on, using much larger models than for the fixed ICSD dataset is possible without overfitting. We performed experiments for the ResNet-10, ResNet-50, and ResNet-101 models. Instead of generating two diffractograms with different uniformly sampled crystallite sizes for each generated crystal (as we did for the "parkCNN big" model), we now created only one diffractogram for each of the 145 crystals generated for one batch. This is due to the slower training of the ResNet models, which means that reusing the same diffractogram with different crystallite sizes is not necessary to generate training data fast enough.

To obtain the highest-possible ICSD test accuracy, we further applied the square root function as a preprocessing step to the input diffractograms of the network when using the ResNet models. This was suggested by Zaloga _et al._[13] and in their case improved classification accuracy by approximately 2 percentage points. Some initial tests suggested that this approach also yields a higher accuracy in our case, so we used this preprocessing step to train the ResNet models.

While we focused mainly on the methodology of using synthetic crystals to extract structural information from powder diffractograms, we also show some initial steps toward applying our methods to experimental data. We used the publicly available RRUFF mineral database [24] which provides experimental measurements, including powder diffractograms. In order to imitate experimental diffractograms, we added Gaussian additive and multiplicative noise and a background function based on samples from a Gaussian process to our simulated diffractograms. Furthermore, we added a small amount of an impurity phase to each diffractogram. Details about the experimental data generation protocol can be found in the SI Section S3. Using the ResNet-50 model, we performed two experiments for experimental data, one with the mentioned impurity phase, and one without.

## 3 Results and discussion

### Synthetic distribution

We first present an analysis of the generated synthetic crystals. Figure 3 shows some randomly selected ICSD and synthetic crystal structures side-by-side. Visually, the crystals appear very similar. However, no physical or chemical considerations regarding stability, clashing atoms, and element combinations are taken into account in the generation of synthetic crystals. As discussed earlier, our goal is to demonstrate that this is not problematic when using these crystals for the extraction of structural information from powder diffractograms. On the contrary, we expect the synthetic crystals to be a better basis for generalization to fundamentally new crystal structures than existing finite databases.

To compare the distribution of ICSD crystals with the synthetic distribution, we evaluated structural descriptors, i.e. density factors, crystallite sizes, unit cell volumes, and numbers of atoms in the asymmetric unit, and compare their histograms in Figure 4. One can see that the overall distributions of the synthetic and ICSD crystals are very similar for all four descriptors. This shows that our chosen generation algorithm reproduces crystals that are similar to ICSD crystals in terms of these more general descriptors.

### Classification results

The main results of our experiments (see Section 2.5) to classify the space group of powder diffractograms can be found in Table 1. The goal of our experiments is to systematically analyse and quantify the changes in classification accuracy introduced by our two main contributions: A more challenging dataset split, and training on continuously generated synthetic data.
We started by repeating previously reported experiments trained directly using ICSD crystals with a random train-test split instead of the split based on structure types (similar to the original approach by Park _et al._[11]). This model achieved a very high test accuracy of 83.2%. We note that Park _et al._ removed data from the training dataset, "[...] heavily duplicated data [...]"[11], but did not specify the exact criterions used. In contrast, we did not exclude any duplicates in this experiment based on a random train-test split. Furthermore, as discussed in Section 2.1, we excluded crystals with a very high unit cell volume and a very high number of atoms in the asymmetric unit. This is likely the reason for the slightly higher classification accuracy that we observed, compared to the originally reported 81.1%.

When splitting randomly, the model merely needs to recognize structures or structure types and assign the correct space group. This task is much easier than actually extracting the space group using more general patterns. When going from random splits to structure type-based splits (see Section 2.4), it becomes obvious that both the "parkCNN big" as well as the "parkCNN medium" models overfit the training data and do not generalize well to unseen structure types in the test set.

Training the same model, in particular the "parkCNN big" model, on synthetic crystals leads to a 1.6 percentage points higher test accuracy than the "parkCNN big" model trained on ICSD diffractograms. At the same time, the training accuracy drops from 87.2% to 74.2% indicating that the model is now limited more by missing capacity rather than by overfitting, which is why we explored larger models, which will be discussed later. The gap between training and test accuracy is 31.1 percentage points when training on ICSD data, while for training using synthetic crystals, the gap is only 16.5 percentage points. We note that this gap between training using synthetic crystals and testing using ICSD crystals cannot stem from overfitting, since no diffractograms are repeated for the synthetic training. The difference rather stems from the differences between the synthetic distribution and the ICSD distribution of crystals.

While the "parkCNN big" model trained on synthetic crystals outperforms the approach of training directly on ICSD crystals by only 1.6 percentage points, the advantage of training on an infinite stream of synthetic data increases when using models with more parameters and thus higher capacity. In contrast to training directly on a finite set of ICSD crystals, it is possible to train very large models using the infinite synthetic data stream without the potential of overfitting. As found in the last lines of Table 1, ResNet-10, ResNet-50, and ResNet-101 based models achieve

Figure 3: a) Some randomly picked examples of ICSD crystals. b) Some randomly picked examples of synthetically generated crystals. While coordination and distances are not chemically correct for the synthetic crystals, crystal symmetries are reproduced correctly.
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{Split} & Training & Testing & \multirow{2}{*}{Model} & Number of & Training acc. & Test acc. \\  & dataset & dataset & & parameters & / \% & / \% \\ \hline Random & ICSD & ICSD & parkCNN medium & 4 246 797 & 88.4 & 83.2 \\ \hline Structure & \multirow{2}{*}{ICSD} & \multirow{2}{*}{ICSD} & parkCNN big & 4 959 585 & 87.2 & 56.1 \\ type & & & parkCNN medium & 4 246 797 & 90.9 & 55.9 \\ \hline \multirow{3}{*}{Structure  type1} & \multirow{3}{*}{synthetic} & \multirow{3}{*}{ICSD} & parkCNN big & 4 959 585 & 74.2 & 57.7 \\  & & & ResNet-10 & 9 395 025 & 87.2 & 73.4 \\ \cline{1-1}  & & & ResNet-50 & 41 362 385 & 91.8 & 79.3 \\ \cline{1-1}  & & & ResNet-101 & 60 354 513 & 92.2 & **79.9** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Results of training on diffractograms simulated from ICSD crystals (random splits as well as structure type-based splits) compared to when training on diffractograms from synthetic crystals. Test accuracy in all cases refers to the accuracy when testing on the ICSD test dataset. The training accuracies are averaged over the last 10 epochs of the respective run. Experiments trained directly on ICSD data overfitted to the training data. Training longer would have further increased the training accuracy, while not increasing the test accuracy.

Figure 4: Histograms comparing the distributions of descriptors of the synthetically generated crystals with the ICSD distribution in the test dataset. a) density factor \(\frac{V_{\text{init.cell}}}{\sum_{4}4/3\pi\left(\frac{V_{\text{init.cell}}}{2} \right)^{3}}=\frac{V_{\text{init.cell}}}{V_{\text{atomic}}}\), b) crystallite sizes, c) unit cell volume (conventional cell settings), d) number of atoms in the asymmetric unit. The probability density of the ICSD is visualized by a stacked bar histogram, where the green portion of the bar was correctly classified and the red portion was incorrectly classified. The probability density of the synthetic crystals is visualized by the dark blue line. The portion between the dark blue line and the light blue line was correctly classified, the portion below the light blue line was incorrectly classified. The reported classification performance is based on the ResNet-101 model trained on diffractograms from synthetic crystals.
ICSD test accuracies of 73.4%, 79.3%, and 79.9%. This is a significant increase from the 57.7% achieved by the "parkCNN big" model. Figure S4 in the SI further shows the top-\(k\) accuracy over \(k\) for the ResNet-101 model. With increasing \(k\) the accuracy exceeds 95% at \(k=5\). This means that our model can not only determine the correct space group with a high probability but can also generate an almost complete list of possible space group candidates.

Figure 5 shows the ICSD test accuracy, the training accuracy (on synthetic data), and the ICSD top-5 test accuracy for all three ResNet variants as a function of epochs trained. For all three metrics, the difference between ResNet-50 and ResNet-101 is comparably small, while the step from ResNet-10 to ResNet-50 is substantial (5.9 percentage points in ICSD test accuracy, see Table 1). This shows that going beyond the model size of the ResNet-101 will likely not yield a big improvement in accuracy. In contrast to the 79.9% accuracy reached in the top-1 ICSD test accuracy, the top-5 ICSD test accuracy of the ResNet-101 model reaches 96%. However, for all three ResNet variants, a gap between training using synthetic crystals and testing using the ICSD remains (12.3 percentage points for ResNet-101). As also shown in Figure S5 in the SI, the accuracy convergence can be approximately described by a power law, indicating that exponentially more training time will substantially reduce classification errors and thus potentially lead to top-1 accuracies of 90% and above, at the cost of a 100-fold increase in training times, which is currently infeasible.

The histograms in Figure 4 show, next to the overall distribution, also the fraction of diffractograms classified wrongly for testing on the ICSD (red bar) and on the synthetic data (below the light blue line) for the ResNet-101 model. First, one can see that throughout almost all regions of the distributions, the accuracy on the synthetic data is slightly higher than that on the ICSD. This is related to the aforementioned gap of 12.3 percentage points between train and test accuracy and can be attributed to differences between the synthetic and ICSD distribution of crystals. This will be discussed in detail in the next section. It is surprising to see that the dependence on crystallite sizes is rather weak, as smaller crystallite sizes result in broader peaks (see Scherrer equation, Eq. 1), potentially making the classification harder due to more peak overlaps.

In summary, the maximum ICSD test accuracy of 79.9% that we achieved using the ResNet-101 model almost reaches the 81.14% that Park _et al._[11] reported for the space group classification. However, our accuracy is based on a train-test split based on structure types, in contrast to the random split used by Park _et al._. This creates a much harder but also realistic task to solve since the model needs to generalize to other structure types without merely recognizing diffractograms or structure types that it has already seen during training. This becomes especially apparent from our experiment directly trained on diffractograms from ICSD crystals with the split based on structure types, which reached only 56.1% instead of the 81.14% reported by Park _et al._[11].

Figure 5: Test accuracy (ICSD), training accuracy (synthetic crystals), and test top-5 accuracy (ICSD) as a function of epochs. We show all three metrics for the models ResNet-101, ResNet-50, and ResNet-10. To better show the scaling behavior, both axes use logarithmic scaling. To better see the exponential behaviour, see Figure S5 in the SI.
### Experimental results

To go beyond simulated diffractograms, we trained ResNet-50 models on calculated diffractograms with background, noise, and impurities and applied the trained models to the RRUFF mineral database. Our results (see Figure S3 in the SI) show that it is essential to include impurity phases in the training data. By doing so, we obtain a top-1 accuracy of 25.2% and a top-10 accuracy of over 60%. This is of high practical relevance since having a short list of potential space groups is often sufficient as a first step to further refinement and analysis.

Vecsei _et al._ performed similar experiments of space group classification on the same database. Using an ensemble of 10 fully connected neural networks, they reached a classification accuracy of 54%[14]. While our obtained accuracy is significantly lower than the one reported by Vecsei _et al._, our approach is much more general: The training dataset used by Vecsei _et al._ was based on simulated diffractograms of structures of the ICSD, which contains almost all RRUFF structures, leading to high similarities of training and testing data. Therefore, the model needed to simply recognize the minerals, instead of directly inferring the space group using the symmetry elements - as our method needs to do.

We want to emphasize that our efforts to apply the methodology to experimental data are only preliminary. We expect improved results with an improved data generation protocol since the procedure contains many parameters to be tuned. Ideally, one would use a generative machine learning approach to add the experimental effects (noise, background, impurities) to the pure diffractograms. We also want to point out that the noise level and quality of data in the RRUFF dataset are limited. Application of the presented methodology on other experimental datasets is desirable. We will address these points in future work, where we focus on improved ways of modeling experimental imperfections.

As discussed above, for the classification of pure diffractograms we observed the ResNet-50 to have the best cost-benefit ratio, since the ResNet-101 yielded only slight improvements. For the more complicated problem of classifying diffractograms with experimental imperfections, bigger models and longer training times might be necessary.

### Differences between synthetic crystals and ICSD crystals

We showed that training directly on crystals from the ICSD yields a gap between the training and test accuracy due to overfitting. The training on the synthetic dataset also shows a gap between the training and test accuracy (see Table 1), but it is smaller than when training directly on ICSD crystals. Furthermore, this gap is not due to overfitting, since overfitting to singular diffractograms is not possible when the model is trained using an infinite stream of generated synthetic crystals. The gap rather stems from systematic differences between the synthetic and ICSD distribution of crystals.

To analyze those differences, we created three modifications of the ICSD test dataset. In the first modification, the fractional coordinates of the atoms in the asymmetric unit of the crystals of the ICSD test dataset were randomly uniformly resampled (as in the synthetic crystal generation algorithm). In the second modification, the lattice parameters were randomized following the KDE used in the synthetic generation algorithm. The third modification combines both previous modifications, i.e. both the coordinates and the lattice parameters were resampled. These three modified test datasets bring the ICSD test dataset closer to the distribution used for training and let us quantify which factors contribute to the gap between training and testing.

We evaluated the test accuracies on these datasets for the experiment using the ResNet-101 model trained using synthetic crystals. We found that randomizing the coordinates yields an increase in test accuracy of 5.96 percentage points. Randomizing the lattice parameters results in an increase of 1.31 percentage points. Randomizing both the coordinates and the lattice parameters leads to an increase of 6.32 percentage points, explaining approximately half of the gap of 12.3 percentage points between synthetic training and ICSD test accuracy.

We also tested the ResNet-101 model on diffractograms simulated based on the statistics dataset from which the statistics for our generation algorithm have been extracted. This returns a 4.33 percentage points higher accuracy than on the test dataset. This is due to the different structure types present in the test and statistics dataset yielding different occupation probabilities (potentially also 0) for Wyckoff positions and overall slightly different statistics for the synthetic generation algorithm.
To summarize, the contributions to the gap stem from differences in the distribution of lattice parameters and coordinates with 6.32 percentage points and from the differences between the statistics dataset and test dataset with 4.33 percentage points. Added together (assuming the effects of geometry and Wyckoff position occupation probabilities are unrelated and thus additive), the contributions almost completely explain the gap of 12.3 percentage points between training and test accuracy. The remaining part is likely due to our algorithm that places atoms on Wyckoff positions not reproducing the ICSD distribution exactly. However, the remaining difference is remarkably small.

In Figure 6 we show the test classification error in each bin for the unit cell volume and the number of atoms in the asymmetric unit using the ResNet-101 model trained on diffractograms of synthetic crystals. The classification error is shown both for testing on diffractograms from synthetic crystals and on ICSD diffractograms. One can see that for small volumes and a small number of atoms in the asymmetric unit, the difference between classifying ICSD diffractograms and diffractograms from synthetic crystals is relatively small. As the volume and number of atoms in the asymmetric unit increase, the gap between the two errors increases, too. We already identified the uniformly sampled atom coordinates in the synthetic distribution as the main contributor to the gap in accuracy between the synthetic crystals and ICSD crystals. Therefore, it seems that the uniform sampling of atom coordinates works well for small number of atoms in the asymmetric unit and small volumes, while the error due to this sampling strategy increases slightly for higher volumes and higher number of atoms in the asymmetric unit.

When looking at the distribution of crystals in the ICSD, the number of atoms in the asymmetric unit tends to be larger for lower-symmetry space groups (for example, in the triclinic crystal system) than for higher-symmetry space groups such as those from the cubic crystal system. Therefore, the increasing test error on diffractograms from ICSD crystals with a higher number of atoms in the asymmetric unit is especially relevant for these lower-symmetry space groups. It might be possible that a different scheme of generating atom positions in the unit cell (compared to the independent uniform sampling that we used) works better for a high number of atoms in the asymmetric unit.

Overall, it is important to note that the distribution of ICSD crystals is (apart from a few Wyckoff position occupation probabilities which are exactly zero in the statistics dataset1) almost completely encompassed by the much larger distribution of synthetic crystals that we used for

Figure 6: Classification performance for each bin of a) the unit cell volume (conventional cell settings) and b) the number of atoms in the asymmetric unit. The classification performance when testing on the ICSD is shown using a stacked bar plot, where the red bar indicates the classification error and the green bar indicates the correctly classified portion of each bin. When testing on the synthetic distribution, the classification error is given by the light blue line, the rest (between dark blue and light blue line) was correctly classified. The reported classification performance is based on the ResNet-101 model trained on diffractograms from synthetic crystals. This visualization clearly shows the error rate within each bin, in contrast to Figure 4, which additionally includes the relative proportion of the crystals of the respective bin to the total amount of crystals.
training. However, due to finite training times and model capacity, a performance gap remains. This gap can be improved by using (substantially) more computing power or by narrowing the very general synthetic distribution, e.g., by using a different algorithm to generate atom positions. This indicates an inherent challenge in XRD classification but more generally in materials property prediction: Machine learning models are ultimately trained to be employed in real-world tasks, which are typically related to novel, i.e. yet unseen materials and structures. At the same time, the machine learning models are tested based on an IID assumption, i.e. the assumption that the distribution of training and testing data is the same. While not being a contradiction in the limit of infinite training data and model capacity, this becomes an (unsolvable) challenge in reality, when facing finite datasets and models. In our case, our model trained on a large distribution of synthetic crystal structures will likely generalize better to completely new crystal structures different from any crystal structure contained in the ICSD database. At the same time, it suffers from smaller ICSD test set errors, even though the ICSD distribution is contained in the synthetic data generation distribution.

## 4 Conclusion

We developed an algorithm based on the symmetry operations of the space groups to generate synthetic crystals that follow the distribution found in the ICSD database in terms of general descriptors like volume, density, or types of elements. The generated crystals have randomly sampled coordinates and span a wide range of structure types, many of which do not appear in the ICSD. We showed that, compared to using ICSD crystals directly, simulating the training data based on the synthetic crystals can improve the performance of tasks that extract structural information from powder diffractograms, in this case, the space group. Our results verify our hypothesis that crystals forming the training dataset do not need to be stable or chemically viable.

We trained on an infinite on-the-fly generated stream of synthetic crystals and simulated batches of diffractograms using a distributed framework based on the _Python_ library _Ray_[25]. This allows the training of very large networks without overfitting. The best-performing model (ResNet-101) reached a space group classification accuracy of 79.9% vs. 56.1% when training on ICSD structures directly. By performing the train-test split using the structure type, we forced our models to not just recognize structure types or individual structures, but to actually learn rules to distinguish different space groups by their symmetry elements. This shows the true generalization capabilities to new structure types and novel classes of materials. We also demonstrated first steps toward applying the presented methodology to an experimental dataset. We expect further improvements in this area using improved models of experimental imperfections, as well as larger ML models and longer training times.

Even though models trained on the synthetic distribution transfer well when tested on ICSD crystals, we found a gap of 12.3 percentage points (ResNet-101) between the training accuracy on synthetic crystals and test accuracy on the ICSD. We showed that the main contribution to this gap stems from the independently uniformly sampled atom coordinates. An improved approach may be needed to artificially generate more ordered structures, which contain more ordered diffraction planes than a cloud of uniformly sampled points. This might be especially important for crystals with a high number of atoms in the asymmetric unit.

Lastly, the developed algorithm to synthetically generate crystals can be used for other XRD-related tasks in the future, such as the extraction of crystallite sizes, lattice parameters, information about the occupation of Wyckoff positions, etc. Furthermore, instead of generating completely random crystals of all space groups, one can also generate random crystals of given structure types to solve more specialized tasks. This would allow the use of very large models for tasks that are typically strongly limited by the dataset size when using only the entries of the ICSD. Also, tasks concerning multi-phase diffractograms or augmentations such as strain in given crystal structures can benefit from our batch-wise online learning approach.

### Data availability

The ICSD data used to evaluate the models belongs to FIZ Karlsruhe, from which academic and non-academic licenses are available.
### Code availability

The source code of all machine learning models, of the generation of synthetic crystals, of the optimized simulation of diffractograms, and of the distributed computing architecture can be found on [https://github.com/aimat-lab/ML4pXRDs](https://github.com/aimat-lab/ML4pXRDs). The used machine learning models are further discussed in detail in the Supplementary Information.

### Acknowledgements

P.F. acknowledges support by the Federal Ministry of Education and Research (BMBF) under Grant No. 01DM21001B (German-Canadian Materials Acceleration Center). H.S. acknowledges financial support by the German Research Foundation (DFG) through the Research Training Group 2450 "Tailored Scale-Bridging Approaches to Computational Nanoscience". The authors acknowledge support by the state of Baden-Wurttemberg through bwHPC. Parts of this work were performed on the HoreKa supercomputer funded by the Ministry of Science, Research and the Arts Baden-Wurttemberg and by the Federal Ministry of Education and Research.

### Author contributions

All authors contributed to the idea and the preparation of the manuscript. H.S. implemented the methods and conducted the computational experiments.

### Competing interests

The authors declare no competing interests.
## References

* [1] Radovic, A., Williams, M., Rousseau, D., Kagan, M., Bonacorsi, D., Himmel, A., Aurisano, A., Terao, K. & Wongjirad, T. Machine Learning at the Energy and Intensity Frontiers of Particle Physics. _Nature_**560,** 41-48. issn: 1476-4687. doi:10.1038/s41586-018-0361-2. (2023-03-22) (7716 2018-08).
* [2] Rahmanian, F., Flowers, J., Guevarra, D., Richter, M., Fichtner, M., Donnely, P., Gregoire, J. M. & Stein, H. S. Enabling Modular Autonomous Feedback-Loops in Materials Science through Hierarchical Experimental Laboratory Automation and Orchestration. _Advanced Materials Interfaces_**9,** 2101987. issn: 2196-7350. doi:10.1002/admi.202101987. (2022-06-24) (2022).
* [3] Harris, K. D. M., Tremayne, M. & Kariuki, B. M. Contemporary Advances in the Use of Powder X-Ray Diffraction for Structure Determination. _Angewandte Chemie International Edition_**40,** 1626-1651. issn: 1521-3773. doi:10.1002/1521-3773(20010504)40:9<1626::AID-ANIE16260>3.0.CO;2-7. (2023-03-22) (2001).
* [4] Holder, C. F. & Schaak, R. E. Tutorial on Powder X-ray Diffraction for Characterizing Nanoscale Materials. _ACS Nano_**13,** 7359-7365. issn: 1936-0851. doi:10.1021/acsnano.9b05157. (2023-03-22) (2019-07-23).
* [5] Lee, J.-W., Park, W. B., Lee, J. H., Singh, S. P. & Sohn, K.-S. A Deep-Learning Technique for Phase Identification in Multiphase Inorganic Compounds Using Synthetic XRD Powder Patterns. _Nat Commun_**11,** 86. issn: 2041-1723. doi:10.1038/s41467-019-13749-3. (2021-09-06) (1 2020-01-03).
* [6] Maffettone, P. M., Banko, L., Cui, P., Lysogorskiy, Y., Little, M. A., Olds, D., Ludwig, A. & Cooper, A. I. Crystallography Companion Agent for High-Throughput Materials Discovery. _Nat Comput Sci_**1,** 290-297. issn: 2662-8457. doi:10.1038/s43588-021-00059-2. (2021-09-30) (4 2021-04).
* [7] Schuetzke, J., Benedix, A., Mikut, R. & Reischl, M. Enhancing Deep-Learning Training for Phase Identification in Powder X-ray Diffractograms. _IUCrJ_**8,** 408-420. issn: 2052-2525. doi:10.1107/S2052252521002402. (2021-09-16) (3 2021-05-01).
* [8] Szymanski, N. J., Bartel, C. J., Zeng, Y., Tu, Q. & Ceder, G. Probabilistic Deep Learning Approach to Automate the Interpretation of Multi-phase Diffraction Spectra. _Chem. Mater._**33,** 4204-4215. issn: 0897-4756. doi:10.1021/acs.chemmater.1c01071. (2021-09-10) (2021-06-08).
* [9] Wang, H., Xie, Y., Li, D., Deng, H., Zhao, Y., Xin, M. & Lin, J. Rapid Identification of X-ray Diffraction Patterns Based on Very Limited Data by Interpretable Convolutional Neural Networks. _J. Chem. Inf. Model._**60,** 2004-2011. issn: 1549-9596. doi:10.1021/acs.jcim.0c00020. (2021-09-29) (2020-04-27).
* [10] Lee, J.-W., Bae Park, W., Kim, M., Singh, S. P., Pyo, M. & Sohn, K.-S. A Data-Driven XRD Analysis Protocol for Phase Identification and Phase-Fraction Prediction of Multiphase Inorganic Compounds. _Inorganic Chemistry Frontiers_**8,** 2492-2504. doi:10.1039/DOI01513J. (2021-09-10) (2021).
* [11] Park, W. B., Chung, J., Jung, J., Sohn, K., Singh, S. P., Pyo, M., Shin, N. & Sohn, K.-S. Classification of Crystal Structure Using a Convolutional Neural Network. _IUCrJ_**4,** 486-494. issn: 2052-2525. doi:10.1107/S205225251700714X. (2021-09-06) (4 2017-07-01).
* [12] Oviedo, F., Ren, Z., Sun, S., Settens, C., Liu, Z., Hartono, N. T. P., Ramasamy, S., DeCost, B. L., Tian, S. I. P., Romano, G., Gilad Kusne, A. & Buonassisi, T. Fast and Interpretable Classification of Small X-ray Diffraction Datasets Using Data Augmentation and Deep Neural Networks. _npj Comput Mater_**5,** 1-9. issn: 2057-3960. doi:10.1038/s41524-019-0196-x. (2021-09-16) (1 2019-05-17).
* [13] Zaloga, A. N., Stanovovov, V. V., Bezrukova, O. E., Dubinin, P. S. & Yakimov, I. S. Crystal Symmetry Classification from Powder X-ray Diffraction Patterns Using a Convolutional Neural Network. _Materials Today Communications_**25,** 101662. issn: 2352-4928. doi:10.1016/j.mtcomm.2020.101662. (2022-04-06) (2020-12-01).
* [14] Vecsei, P. M., Choo, K., Chang, J. & Neupert, T. Neural Network Based Classification of Crystal Symmetries from X-Ray Diffraction Patterns. _Phys. Rev. B_**99,** 245120. doi:10. 1103/PhysRevB.99.245120. (2021-09-06) (2019-06-11).
* [15] Suzuki, Y., Hino, H., Hawai, T., Saito, K., Kotsugi, M. & Ono, K. Symmetry Prediction and Knowledge Discovery from X-ray Diffraction Patterns Using an Interpretable Machine Learning Approach. _Sci Rep_**10,** 21790. issn: 2045-2322. doi:10.1038/s41598-020-77474-4. (2021-09-26) (1 2020-12-11).
* [16] Chakraborty, A. & Sharma, R. A Deep Crystal Structure Identification System for X-ray Diffraction Patterns. _Vis Comput_**38,** 1275-1282. issn: 1432-2315. doi:10.1007/s00371-021-02165-8. (2022-09-13) (2022-04-01).
* [17] Ozaki, Y., Suzuki, Y., Hawai, T., Saito, K., Onishi, M. & Ono, K. Automated Crystal Structure Analysis Based on Blackbox Optimisation. _npj Comput Mater_**6,** 1-7. issn: 2057-3960. doi:10.1038/s41524-020-0330-9. (2021-09-17) (1 2020-06-05).
* [18] Feng, Z., Hou, Q., Zheng, Y., Ren, W., Ge, J.-Y., Li, T., Cheng, C., Lu, W., Cao, S., Zhang, J. & Zhang, T. Method of Artificial Intelligence Algorithm to Improve the Automation Level of Rietveld Refinement. _Computational Materials Science_**156,** 310-314. issn: 0927-0256. doi:10.1016/j.commatsci.2018.10.006. (2021-09-13) (2019-01-01).
* [19] Dong, H., Butler, K. T., Matras, D., Price, S. W. T., Odarchenko, Y., Khatry, R., Thompson, A., Middelkoop, V., Jacques, S. D. M., Beale, A. M. & Vamvakeros, A. A Deep Convolutional Neural Network for Real-Time Full Profile Analysis of Big Powder Diffraction Data. _npj Comput Mater_**7,** 1-9. issn: 2057-3960. doi:10.1038/s41524-021-00542-4. (2022-06-01) (1 2021-05-21).
* [20] Chitturi, S. R., Ratner, D., Walroth, R. C., Thampy, V., Reed, E. J., Dunne, M., Tassone, C. J. & Stone, K. H. Automated Prediction of Lattice Parameters from X-ray Powder Diffraction Patterns. _J Appl Cryst_**54,** 1799-1810. issn: 1600-5767. doi:10.1107/S1600576721010840. (2021-12-22) (6 2021-12-01).
* [21] Habershon, S., Cheung, E. Y., Harris, K. D. M. & Johnston, R. L. Powder Diffraction Indexing as a Pattern Recognition Problem: A New Approach for Unit Cell Determination Based on an Artificial Neural Network. _J. Phys. Chem. A_**108,** 711-716. issn: 1089-5639. doi:10.1021/jp0310596. (2021-10-01) (2004-02-01).
* [22] Banko, L., Maffettone, P. M., Naujoks, D., Olds, D. & Ludwig, A. Deep Learning for Visualization and Novelty Detection in Large X-ray Diffraction Datasets. _npj Comput Mater_**7,** 1-6. issn: 2057-3960. doi:10.1038/s41524-021-00575-9. (2021-09-30) (1 2021-07-09).
* [23] Bergerhoff, G. & Brown, I. in _Crystallographic Databases_ F.H. Allen et al. (Ed.) Chester, International Union of Crystallography (1987).
* [24] Lafuente, B., Downs, R. T., Yang, H. & Stone, N. _The Power of Databases: The RRUFF Project in Highlights in Mineralogical Crystallography_ isbn: 978-3-11-041710-4 (De Gruyter (O), 2015-11-13).
* [25] Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., Jordan, M. I. & Stoica, I. _Ray: A Distributed Framework for Emerging AI Applications_ in. 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) (2018), 561-577. isbn: 978-1-939133-08-3. [https://www.usenix.org/conference/osdi18/presentation/moritz](https://www.usenix.org/conference/osdi18/presentation/moritz) (2022-09-13).
* [26] Fredericks, S., Parrish, K., Sayre, D. & Zhu, Q. PyXtal: A Python Library for Crystal Structure Generation and Symmetry Analysis. _Computer Physics Communications_**261,** 107810. issn: 0010-4655. doi:10.1016/j.cpc.2020.107810. (2022-06-27) (2021-04-01).
* [27] Sun, Y., Wong, A. K. C. & Kamel, M. S. Classification of Imbalanced Data: A Review. _Int. J. Patt. Recogn. Artif. Intell._**23,** 687-719. issn: 0218-0014. doi:10.1142/S0218001409007326. (2022-06-24) (2009-06).
* [28] Ong, S. P., Richards, W. D., Jain, A., Hautier, G., Kocher, M., Cholia, S., Gunter, D., Chevrier, V. L., Persson, K. A. & Ceder, G. Python Materials Genomics (Pymatgen): A Robust, Open-Source Python Library for Materials Analysis. _Computational Materials Science_**68,** 314-319. issn: 0927-0256. doi:10.1016/j.commatsci.2012.10.028. (2022-06-21) (2013-02-01).
* [29] Lam, S. K., Pitrou, A. & Seibert, S. _Numba: A LLVM-based Python JIT Compiler_ in _Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC_ (Association for Computing Machinery, New York, NY, USA, 2015-11-15), 1-6. isbn: 978-1-4503-4005-2. doi:10.1145/2833157.2833162. (2022-09-13).
* [30]_International Tables for Crystallography Volume H: Powder Diffraction_ 1st ed. (eds Gilmore, C. J., Kaduk, J. A. & Schenk, H.) isbn: 978-1-118-41628-0 (Wiley, 2019-09-24).
* [31] He, K., Zhang, X., Ren, S. & Sun, J. _Deep Residual Learning for Image Recognition_ in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016-06), 770-778. doi:10.1109/CVPR.2016.90.
Supplementary Information

to

Neural networks trained on synthetically generated crystals can extract structural information from ICSD powder X-ray diffractograms

Henrik Schopmans, Patrick Reiser, and Pascal Friederich*

## S1 Generating synthetic crystals

Here we describe the algorithm to generate synthetic crystals in more detail. To generate a single crystal, the following steps are executed:

1. Random selection of a space group. We follow the space group distribution of the ICSD to allow comparison with previous work (see Section 2.1 of the paper).
2. The number of unique elements in the crystal is drawn from a discrete distribution extracted from the crystals in the ICSD belonging to the specified space group.
3. The unique elements are drawn, also from a discrete probability distribution from the crystals in the ICSD belonging to the specified space group.
4. For each of the unique elements, the number of repetitions in the asymmetric unit is chosen, and for each repetition, a Wyckoff position is randomly selected. Again, both the probability of the number of repetitions and the Wyckoff occupation probabilities are extracted from the ICSD for the specified space group. We do not place more than one atom onto a Wyckoff position that does not have a degree of freedom.
5. For each atom placed on a Wyckoff position, uniformly distributed random fractional coordinates are drawn.
6. Lattice parameters (normalized to unity volume) of the crystal system that the specified space group belongs to are drawn from a kernel density estimate (KDE) of the ICSD. The bandwidth is chosen based on Scott's rule (see the _SciPy_1 implementation of KDE). Footnote 1: For this conditional kernel density estimate, we used the implementation of _statsmodels_2 with the normal reference rule of thumb to estimate the bandwidth.
7. We generated a KDE of the volume conditioned1 on \(\sum_{i}4/3\pi\left(\frac{r_{i\mathrm{cov}}+r_{i\mathrm{VdW}}}{2}\right)^{3} =V_{\mathrm{atomic}}\), where the sum covers all atoms in the conventional unit cell, \(r_{i\mathrm{cov}}\) is the atomic covalent radius, and \(r_{i\mathrm{;VdW}}\) is the atomic van der Waals radius. The KDE was generated from all crystals of the ICSD belonging to the specified space group. Then, \(V_{\mathrm{atomic}}\) is calculated for the chosen atoms in the conventional unit cell and the volume is drawn based on the KDE conditioned on \(V_{\mathrm{atomic}}\). The lattice parameters (chosen in the previous step) are further scaled by the cube root of the chosen volume. Footnote 1: For this conditional kernel density estimate, we used the implementation of _statsmodels_3.
8. Space group symmetry operations are applied using _Python_ library _PyXtal_3.

Footnote 3: Reference rule of thumb to estimate the bandwidth.

When generating a crystal of a specific space group without placing an atom on the general Wyckoff position, it is not always the case that the crystal belongs to that space group. To prevent wrong space group assignments, we use the _Pymatgen_4 interface to _Spglib_5 to check the space group of each crystal after its generation. If the space group deviates, we generate a new crystal with the same number of unique elements as before, in order to not distort the distribution of number of unique elements extracted from the ICSD. If the generation fails 20 times in total, we start from the beginning with a new number of unique elements.
Machine learning

### Models

We now want to describe the machine learning models that we used for the classification of space groups in more detail. Powder diffractograms include similar features (peaks) at different locations and the position of a feature in the diffractogram has a spatial meaning. This suggests that the properties of the convolution operation, namely the parameter sharing (with sparse connectivity) and equivariance[6], might be beneficial when processing powder diffractograms.

As a baseline, we first used the two CNN architectures used by Park _et al.[7]_ for the classification of extinction groups and space groups. Since our training dataset is an infinite stream of diffractograms and we do not have to worry about overfitting, we further used the deeper architectures ResNet-10, ResNet-50, and ResNet-101[8]. All architectures are now described in detail.

#### Architectures by Park _et al._

Park _et al.[7]_ introduced three models, one for the classification of the crystal system, one for extinction groups, and one for space groups. We used only the last two models and call them "parkCNN medium" and "parkCNN big", respectively.

"parkCNN big" consists of three convolution layers with average pooling, two hidden fully connected layers with 2300 and 1150 nodes, and a 145-dimensional softmax output. The architecture is summarized in Figure S1. The "parkCNN medium" model has fewer parameters than "parkCNN big" since the two hidden fully connected layers have 4040 and 202 nodes.

#### ResNet architecture

With increasing size of the training dataset and increasing difficulty of the chosen task, the number of model parameters needs to be increased, too.

In principle, a deeper model with additional layers should always be able to express the same solution of a shallower model by simply "learning" an identity map in addition to the shallower model. In practice, however, a degradation problem for CNNs with increasing depth has been observed and very deep models can perform worse than their shallow counterpart[8]. Therefore, the ResNet architecture developed by He _et al.[8]_ at Microsoft in 2015 introduced additional skip connections, where information is able to simply flow past the convolution layer and is added to its output. This makes it possible for needed information of the input or earlier layers to flow further into the model without degradation.

Figure S2a visualizes the residual block used for the shallower versions of the ResNet architecture (up to 34 layers). Figure S2b visualizes the bottleneck block used for the deeper variants (50 and more layers). This type of building block is called a bottleneck block since it first reduces the number of channels using a 1x1 convolution operation with N filters. Then, the main convolution with a 3x3 kernel is performed, followed by a third 1x1 convolution that upscales to 4N channels. This down- and upscaling of the number of channels is performed to increase the performance of the model. All convolution operations of both types of building blocks are followed by batch normalization implicitly.

In the simplest case, the skip connection of the residual and bottleneck block is simply an identity mapping and added to the output of the block. However, if the number of input channels and dimensions of a block are different from those in the output, a projection in the form of a 1x1 convolution with the necessary number of filters and stride (usually 2) is used instead of the identity.

Table S1 summarizes the ResNet-10, ResNet-50, and ResNet-101 models. Each architecture is to be read from top to bottom. The square brackets indicate a residual or bottleneck block with their two or three convolution operations and respective number of filters. The number after the square brackets "\(\times\)_N_" indicates how often the building block is to be repeated in the respective block group.

If the output dimension changes from one block group to the next, the first building block of the next block group downsamples the dimensions by using a stride of 2 for the first 3x3 convolution in the case of a residual block and for the middle 3x3 convolution in the case of a bottleneck block. All other convolution operations of the building blocks are performed with stride 1.
We used the ResNet implementation as found in the _TensorFlow_ model garden[9]. Since our data is one-dimensional, we used an adapted 1D version. We replaced all 2D convolutions and pooling operations with their 1D equivalent (\(N\times N\to N\)). Furthermore, we used a kernel size of 9 in place of the 3x3 kernels and stride 4 instead of stride 2 in the bottleneck blocks and projection skip connections (\(N\times N\to N^{2}\)). This squaring was performed to obtain a better distribution of the number of weights throughout the architecture (similar to the original 2D case). We further added an additional fully connected layer with 256 nodes after the flatten layer in the end of the ResNet models, followed by the output layer.

We were not able to achieve good results when using the original ResNet architecture with batch normalization (similar observations were made by Schuetzke _et al.[10]_). The test accuracy calculated after each epoch was highly unstable and had high fluctuations, probably in part due to the moving averages of the batch normalization not converging properly. This is possibly caused by using an infinite stream of batches of diffractograms, instead of using a training dataset of fixed size. We fixed this problem by using group normalization[11] with 32 groups instead of batch
normalization.

### Setup

We used a distributed architecture on multiple nodes using the _Python_ framework _Ray_12. The configuration utilized throughout this study is visualized in Figure 2b of our paper. Training took place on a _Ray_ head node with one or two2 RTX 2080 Ti GPUs. Training on two GPUs was performed using the _MirroredStrategy_ of _TensorFlow_13. 28 out of the 32 cores of the head node were used for the generation of diffractograms. Next to the head node, we used two additional compute nodes with 128 cores each to generate diffractograms. Communication between the nodes took place using a _Ray queue_ object to access the simulated diffractograms.

Footnote 1: The “parkCNN” models used a single GPU, while the ResNet models were trained on two GPUs.

Depending on the model size and corresponding training speed, this setup allows training with up to millions of unique diffractograms per hour. For the larger ResNet variants ResNet-50 and ResNet-101, we efficiently used the GPUs. For the "parkCNN" and ResNet-10 models, training is
fast and the data generation becomes the bottleneck.

To train all models, we used _Keras_14 with _TensorFlow_ 2.313. Optimization was performed using _Adam_15 with \(\beta_{1}=0.9\) and \(\beta_{2}=0.999\) (_Keras_ default parameters). We also tried to use stochastic gradient descent (SGD) with momentum and a stepwise learning rate scheduler, but this did not yield good results. Depending on the initial conditions, most of the training runs using SGD were either unstable or reached low accuracies. For all experiments, a cross-entropy loss was utilized.

Footnote 13: Note that the division into epochs for training using synthetic crystals is rather arbitrary since each epoch contains different diffractograms simulated from different crystals. However, the division makes it easy to calculate the performance on the test dataset after each epoch.

We used a batch size of 870 for the "parkCNN" models and a batch size of 145 for the three ResNet models. Furthermore, the "parkCNN" models were trained for 1000 epochs with a learning rate of 0.001, while the ResNet models were trained for 2000 epochs with a learning rate of 0.0001. The ResNet models used a step decay of the initial learning rate, halving the learning rate after every 500 epochs.

When training on diffractograms simulated from synthetic crystals, we used 150 generated batches per epoch when a batch size of 870 was used ("parkCNN medium" and "parkCNN big") and 900 batches per epoch when a batch size of 145 was used (ResNet). This means that each epoch always contained 130 500 diffractograms1.

Footnote 1: Note that the division into epochs for training using synthetic crystals is rather arbitrary since each epoch contains different diffractograms simulated from different crystals. However, the division makes it easy to calculate the performance on the test dataset after each epoch.

For the experiments performed directly on diffractograms simulated from ICSD crystals, we used the statistics dataset directly to pre-generate the training dataset. We excluded the same 85 space groups that were not used by the synthetic training due to missing statistics. We used two different crystallite sizes for each crystal in the statistics dataset, yielding \(148\,466\cdot 2=296\,932\) diffractograms in the training dataset.

## S3 Application to experimental diffractograms

### Dataset

To test the performance on experimental diffractograms, we used the publicly available RRUFF mineral database16. It contains 5829 mineral samples with multiple types of measured spectra and, most important for us, powder XRD measurements with K\({}_{\alpha}\) radiation for 2952 samples. Of these 2952 samples, 2875 samples provide the output of a Rietveld refinement, including the space group label. We further removed by hand some samples that had excessive amounts of noise and were of bad quality. Many samples further only provide a simulated diffractogram without noise or background. They were also excluded. This left 942 diffractograms for our analysis.

Footnote 16: Note that the division into epochs for training using synthetic crystals is rather arbitrary since each epoch contains different diffractograms simulated from different crystals. However, the division makes it easy to calculate the performance on the test dataset after each epoch.

### Data generation

To be able to apply our models to experimental diffractograms, we added an additional background function and noise to the generated diffractograms to make them more similar to experimental data. We used a Gaussian process to generate random background functions and added additive and multiplicative Gaussian noise. All diffractograms were generated in the range \(2\theta\in[5,90]\) * with step size \(0.01^{*}\).

The generation protocol is as follows:

1. Sample the background function from a Gaussian process with radial basis function kernel without any conditioning: \[k(x_{i},x_{j})=a\exp\left(-\frac{|x_{i}-x_{j}|^{2}}{2l^{2}}\right)\] (1) We chose \(a=1\) and sampled \(l\) uniformly in [7, 40] for each diffractogram.
2. Subtract the minimum intensity from the background function obtained from the Gaussian process.
3. Divide it by the sum of the 8501 (\(2\theta\)-range) entries.
4. Multiply it by a scaling factor drawn from a truncated normal distribution in the range \([0,150]\) with mean \(0\) and standard deviation \(75\).
5. Add the pure diffractogram (intensity-range \([0,1.0]\)).
6. Add Gaussian additive noise with mean \(0\) and standard deviation uniformly drawn in \([0,0.02]\).
7. Multiply with multiplicative Gaussian noise with mean \(1\) and standard deviation uniformly drawn in \([0,0.16]\).

To resemble a more realistic peak profile, we used the pseudo-Voigt profile instead of the Gaussian profile that we used for the classification of pure diffractograms. The pseudo-Voigt profile uses the full width at half maximum (FWHM) \(\Gamma_{\mathrm{G}}\) of the Gaussian \(G\), the FWHM \(\Gamma_{\mathrm{L}}\) of the Lorentzian \(L\), and a mixing parameter \(\eta\) as parameters[17]:

\[\mathrm{PV}\left(\theta-\theta_{0};\Gamma_{\mathrm{L}},\Gamma_{\mathrm{G}} \right)=\eta G\left(\theta-\theta_{0};\Gamma_{\mathrm{G}}\right)+(1-\eta)L \left(\theta-\theta_{0};\Gamma_{\mathrm{L}}\right) \tag{2}\]

with

\[G(\theta-\theta_{0};\sigma=\frac{\Gamma_{\mathrm{G}}}{2\sqrt{2\ln 2}})= \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{\theta-\theta_{0}}{ \sigma}\right)^{2}} \tag{3}\]

and

\[L(\theta-\theta_{0};\gamma=\frac{1}{2}\Gamma_{\mathrm{L}})= \frac{1}{\pi\gamma}\left[\frac{\gamma^{2}}{\left(\theta-\theta_{0}\right)^{2 }+\gamma^{2}}\right]\,. \tag{4}\]

The FWHM \(\Gamma_{\mathrm{G}}\) of the Gaussian is typically parametrized using the Caglioti equation[17] as

\[\Gamma_{\mathrm{G}}^{2}=U\tan^{2}\theta+V\tan\theta+W\,. \tag{5}\]

Following the suggestions for typical Rietveld parameter ranges by Kaduk & Reid[18] and comparing the resulting peaks with the ones from the RRUFF dataset, we decided to sample the Caglioti parameters uniformly in the following ranges: \([0,0.01]\) for U, \([0,0.01]\) for W, and V was fixed at \(V=0.0\). We further used a single mixing parameter \(\eta\) uniformly sampled in \([0.0,1.0]\) for the full \(2\theta\)-range. For simplicity, we used the same FWHM from the Caglioti equation for both the Gaussian and Lorentzian of the pseudo-Voigt. We further considered the K\({}_{\alpha_{1}}\) / K\({}_{\alpha_{2}}\) splitting of the K\({}_{\alpha}\) line since for some diffractograms in the RRUFF dataset, this splitting is visible.

We further implemented the option to add impurity phases to the training (and simulated ICSD test) data. The minerals of the RRUFF database are not all made up of one phase, but most of them contain small amounts of one or more impurity phases. To model this, for each training diffractogram, we used a superposition of the main phase to be classified and an impurity phase of a random space group (\(a\) is uniformly sampled in \([0,0.05]\)):

\[I(\theta)=(1-a)I_{\mathrm{pure}}+aI_{\mathrm{impurity}} \tag{6}\]

### Experiments

For our experiments on experimental data, we used the same split based on structure types as we used for pure diffractograms. We performed two experiments using the ResNet-50 architecture, one with impurity phases and one without. For both, a learning rate of \(0.0001\), a batch size of \(145\), and \(1000\) epochs where used.
## S4 Additional figures