Machine learning of phases and structures for model systems in physics
Djenabou Bayo Burak Çivitcioğlu Joseph J Webb Andreas Honecker Rudolf A. Römer

JuliaQCD: Portable lattice QCD package in Julia language
    Akio Tomiya
    September 9, 2024
========================================================






§ INTRODUCTION




Identification of critical points separating distinct phases of matter is a central pursuit in condensed matter and statistical physics <cit.>. This task requires a thorough understanding of the global behavior of the many-body system because phenomena may emerge that are very difficult to derive from microscopic rules <cit.>.
Traditional analytic methods and numerical simulations have proven effective in understanding these complex systems <cit.>, but they often come with limitations, particularly in high-dimensional parameter space <cit.>.  

Machine-learning methods, particularly supervised <cit.> and unsupervised learning techniques <cit.>, have in the last years appeared in physics as a novel strategy to bypassing some of these limitations <cit.>. Convolution neural networks (CNN), a class of deep, i.e., multi-layered, neural networks (DNNs) in which spatial locality of data values is retained during training, have, when coupled with a form of residual learning <cit.>, shown to allow astonishing precision when classifying images, e.g., of animals <cit.> and handwritten characters <cit.>, or when predicting numerical values, e.g., of market prices <cit.>. 

These supervised learning strategies similarly yield promising predictions in identifying critical points or phases in parameter space <cit.>, providing an alternative and potentially more efficient way of exploring complex systems.

By now, the evidence in favour of supervised machine-learning methods' efficacy in identifying different phases of a physical system appears compelling <cit.>. 

Unsupervised learning and semi-unsupervised learning approaches have also demonstrated the ability to reconstruct the outlines of a system's phase diagram.<cit.>. The potential to identify structural changes within a system further supports the significance of these techniques in modern scientific exploration <cit.>.


Among the various models studied in the context of machine learning and statistical physics, the Ising model on the square lattice has served as an important benchmark <cit.> due to the simplicity of its two thermal phases, the low-temperature ferromagnet and the high-temperature paramagnet, and the ready availability of its exact solution <cit.> with exactly known critical temperature. 

We note that the use of ML to determine phases from just the spin configurations suggests that these themselves should contain sufficient information to identify phases, providing a level of physical insight that was, while not unknown, at least not as clear as it now seems. 
We also mention related work on multi-layer <cit.> and Potts models <cit.>, where the latter include the Ising model as the q=2 case.


Percolation can be considered as the q→1 limit of the Potts model <cit.>
and yields another class of paradigmatic models to which machine-learning techniques have been applied to identify the non-spanning and spanning phases<cit.>. 

Previous ML studies have mostly used supervised learning in order to find the two phases via ML classification <cit.>. An estimate of the critical exponent of the percolation transition has also been given <cit.>. The task of determining the transition threshold, p_c, was further used to evaluate different ML regression techniques<cit.>.
For unsupervised and generative learning, less work has been done <cit.>. While some successes have been reported <cit.>, other works show the complexities involved when trying to predict percolation states <cit.>.


Disordered electron systems provide quantum systems with similarly rich phase diagrams. Examples are given by the Anderson insulator<cit.>, diffusive metals<cit.>, the quantum Hall<cit.> and quantum anomalous Hall insulators<cit.>, Weyl semimetals<cit.>, as well as topological insulators<cit.>. In these cases, the thermal states investigated for Ising-type models are replaced by quantum mechanical eigenfunctions, or variations thereof such as the local density of states (LDOS). These have specific features in each phase but, due to the random nature of these systems, precisely determining a phase from
an LDOS is difficult.<cit.> Recent supervised learning work on the Anderson model of localization, capturing the features of eigenfunctions across the delocalization–localization transition,<cit.> as well as further transfer-learning approaches to the disordered Chern insulator–Anderson insulator transition,<cit.> have shown to allow a seemingly accurate description of phases and phase boundaries.

The power of generative machine learning has not yet been harnessed to the same extent. This is partly because it is still a relatively novel machine learning strategy <cit.>. In brief, the difference to the supervised methods lies in the generative methods being able to seemingly create novel predictions which do not appear in any of the provided data. 
For example, in computer vision, generative networks construct previously non-existent high-resolution images, conditional on information from other images <cit.>.

Here, we will show how to use such generative ML strategies to study the phases for the J_1-J_2 Ising model, an extension of the Ising model that incorporates competing interactions across the diagonals of the Ising squares and presents a more challenging 3-phase structure. As ML generator, we shall use a so-called variational autoencoder (VAE), a type of neural network that reconstructs a given predicted state after being trained on a selected set of states <cit.>.


The application of ML to structure determination via electron diffraction has also blossomed in the last decade <cit.>. ML strategies have been used to reduce the data flow in single-molecule data classification <cit.>, convolutional neural nets (CNNs) were shown to help with phase reconstruction for convergent-beam electron diffraction (CBED)-based scanning transmission electron microscopy (TEM) <cit.> while molecular structure imaging was found to benefit from such CNNs as well <cit.>. At the core of the deep learning methods employed in these works lie the same supervised DL techniques as used for phase determination. Again, generative ML for electron diffraction is not so common. Here, we will show how a so-called conditional generative adversarial network (cGAN) can be used to make accurate predictions of large-angle CBED (LACBED) images from just standard crystal information as encoded, e.g., in the usual text information<cit.> given in the Inorganic Crystal Structure Database (ICSD) <cit.>, the world’s largest such database. 




§ A BRIEF RECAP OF THE ML APPROACH TO PHASES AND STRUCTURES







 §.§ Classification and regression



Machine learning (ML) differs from traditional programming in that it does not rely on explicit rules to solve tasks. Instead, the network is expected to develop a strategy based on the input dataset to accomplish the required task.
There are three primary types of learning: supervised learning, unsupervised learning, and reinforcement learning.<cit.> Here, we will focus mainly on the first two.<cit.>

Supervised learning aims to discover the optimal strategy for performing a task by using a labeled dataset.
Within supervised learning, two key tasks can be identified: classification and regression.
In classification, the ML model learns to divide data into distinct categories. Essentially, it finds an optimal representation of the dataset that separates samples into different classes.
In regression, the algorithms are trained to understand the relationship between inputs and labels, enabling them to make continuous predictions for new, unseen labels based on the given inputs. This sets regression apart from classification, as it allows the model to predict values for data not encountered during training.

The second type of learning is unsupervised learning. In this approach, the ML algorithm processes unlabeled data and is expected to uncover hidden patterns or correlations without any external guidance.
Unsupervised learning is further divided into three categories: clustering, dimensionality reduction, and association learning.
Clustering aims to group similar samples within the dataset. Dimensionality reduction seeks to simplify the data representation while retaining its essential characteristics. Association learning explores relationships between different samples in the dataset. Unsupervised learning has a wide range of applications. It can be used as a preprocessing step to reveal the structure of a dataset before supervised learning begins.<cit.> It also powers generative methods, such as VAEs and GANs, which create new data samples.
    






















 §.§ Generative ML: VAEs and cGANs





A Variational Autoencoder (VAE) represents a relatively recent deep learning architecture that integrates standard compression techniques with the regularization strategies of machine learning, functioning simultaneously as a generative model <cit.>.
In essence, a VAE comprises an encoder, which is a multilayered neural network trained on input data to generate output parameters for a variational distribution. These parameters define a low-dimensional probabilistic distribution, referred to as the latent space.
The decoder, another deep neural network architecture, then reconstructs the output data from the latent space, drawing samples from this space rather than selecting deterministic points.

When the latent space dimensionality, d, is significantly smaller than the information content of the input data, some degree of information loss is inevitable. Thus, the goal is to design the encoder and decoder in such a way that maximizes the preservation of information during encoding while minimizing the error in the reconstructed data during decoding.

To effectively train a VAE, two primary loss functions are utilized. The reconstruction loss ℓ_ε measures the discrepancy between the input and reconstructed output during training. Additionally, the Kullback-Leibler divergence <cit.>, which serves as a regularization term, ensures that the latent space approximates a standard normal distribution <cit.>.

In practice, the training process involves minimizing a total loss ℓ, which is a combination of the reconstruction loss ℓ_ε and the Kullback-Leibler loss ℓ_KL, such that ℓ = ℓ_ε + c  ℓ_KL, where c is a hyperparameter that balances the two components <cit.>.













GANs have emerged as a highly popular architecture for image-to-image translation tasks <cit.>. While VAEs are known to struggle with producing high-fidelity outputs, often resulting in blurriness <cit.>, GANs inherently avoid this issue by design <cit.>. An absence of blurriness is particularly critical in quantitative electron diffraction, where clarity is essential. For this reason, we focus on conditional GANs (cGANs) <cit.>, which are well-suited to our image-to-image task involving the learning of a mapping from an input image x and random noise vector z to a target image y, denoted as G: x, z→ y. In this context, G represents the generator.

GANs also introduce a second component, the discriminator, denoted as D. The discriminator is trained to differentiate between `real' images from the dataset and `fake' images generated by G. This adversarial setup ensures that the generator improves over time, as the discriminator learns to recognize blurry images as fake, thereby driving the generator to produce sharper outputs.

Unlike VAEs, which rely on a predefined loss function, GANs instead learn a loss function for the desired task, solving another problem: deciding which loss function to use for comparing diffraction patterns is not apriori clear and can vary between different applications.













§ PREDICTING PERCOLATING CLUSTERS WITH CNNS




This section reviews work done previously,<cit.>
where we showed that standard CNNs, usually employed in image recognition ML tasks, also work very well for classifying site percolation states according to occupation probability p as well as for regression when determining p from such states. 

However, analyzing in detail whether spanning clusters at p < p_c or non-spanning clusters at p> p_c are correctly identified, we found that the same CNNs consistently fail to reflect the ground truth. Rather, it appears that the CNNs use p as a proxy measure to inform their classification predictions — a strategy that is obviously false for the percolation problem.




 §.§ The physics model of “percolation”


The percolation problem is well-known with a rich history across the natural sciences <cit.>. It  provides the usual statistical characteristics across a second-order transition such as, e.g., critical exponents, finite-size scaling, renormalization and universality <cit.>. 



Briefly, on a percolation lattice of size L × L, individual lattice sites x⃗=(x,y), x,y ∈ [1,L], are randomly occupied with occupation probability p such that the state ψ of site x⃗ is ψ(x⃗)=1 for occupied and ψ(x⃗)=0 for unoccupied sites. 

We say that a connection between neighboring sites exists when these are side-to-side nearest-neighbors on the square lattice, while diagonal sites can never be connected. A group of these connected occupied sites is called a cluster (cf. Fig. <ref>(a)). 



Such a cluster then percolates when it spans the whole lattice either vertically from the top of the square to the bottom or, equivalently, horizontally from the left to the right. Obviously, for p=0, all sites are unoccupied and no spanning cluster can exist while for p=1 the spanning cluster trivially extends throughout the lattice.
In Fig.  <ref>(a), we show examples of percolation clusters generated for various p values.










The percolation threshold is at p=p_c(L), such that for p< p_c(L) most clusters do not span while for p > p_c(L)

there is at least one spanning cluster. 
This can be expressed via
the quantities P(p), Q(p)=1-P(p) 
that denote the probabilities of the presence or absence of the spanning cluster at a given p, respectively (cf. Fig. <ref>(b)). 
We note that P is a finite-L version of ψ in the notation of <cit.>.
We will occasionally emphasize this point using  P_L and, likewise, Q_L.

For an infinite system (L→∞), one finds the emergence of an infinite spanning cluster at p_c=0.59274605079210(2). This estimate has been determined numerically evermore precisely over the preceding decades <cit.> while no analytical value is yet known <cit.>.




 §.§ The ML approach to the percolation problem and the generation of ML “data”



Several ML studies on the percolation model have been


published, mostly using supervised learning in order to identify the two phases via ML classification <cit.>. 




In order to facilitate the recognition of percolation with image recognition tools of ML, we have generated finite-sized L × L, with L=100, percolation states, denoted as ψ_i(p), for the 31 p-values 0.1, 0.2, …, 0.5, 0.55, 0.555, 0.556, …, 0.655, 0.66, 0.7, …, 0.9. For each such p, N=10000 different random ψ_i(p) have been generated. 
Each state ψ_i(p), i=1, …, N, is of course just an array of numbers with 0 denoting unoccupied and 1 occupied sites. Nevertheless, we occasionally use for convenience the term “image” to denote ψ_i(p).
The well-known Hoshen-Kopelman algorithm <cit.> is employed to identify and label clusters from which we (i) compute s(p) and (ii) determine the presence or absence of a spanning cluster. Correlation measures have also been calculated but are not shown here for brevity.<cit.>








We emphasize that in the construction, we took care to only construct states such that for each p, the number of occupied sites is exactly N_occ= p × L^2 and hence p can be used as exact label for the supervised learning approach. Hence p= N_occ / L^2 can also be called the percolation density. 

For the ML results discussed below, it will also be important to note that the spacing between p values reduces when p reaches 0.5 with the next p value given by 0.55 and then 0.555. Similarly, the p spacing increases as 0.655, 0.66, 0.7. We will later see that this results in some deviations from perfect classification/regression. For reference, we now have 12 values p=0.1, …, 0.58< p_c(100) and 18 values p=0.59, …, 0.9> p_c(100). We also note that the training set contains 92.7% of states without a spanning cluster below p_c and 94.8% are spanning above p_c.

We have also generated similar training and test sets for L=200; our results do not change significantly<cit.>.

Last, all our ML results have been obtained from ten training, validation and test cycles allowing us to quote ML indicators, such as losses, accuracies, in terms of averages and their errors.<cit.> Our CNN uses the ResNet18 implementation of PyTorch.<cit.>




 §.§ Results for ML classification according to spanning or non-spanning properties

 
The hallmark of the percolation transition is the existence of a spanning cluster which determines whether the system is percolating or not <cit.>. 


We now want to check this and label all states according to whether they are spanning or non-spanning. From Fig. <ref>(b), it is immediately clear that for finite-sized systems considered here, there are a non-negligible number of states which appear already spanning even when p < p_c and, vice versa, are still non-spanning when p > p_c. Furthermore, we note that for such L, the difference between p_c and p_c(L) is large enough to be important and we hence use p_c(L) as the appropriate value to distinguish the two phases.

Fig. <ref> shows the averaged results after ϵ=20 with a validation loss of min_ϵ[⟨ l_c,val⟩]=0.165  ± 0.001 (corresponding to a maximal validation accuracy max_ϵ[⟨ a_c,val⟩]= 92.702%± 0.001).



At first glance, the figure seems to indicate a great success: from the 31000 states present in τ, 11510.6 have been correctly classified as non-spanning (i.e., N→ N'), and 17206.9 as spanning (S→ S') while only 1223.1 are wrongly labeled as non-spanning (S→ N') and 1059.4 as spanning (N→ S') (We note that these numbers are not integers since they are computed as averages over 10 independent training runs <cit.>). 
Overall, we would conclude that 92.6% of all test states are correctly classified while 7.4% are wrong. 




However, from the full percolation analysis for τ, we can compute that there are 11127 states (92.7%) without a spanning cluster below p_c(L) while 873 states (7.3%)

already contain a spanning cluster. Similarly, for p>p_c(L), 94.9% of states, equivalent to 17075 states, are spanning and 5.1% are not, corresponding to 925 states. At p_c(L)=0.585, we furthermore have 482 spanning and 518 non-spanning states. Hence in total, we expect 2280 wrongly classified states.

Since the last number is very close to the actual number of 2282.5 of misclassified states, this suggests that it is precisely the spanning states below p_c(L) and the non-spanning ones above p_c(L) which the DL network is unable to recognize. 

Let us rephrase for clarity: it seems that the CNN, when trained in whether a cluster is spanning or non-spanning, completely disregards this information in its classification outputs. 
We show that this is indeed the case by a detailed analysis of the clusters around p_c as well as test sets which have been constructed to allow testing for the existence of the spanning cluster.<cit.> 

In summary, when looking at p, classification and regression techniques for percolation states allow us to obtain good recognition with near-perfect ⟨ a_c,val⟩ = 99.323%± 0.003) for classification (cf. also Fig. <ref>(c)) and near-zero ⟨ l_r,val⟩ =0.000062 ± 0.000012 average mean-square loss for regression.<cit.>

On the other hand, the DL network completely ignores whether a cluster is spanning or non-spanning, essentially missing the underlying physics of the percolation problem — it seems to still use p as its main ordering measure. We believe that the root cause of the failure to identify the spanning clusters, or their absence, lies in the fundamentally local nature of the CNN: the filter/kernels employed in the ResNets span a few local sites only. Hence it is not entirely surprising that such a CNN cannot correctly identify the essentially global nature of spanning clusters. But it is of course exactly this global percolation that leads to the phase transition. This should serve as a warning to enthusiastic proponents of the ML approach not to ignore the physics undeservedly.






§ RESOLVING DISORDER STRENGTHS FROM IMAGES OF THE 3D ANDERSON MODEL








One of the hardest challenges in modern eigenvalue computation is the numerical solution of large-scale eigenvalue problems, in particular those arising from quantum physics<cit.>. 
Typically, these problems require the computation of some eigenvalues and eigenvectors for systems which have up to several million unknowns due to their high spatial dimensions.
Here, the Anderson model of localization<cit.> is a particularly paradigmatic model as its underlying structure involves random perturbations of matrix elements which invalidates simple preconditioning approaches based on the graph of the matrices.<cit.> 
Its physical importance comes from the prediction of a spatial conﬁnement of the electronic motion upon increasing the disorder – the so-called Anderson localization.<cit.> When the model is used in three spatial dimensions, it exhibits a metal-insulator transition in which the disorder strength w mediates a
change of transport properties from metallic behavior at small w via critical behavior at the transition w_c ∼ 16.57 to insulating behavior and strong localization at larger w> w_c.<cit.> 

The 3D Anderson model hence provides us with a physically meaningful quantum problem in which to use ML strategies to distinguish its two phases, namely the metallic phase with extended states at w<w_c and the insulating phase with localized states at w>w_c (Occasionally, one might want to also study w ≈ w_c as a 3rd phase), while avoiding the many challenges of fully interacting quantum systems.<cit.> In this sense, it can be seen as the quantum ML test partner to complement the classical statistical physics tests available via the percolation and Ising-type models.

Similarly to the percolation model, previous ML studies have already been performed and showed good success for ML classification with CNNs to identify the two phases of the system <cit.>. 
Here, we show that not only phases but also disorder strengths can be recovered from eigenstates of the 3D Anderson model.






 §.§ The formulation of the Anderson model in 3D


In its usual form, the localization problem in 3D with coordinates x, y, z corresponds, in the absence of a magnetic field, to a Hamilton operator in the
form of a real symmetric matrix H, with quantum mechanical energy levels given by
the eigenvalues E_n. The respective wave functions are simply the eigenvectors
of H, i.e., vectors ψ_n(r⃗) ∈ for r⃗=(x,y,z). With N = M^3 sites, the quantum mechanical (stationary) Schrödinger equation is equivalent to the eigenvalue equation
H ψ_n = E_n ψ_n, which in site representation reads as

    ∑_σ=±ψ_n(r⃗+σa⃗) + ψ_n(r⃗+σb⃗) + ψ_n(r⃗+σc⃗)                
        = [ E_n - ε(r⃗) ] ψ_n(r⃗),

with a⃗=(1,0,0), b⃗=(0,1,0) and c⃗=(0,0,1) denoting the lattice vectors of a periodic, simple cubic lattice. 
The disorder usually<cit.> enters the matrix on the diagonal, where the entries ε_n(r⃗) correspond to a spatially
varying disorder potential and are selected randomly according to a suitable distribution.<cit.> Here, we shall use the standard box distribution ε(r⃗) ∈ [-w/2,w/2] such that w parameterizes the aforementioned disorder strength. 


For disorders w ≪ w_c, most of the eigenvectors are extended, i.e., ψ_n(r⃗) fluctuating from site to site, but the envelope
|ψ_n| is approximately a nonzero constant. For large disorders w > w_c, all eigenvectors
are localized such that the envelope |ψ_n| of the nth eigenstate may be approximately
written as ∼exp[ -|r⃗ - r⃗_n|/ξ(w) ] with ξ(w) denoting the localization length of the eigenstate. 

Directly at w = w_c, the last extended states at E=0 vanish. 
The wave function vector ψ_E=0(r⃗) appears simultaneously extended and localized and has multifractal properties <cit.>. 
In Fig. <ref>, we show examples of such states. 

In order to numerically distinguish the two (or three) phases mentioned before, one usually needs to (i) go to rather large system sizes of order N^3=10^6 to 10^8 and (ii) average over many different realizations of the disorder, i.e., compute eigenvalues or eigenvectors for many matrices with different diagonals.<cit.>

In the present work, we concentrate on the computation of a few eigenvalues and corresponding eigenvectors for the physically most interesting case around the critical disorder w_c and in the center of the spectrum σ(H), i.e., at E = 0, for large system sizes.








 §.§ Hamiltonian eigenfunctions as data


The square-normalized eigenstates ψ_n= ∑_x,y,zψ_n(x,y,z) |x,y,z⟩ have been numerically obtained using the Jadamilu library <cit.>. The |x,y,z⟩ indicate the orthonormal Wannier basis in the usual tight-binding formulation.
For the 17 disorders w = 15, 15.25, …, 16, 16.2, …, 17, 17.25, … 18 we consider for training and validation a previously used dataset<cit.> with 5000 disorder realization for each disorder and system sizes N= 20^3, 30^3, …, 100^3. 

For all the data,<cit.> we have considered a single eigenstate per sample (disorder realization) with energy close to E = 0. 
This is costly in terms of computing time but essential to avoid the noticable correlations that exist between eigenstates of the same sample.<cit.> 
In addition, we have generated, for each of the disorders, 500 independent test wave functions at E=0, i.e., using random numbers with different seeds. 




In order to be able to use standard 2D image recognition machine learning tools, we represent the ψ_n graphically as in Fig. <ref>. We remove the black box and the color scale before using the images for training, validation and testing purposes. Furthermore, the images are converted from their original postscript, using the ImageMagick set of routines, and rendered as portable network graphics (PNG) in the pixel resolutions of s=100× 100, 200× 200 and 500× 500. This conversion results in some changes in the visual presentation as shown in Fig. <ref>.




 §.§ ML models and results





Previous ML studies of the Anderson model use CNNs composed of 6 convolutional layers and a fully connected layer to identify the extended and localized phases from the |ψ(x,y,z)|^2 <cit.>.
Here, our goal is to expand on these results show that a ResNet18, as used in section <ref>, can also recover the value of w used in images made from these |ψ|^2.






We first establish the capacity of the ResNet18 to identify the two phases of the 3D Anderson model of localisation from images (not shown here). Here, we want to train a network to identify individual disorder values. Following a similar strategy as in section <ref>, we train our network for 17 disorder values w= 15, 15.25, …, 16, 16.2, …, 17, 17.25, … 18 for fixed N=20^3, 40^3, 100^3 and s=100^2. 
After training the 17 disorder values for N=20^3, we obtain a min_ϵ[⟨ l_c,val⟩]=2.408 ± 0.003 (corresponding to an accuracy of max_ϵ[⟨ a_c,val⟩] =15.9%± 0.2). 
At first, the performance of the network on this system appears to be rather limited. From the confusion matrix obtained after training (not shown), we notice that only the smallest and largest disorders, i.e., w=15 and w=18, are perfectly classified. 

We increase the size of the system and train our network for N=40^3. Following the training we reach min_ϵ[⟨ l_c,val⟩]=1.951 ± 0.004 (corresponding to an accuracy of max_ϵ[⟨ a_c,val⟩] =25.7%± 0.2). Looking at the metrics in Fig. <ref> (b), we notice the decrease of ⟨ l_c,val⟩ and ⟨ l_c,val⟩ between the training for N=20^3 and N=40^3. 
Still, the apparent improvement in the performance of the network is not yet convincing. 
We finally train for N=100^3 and s=100 × 100. We obtain min_ϵ[⟨ l_c,val⟩]=1.327 ± 0.006 (corresponding to an accuracy of max_ϵ[⟨ a_c,val⟩] =43.3%± 0.3). 
This is an increase of almost 18%. Even though the accuracy is still less than 50%, the network seems to be getting better at recognizing the w values. The confusion matrix obtained after this training is given in Fig. <ref>(a). Clearly, the matrix is heavily diagonally dominant: misclassifications appear to exist mostly between directly adjacent disorder values. Thus, while the training does not result in a perfect recognition of w's, it is nevertheless already very good in recognizing the vicinity of each w, even very close to the metal-insulator transition.



Increasing the size of the input images to s=200 × 200 does not help to provide significant improvement. After training we obtain  min_ϵ[⟨ l_c,val⟩]=1.216 ± 0.003 (corresponding to an accuracy of max_ϵ[⟨ a_c,val⟩] =47.9%± 0.2). Furthermore, training for such a large input leads to a substantial increase in training time.

In summary, we find that even using images of eigenstates allows to distinguish the phase of the 3D Anderson model well, while the classification of w values proceeds with nearly the same accuracy as in the case of classifying p for percolation in section <ref>. Furthermore, increasing the system size from N=20^3 to 100^3 improves the predictions considerably. Such finite-size effects remind us rather reassuringly that the ML strategies are obviously subject to the same physics constraints as standard approaches.






§ PREDICTING PHASES OF THE J_1-J_2 ISING MODEL WITH VAES




The J_1-J_2 Ising model serves as a still relatively simple system to illustrate an already more complex 3-phase behavior.<cit.>.

With J_1 denoting the nearest-neighbor interaction, the competing second-neighbor interaction J_2 gradually suppresses the ordering temperature, until it vanishes completely when J_2=|J_1|/2 <cit.>. Furthermore, beyond this point, a new ordered “superantiferromagnetic phase” appears.
The universality class of the transition into the superantiferromagnetic phase has been investigated early on  
<cit.>, but still continues to attract attention since its nature remains controversial.<cit.>
There is at least also one investigation of this model on the D-wave quantum annealer <cit.>
and a small number of machine-learning investigations
<cit.>.







 §.§ Definition of the J_1-J_2 Ising model


The Hamiltonian of the J_1-J_2 Ising model can be expressed as 


    H_J_1J_2 = 
    -J_1  ∑_⟨ i,j ⟩ s_i  s_j + 
    J_2  ∑_⟨⟨ i,j ⟩⟩ s_i  s_j  ,


where s_i represents the spin at site i, which can be either up (+1) or down (-1); ⟨ i,j ⟩ refers to nearest-neighbor pairs, ⟨⟨ i,j ⟩⟩ denotes next-nearest neighbor pairs, while J_1, J_2 ≥ 0  signify the interaction strengths between the nearest and next-nearest neighbors, respectively.
Our chosen sign conventions in Eq. (<ref>) lead to a ferromagnetic coupling for J_1 pairs while next-nearest neighbors prefer to align in an antiferromagnetic structure.<cit.>

The three distinct phases of the model correspond to (i) a low-temperature, low-J_2 ferromagnet, (ii) a low-temperature, high-J_2 superantiferromagnet and (iii) the high-temperature paramagnet. We illustrate spin configurations representative of these phases and close-to-phase transitions in Fig. <ref>.

In this work, we review recent work aiming to predict the three phases with a generative VAE, using a spin-adapted mean-squared error ε as ML cost function.<cit.> 




 §.§ Generating states as ML training data via the Metropolis Monte-Carlo approach



To generate the necessary input data for the training of the VAE, we utilize the Metropolis algorithm, a well-established method for simulating statistical models at finite temperature <cit.>.

In the present investigation, we initially focus on a system size of 30×30 with periodic boundary conditions <cit.>. In order to assess the influence of the size of the system, we also investigate  60×60 and 120×120 square lattices.

Equilibration of the model can be difficult, in particular in the regime of
J_2 ≈|J_1|/2 <cit.>. We assure proper thermalization by successively cooling our configurations for fixed J_2/|J_1|.<cit.> 

We set the energy scale with J_1=1.

For J_2=0, we are back to the nearest-neighbor Ising model with known critical temperature T_c,Ising≈ 2.269 <cit.>. We can therefore confidently start our exploration of the as-yet unknown phase diagram by choosing an initial temperature range of 0 ≤ T ≤ 4 ≈ 2 × T_c,Ising containing T_c,Ising. 

We also know that the ferromagnetic-to-superantiferromagnetic transition is at J_2=1/2.<cit.> Hence we choose a range for J_2 from 0 to 1.5. Should we later see that these ranges do not suffice to capture all phases, we could further increase the maximal T and J_2 values.


Using Δ T = 0.025, we thus proceed with a set T of |𝒯|=157 temperatures with T ∈ [0.1,4] for T ∈ T. The Monte-Carlo construction is repeated with different random numbers until we have C=40 configurations for each temperature at the given values of J_2. Let J_2 = {
0, 0.1, 0.2, 0.3, 0.4, 0.45, 0.48, 0.49, 0.495, 0.5, 0.505,
0.51, 0.52, 0.55, 0.6, 0.65, 0.7, 0.8, 0.9, 1, 1.2, 1.5 
} denote the |𝒥_2|=22 chosen distinct values. 
In total, this results in a dataset containing |𝒯|× |𝒥_2| × C = 157 × 22 × 40 = 138 160 independent configurations for a given system size. 












 §.§ Reconstruction of the phase diagram using single-region VAEs




We can now use the VAE architecture to identify the phases of the J_1-J_2 model as a function of T and J_2 for constant J_1=1. Details of the VAE implementation can be found elsewhere.<cit.>




We start the training of the VAE for T ≪ T_c,Ising in two distinct regions, namely (i) J_2 < 1/2 and (ii) J_2 > 1/2. Consequently, we have two restricted training data regions
ρ_low-J_2 and ρ_high-J_2. In order to have a reasonable amount of training data, we use all 40 values for each (T, J_2) in each training region. For the results underlying  Fig. <ref>(a), this amounts to 1440 training configurations in ρ_low-J_2, while for  Fig. <ref>(b), we have 1800 configurations in ρ_high-J_2. 





From  Fig. <ref> we see that indeed two distinct regions emerge. The low-T, low-J_2 region shown in (a) is clearly separated from the rest of the (T, J_2) plane. Similarly, panel (b) establishes a low-T, high-J_2 region.
We note that in both cases, the ε values in the low/high-J_2 regions are close to zero, while in the other regions we have ε≈ 0.5.
This value suggests that in both cases, the out-of-region configurations have about 50% of spins different, in agreement with the behavior in the known phases.
We can therefore conclude the existence of two low-T regions identified in Fig. <ref>. By exclusion, the third region corresponds to ε≈ 0.5 from both trainings.

Indeed, these regions agree very well with the previously established phase boundaries shown in Fig. <ref>. The ε values of 0, 0.25, and 0.5 indicate best, random, and worst reconstruction possible, respectively, compatible with the spin configurations in each phase. 
Clearly, the regions with ε≈ 0 correspond to the ordered ferro- and superantiferromagnetic phases in Fig. <ref> (a) and (b), respectively.

Further results with similar ML strategies as well as on a direct comparison of states can be found elsewhere.<cit.>






§ MICROSCOPY WITH GANS




Convergent-beam electron diffraction (CBED) <cit.> is a transmission electron microscopy (TEM) technique with unparallel sensitivity <cit.>. Its origins date back nearly 100 years to pioneering work<cit.> and its modern applications include crystal symmetry classification <cit.>, lattice parameter determination <cit.>, strain & defect analysis <cit.>, and more <cit.>. 

However, CBED sees the majority of its use in symmetry determination <cit.> and charge density refinement <cit.> and is still lacking in popularity when compared to the more established structure solution and refinement methods of X-ray and neutron diffraction  <cit.>.

Collecting the necessary amount of high-quality diffraction data from a TEM, to construct a large-angle CBED (LACBED) image, is one of the inherent challenges of the method. Here, modern computer-controlled TEM setups offer a clear advantage and can make the task near automatic <cit.>.

A perhaps even more constraining challenge lies in the fact that the complexity introduced by multiple scattering of electrons as they propagate through the specimen <cit.> requires sophisticated modelling techniques to construct the theoretical predictions to compare with TEM results.

To make CBED a more accessible approach, there have been two major computational methods developed: (i) the Bloch-wave method <cit.>, and (ii) Multislice <cit.>. Whilst both have seen success in accurately generating CBED patterns, they even today remain computationally resource- and time-intensive, often well beyond what a standard desktop computer can provide <cit.>.




 §.§ Selection of training data




For our aim to generate LACBED patterns via ML, we require a large body of data in which crystal structure information has been paired with corresponding bright field LACBED images. Experience from previous such machine learning tasks in computer vision <cit.> and related applications <cit.>, as well as in the previous sections, suggests that often more than 10,000 such training pairs are needed. On the scale necessary for a successful model, it is infeasible to use experimental data for such patterns. 

Fortunately, the Inorganic Crystal Structure Database (ICSD) <cit.>, as the world’s largest such database, provides ready access to the full structural information for more than 240,000 crystals in the form of a `Crystallographic Information File' (CIF), a standard text file format <cit.>.





Direct training with structured textual data, as available in the CIFs, is still a major challenge for machine learning tasks <cit.>. Even small changes in, e.g., numerical values of the lattice parameters, can have major changes in the resulting CBED patterns. 

On the other hand, existing image-to-image translation tasks have been primarily optimised for 2D data. To harness this knowledge, we need a feasible way to represent the CIF information in 2D image form as well. 

Fortunately, the projected electron potential ρ is a convenient such image representation. Since we have decided to concern ourselves only with cubic (isometric) crystals, we can nicely project the electron potential along z to a 2D image as shown in Fig. <ref> below.
Using the structure factors F(𝐠) of the crystal, obtained from Felix <cit.>, we generate the projected potential using 
ρ (𝐫) ∝
∑_g F (𝐠) ·exp[-2π i 𝐠·𝐫]
.   
Here, the 𝐠 are the lattice vectors of the unit cell in reciprocal space. 
We normalize the resulting image of electron potential strength and also restrict their size to 128× 128. 
Note that in requiring all of these inputs to be the same image dimensions for the machine learning model, we lose information regarding the size of the crystal.

Next, we need a method to construct the LACBED patterns corresponding to each CIF and projected electron potential ρ. We employ Felix, an open-source software implementation of the Bloch-wave method<cit.> for generating LACBED images originally developed in part by two of us <cit.>.

Felix has been shown to provide atomic coordinate refinements with sub-picometer accuracy <cit.>, and can accurately simulate LACBED patterns<cit.>.

The software takes as input a CIF, beam parameters, microscope settings, crystal settings, and the desired beam direction. Most of these values were calibrated previously<cit.>. 

In our simulations, we only consider the (0,0,0) beam direction for simplicity. The other simulation parameters used here are provided in the code accompanying the present work <cit.>. We use Felix to generate LACBED images of size 128× 128. Such sizes are sufficient for many computer-vision-based machine learning tasks <cit.>, whilst remaining small enough to allow the generation of results on a large scale for our dataset.

Our strategy in generating the necessary input from the information provided in the ICSD is then as follows: (i) We convert the textual information provided by each crystal's CIF into the normalized projected electronic potential, i.e., a 2D image. (ii) we compute, via the Bloch-wave code Felix, the corresponding bright-field LACBED images.

While the construction of the electron potential images is very fast, generating the LACBED dataset takes a few weeks using bespoke high-performance computing architecture. 
In the current proof-of-principle work, we focus on the 12454 CIFs each corresponding to a unique cubic crystal.
We ultimately have a dataset of 12454 image pairs of size 128× 128, with pixel values between 0 and 255. Each pair consists of a crystal's projected electron potential and its simulated (0,0,0) LACBED diffraction pattern. This dataset is publicly hosted <cit.>.

Whilst we use as many cubic crystals as we can, since after all, a much higher number is still desired, we encounter significantly imbalanced data in many areas. For example, when resolved according to their space group classification, we find that the ICSD data is highly imbalanced. As shown in Fig. <ref>, some space groups contain less than 10 ICSD entries while others have many thousands. 

It is well known that machine learning methods, and in particular our chosen adversarial network architecture, suffer in their predictive strength when using imbalanced data <cit.>. Hence it could be worthwhile in future studies to include other crystal data from the ICSD beyond the cubic ones.




 §.§ Results


We train a cGAN to create LACBED patterns by providing the projected electron density as input.<cit.> In Fig. <ref> we show some results.



Before going into details on how we create these images, we start by noting that the ground truth LACBED images shown in the figure, with whom we compare our predictions, needed about 400 seconds each to be constructed by the Bloch-wave method on a high-performance compute cluster while our predicted LACBED images arrived within 20 milliseconds on a modern, i.e., GPU-supported, desktop. 

The images given in Fig. <ref> show, in the left column, the computed project potential for three crystal structures from different space groups, namely ,  and  from top to bottom. Each such projected potential has been normalized. The comparison between the CBED predictions of the cGAN and the expected behavior from the Felix results show an overall good agreement, in particular w.r.t. the underlying two-fold symmetries. While we use the standard mean-squared error in training the cGAN, in Fig. <ref>, we report a shifted zero-mean normalized cross-correlation ﬁt index R for pixel intensities, 

    R(𝐲, 𝐲̂) = 1/2 + 1/2 n^2∑_i,j^ny_ij-⟨𝐲⟩/σ(𝐲)·ŷ_ij-⟨𝐲̂⟩/σ(𝐲̂) ,

which has often been used in CBED image comparison studies <cit.>. 
In the normalization used here, the value R=1 corresponds to a perfect fit, while R=0 is perfectly anti-correlated. The values R=0.5 and 0.75 emerge when 1/2 or 3/4 of the image pixels are correlated and 1/2 and 1/4 are anti-correlated, respectively. Also, R=0.5 corresponds to two images with uncorrelated intensities. 
Our resulting R values for the LACBED image reconstruction, as given in the caption of Fig. <ref>, indicate an overall very good agreement.






§ CONCLUSIONS AND OUTLOOK





The learning aspects of DL networks are often referred to as “black boxes”, highlighting that it appears occasionally surprising how a DNN arrives at its classification, regression or generative predictions. On the other hand, it is exactly this lack of apriori imposed basic descriptors that allows a DL architecture to variationally construct its own set of descriptors to achieve an optimal prediction.


So when ML succeeds in classifying states of Ising-type, percolation, and Anderson models, this also shows that the phase information must be encapsulated directly in the states alone, even for those relatively close to the phase boundaries as shown by the overall good reconstruction of these phases. While this was not unknown before or unexpected,<cit.> it is nevertheless an interesting qualitative insight to have re-emphasized. 

Conversely, this also suggests that simply comparing states with each other, by mean-squared deviations, R correlation or otherwise, might also be an alternative quantitative method for phase diagram construction - as already demonstrated.<cit.>


The caveat discovered when studying the globally spanning cluster for the percolation problem with locally focused CNNs, i.e., the failure of such CNNs to correctly identify the percolating cluster,<cit.> furthermore suggests that even the power of modern ML approaches can fail when the underlying physics is ignored.<cit.>

In this context it is also important to mention that the cGAN predictions for the outcome of electron interference experiments, i.e., the LACBED intensities, do not somehow circumvent the quantum mechanical measurement problem. Rather, they simply provide a good interpolation to the various diffraction solutions of the electron dynamical scattering problem provided by the Bloch wave calculations the cGAN was trained on. 


Last, the review given here clearly reflects the prejudices and preferences of its authors in selecting the applications of ML to physics. Many other applications and application areas have been ignored such as Boltzmann machines<cit.> and the extremely interesting approaches to finding states of many-body systems<cit.>.






  
Acknowledgments J.J.W. and R.A.R. gratefully acknowledge discussions with Richard Beanland, Warwick, on the results presented here for the LACBED reconstructions. This work forms part of an ongoing project of all three of us.
D.B., A.H., and R.A.R. are grateful for co-tutelle funding via the EUtopia Alliance. R.A.R. also very grateful acknowledges the CY Initiative of Excellence (grant “Investissements d’Avenir” ANR-16-IDEX-0008).
The computations presented here used the University of Warwick's Research Technology Platform (RTP Scientific Computing) and the Sulis Tier 2 HPC platform hosted by the RTP. Sulis is funded by EPSRC Grant EP/T022108/1 and the HPC Midlands+ consortium. 














Djena Bayograduated from the Sorbonne Université in 2017. Following this, she obtained a Master's degree from CY Cergy Paris Université in 2019. As part of the Master's program, she did an internship under the supervision of Vita Ilakovac on “Calculation of vibrations in core excited molecules" at the Laboratoire de Chimie Physique-Matière et Rayonnement (Sorbonne Université). In  2019, she joined Römer at the University of Warwick as a co-tutelle PhD student with Honecker at CY Cergy Paris Université. She defended her PhD thesis<cit.> in April 2024.

Burak Çivitcioğluwas born in Antalya, Türkiye. After a first degree at  Koç University / Istanbul, he obtained his Master degrees from the Université Paris Diderot and CY Cergy Paris Université. In 2020, he started a PhD there with Honecker, informally co-supervised by Römer. He is about to defend his PhD thesis.<cit.>

Joe Webb
was born in Essex in the UK.
He recently received a BSc in Mathematics and Physics from the University of Warwick.
There he founded and published two editions of the Poincaré student magazine and collaborated with Römer on ML applications to LACBED patterns.
He is currently a scholar at Worcester College, Oxford,
studying for an MSc, with plans to pursue a PhD.


Andreas Honeckerwas born in Tübingen, Germany in 1967 and obtained first his diploma and then in 1995 his PhD degree in physics from the University of Bonn under the supervision of Werner Nahm and Günter von Gehlen, respectively. He went on his post-doctoral journey to the FU Berlin, SISSA Trieste, and ETH Zürich. After that, he worked at the TU Braunschweig where he obtained his Habilitation in 2003 under the direction of Wolfram Brenig. 
He continued on to the Georg-August-Universität Göttingen, with partial support via a Heisenberg fellowship,
where he was awarded a professor title (apl.). He held a number of further fixed-term professorial appointments at Hannover, Strasbourg, and Lyon before he became full professor at CY Cergy Paris Université, France in 2014. Beyond broad scientific interests in statistical physics and condensed matter theory, he also contributes to administrative obligations. In particular, he served as director of the physics department and is presently serving as adjoint director of the Institut des Sciences et Techniques of CY Cergy Paris Université. He proudly supports new publishing initiatives such as SciPost Physics as editor.

Rudo Römerwas born in Gedern, Hessen, Germany in 1966. He attended the Wolfgang-Ernst Gymnasium in Büdingen, 
studied with Robert Schrader for a Dipl.-Phys. at the FU Berlin, obtained a PhD with Bill Sutherland at the University of Utah in 1994, postdoc'ed as Feodor-Lynen fellow with Sriram Shastry at the IISc in Bangalore, Dieter Vollhardt at the RWTH Aachen, and achieved his Habilitation qualification with Michael Schreiber at the TU Chemnitz in 2000. In 2002 he was appointed at the University of Warwick, UK, where he heads the disordered quantum systems research group. Since then he has spent research stays at the MPIPKS in Dresden, the ICCMP at UN Brasilia (now in Natal), the Chinese Academy of Sciences at the Wuhan Institute of Physics and Mathematics, was appointed Lotus (Fu Rong) Visiting Professor at Xiangtan University, Xiangtan, Hunan province, China, and as Senior Fellow at CY Cergy Paris Université. He is on a number of editorial and advisory boards and editor-in-chief for Physica E.






















100

Ashcroft1976SolidPhysics
N. W. Ashcroft and N. D. Mermin, Solid State Physics.
Saunders College Publishing, Fort Worth, 1976.

Diep2013FrustratedSystems
H. T. Diep, https://doi.org/10.1142/11660 Frustrated Spin Systems.
World Scientific, 2020.
https://doi.org/10.1142/11660.

Anderson1972MoreDifferent
P. W. Anderson, “More Is Different,” https://dx.doi.org/10.1126/science.177.4047.393Science 177 no. 4047, (1972) 393–396. https://www.science.org/doi/abs/10.1126/science.177.4047.393.

Yu2016DimensionlessTransitions
Y.-C. Yu, Y.-Y. Chen, H.-Q. Lin, R. A. Römer, and X.-W. Guan, “Dimensionless ratios: Characteristics of quantum liquids and their phase transitions,” https://dx.doi.org/10.1103/PhysRevB.94.195129Phys. Rev. B 94 no. 19, (11, 2016) 195129. https://link.aps.org/doi/10.1103/PhysRevB.94.195129.

Wang2024DistinguishingStatistics
C.-Y. Wang, T.-G. Zhou, Y.-N. Zhou, and P. Zhang, “Distinguishing Quantum Phases through Cusps in Full Counting Statistics,” https://dx.doi.org/10.1103/PhysRevLett.133.083402Phys. Rev. Lett. 133 no. 8, (8, 2024) 083402. https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.133.083402.

Gomez2019AProblems
H. Gomez, M. Bures, and A. Moure, “A review on computational modelling of phase-transition problems,” https://dx.doi.org/10.1098/rsta.2018.0203Phil. Trans. R. Soc. A 377 no. 2143, (2019) 20180203.

Alpaydin2020IntroductionEdition
E. Alpaydin, Introduction to Machine Learning, fourth edition.
Adaptive Computation and Machine Learning series. MIT Press, 2020.
https://books.google.fr/books?id=uZnSDwAAQBAJ.

Hinton1999UnsupervisedComputation
G. Hinton and T. J. Sejnowski, https://dx.doi.org/10.7551/mitpress/7011.001.0001Unsupervised Learning: Foundations of Neural Computation.
The MIT Press, 8, 1999.
https://doi.org/10.7551/mitpress/7011.001.0001.

Mehta2019APhysicists
P. Mehta, M. Bukov, C.-H. Wang, A. G. Day, C. Richardson, C. K. Fisher, and D. J. Schwab, “A high-bias, low-variance introduction to Machine Learning for physicists,” https://dx.doi.org/10.1016/j.physrep.2019.03.001Phys. Rep. 810 (5, 2019) 1–124. https://linkinghub.elsevier.com/retrieve/pii/S0370157319300766.

Carleo2019MachineSciences
G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborová, “Machine learning and the physical sciences,” https://dx.doi.org/10.1103/RevModPhys.91.045002Rev. Mod. Phys. 91 no. 4, (12, 2019) 045002. https://link.aps.org/doi/10.1103/RevModPhys.91.045002.

Schurov2024InvariantApplications
I. Schurov, D. Alforov, M. Katsnelson, A. Bagrov, and A. Itin, “Invariant multiscale neural networks for data-scarce scientific applications,”. <https://arxiv.org/abs/2406.08318v1>

He2016a
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” https://dx.doi.org/10.1109/CVPR.2016.90Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2016-Decem (2016) 770–778

Tabak2019
M. A. Tabak, M. S. Norouzzadeh, et al., “Machine learning to classify animal species in camera trap images: Applications in ecology,” https://dx.doi.org/10.1111/2041-210X.13120Methods in Ecology and Evolution 10 no. 4, (2019) 585–590. https://onlinelibrary.wiley.com/doi/10.1111/2041-210X.13120

Zhang2017CombinationRecognition
R. Zhang, Q. Wang, and Y. Lu, https://dx.doi.org/10.1109/ICDAR.2017.324“Combination of ResNet and Center Loss Based Metric Learning for Handwritten Chinese Character Recognition,” in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 25–29.
IEEE, 2017.
http://ieeexplore.ieee.org/document/8270270/

Zhao2020
Y. Zhao and M. Khushi, https://dx.doi.org/10.1109/ICDMW51313.2020.00060“Wavelet Denoised-ResNet CNN and LightGBM Method to Predict Forex Rate of Change,” in 2020 International Conference on Data Mining Workshops (ICDMW), vol. 2020-Novem, pp. 385–391.
IEEE, 11, 2020.
https://ieeexplore.ieee.org/document/9346446/

Carrasquilla2017
J. Carrasquilla and R. G. Melko, “Machine learning phases of matter,” https://dx.doi.org/10.1038/nphys4035Nat. Phys. 13 no. 5, (2017) 431–434. https://www.nature.com/articles/nphys4035.pdf

ChNg2018
K. Ch'ng, N. Vazquez, and E. Khatami, “Unsupervised machine learning account of magnetic transitions in the Hubbard model,” https://dx.doi.org/10.1103/PhysRevE.97.013306Phys. Rev. E 97 no. 1, (1, 2018) 013306. https://link.aps.org/doi/10.1103/PhysRevE.97.013306

Tanaka2017DetectionNetworks
A. Tanaka and A. Tomiya, “Detection of Phase Transition via Convolutional Neural Networks,” https://dx.doi.org/10.7566/jpsj.86.063001J. Phys. Soc. Jpn. 86 no. 6, (6, 2017) 63001. http://doi.org/10.7566/JPSJ.86.063001

Huembeli2018a
P. Huembeli, A. Dauphin, P. Wittek, and C. Gogolin, “Automated discovery of characteristic features of phase transitions in many-body localization,” https://dx.doi.org/10.1103/PhysRevB.99.104106Phys. Rev. B 99 no. 10, (3, 2019) 104106. https://link.aps.org/doi/10.1103/PhysRevB.99.104106

Dong2019MachineTransitions
X.-Y. Dong, F. Pollmann, and X.-F. Zhang, “Machine learning of quantum phase transitions,” https://dx.doi.org/10.1103/PhysRevB.99.121104Phys. Rev. B 99 no. 12, (3, 2019) 121104. https://link.aps.org/doi/10.1103/PhysRevB.99.121104

Canabarro2019UnveilingLearning
A. Canabarro, F. F. Fanchini, A. L. Malvezzi, R. Pereira, and R. Chaves, “Unveiling phase transitions with machine learning,” https://dx.doi.org/10.1103/PhysRevB.100.045129Phys. Rev. B 100 no. 4, (7, 2019) 045129. https://link.aps.org/doi/10.1103/PhysRevB.100.045129

Shinjo2019MachineModel
K. Shinjo, K. Sasaki, S. Hase, S. Sota, S. Ejima, S. Yunoki, and T. Tohyama, “Machine Learning Phase Diagram in the Half-filled One-dimensional Extended Hubbard Model,” https://dx.doi.org/10.7566/JPSJ.88.065001J. Phys. Soc. Jpn. 88 no. 6, (6, 2019) 065001. https://journals.jps.jp/doi/10.7566/JPSJ.88.065001

Wang2016DiscoveringLearning
L. Wang, “Discovering phase transitions with unsupervised learning,” https://dx.doi.org/10.1103/PhysRevB.94.195105Phys. Rev. B 94 no. 19, (11, 2016) 195105. https://link.aps.org/doi/10.1103/PhysRevB.94.195105

Kottmann2020UnsupervisedDetection
K. Kottmann, P. Huembeli, M. Lewenstein, and A. Acín, “Unsupervised Phase Discovery with Deep Anomaly Detection,” https://dx.doi.org/10.1103/PhysRevLett.125.170603Phys. Rev. Lett. 125 no. 17, (10, 2020) 170603. https://link.aps.org/doi/10.1103/PhysRevLett.125.170603

Alexandrou2020TheAutoencoders
C. Alexandrou, A. Athenodorou, C. Chrysostomou, and S. Paul, “The critical temperature of the 2D-Ising model through deep learning autoencoders,” https://dx.doi.org/10.1140/epjb/e2020-100506-5Eur. Phys. J. B 93 no. 12, (12, 2020) 226. https://doi.org/10.1140/epjb/e2020-100506-5

DAngelo2020LearningNetworks
F. D'Angelo and L. Böttcher, “Learning the Ising model with generative neural networks,” https://dx.doi.org/10.1103/PhysRevResearch.2.023266Phys. Rev. Res. 2 no. 2, (6, 2020) 023266. https://link.aps.org/doi/10.1103/PhysRevResearch.2.023266

Shiina2020Machine-LearningModels
K. Shiina, H. Mori, Y. Okabe, and H. K. Lee, “Machine-Learning Studies on Spin Models,” https://dx.doi.org/10.1038/s41598-020-58263-5Sci. Rep. 10 no. 1, (2, 2020) 2177. https://doi.org/10.1038/s41598-020-58263-5

Corte2021ExploringModels
I. Corte, S. Acevedo, M. Arlego, and C. A. Lamas, “Exploring neural network training strategies to determine phase transitions in frustrated magnetic models,” https://dx.doi.org/https://doi.org/10.1016/j.commatsci.2021.110702Comput. Mater. Sci. 198 (2021) 110702. https://www.sciencedirect.com/science/article/pii/S0927025621004298

Walker2018IdentifyingMethods
N. Walker, K.-M. Tam, B. Novak, and M. Jarrell, “Identifying structural changes with unsupervised machine learning methods,” https://dx.doi.org/10.1103/PhysRevE.98.053305Phys. Rev. E 98 (11, 2018) 053305. https://doi.org/10.1103/PhysRevE.98.053305

Morningstar2018DeepCriticality
A. Morningstar and R. G. Melko, “Deep Learning the Ising Model Near Criticality,” J. Mach. Learn. Res. 18 no. 163, (2018) 1–17. http://jmlr.org/papers/v18/17-527.html

Suchsland2018
P. Suchsland and S. Wessel, “Parameter diagnostics of phases and phase transition learning by neural networks,” https://dx.doi.org/10.1103/PhysRevB.97.174435Phys. Rev. B 97 no. 17, (2018) 1–13

Efthymiou2019Super-resolvingNetworks
S. Efthymiou, M. J. S. Beach, and R. G. Melko, “Super-resolving the Ising model with convolutional neural networks,” https://dx.doi.org/10.1103/PhysRevB.99.075113Phys. Rev. B 99 no. 7, (2, 2019) 075113. https://link.aps.org/doi/10.1103/PhysRevB.99.075113

Walker2020DeepAutoencoder
N. Walker, K.-M. Tam, and M. Jarrell, “Deep learning on the 2-dimensional Ising model to extract the crossover region with a variational autoencoder,” https://dx.doi.org/10.1038/s41598-020-69848-5Sci. Rep. 10 no. 1, (8, 2020) 13047. https://doi.org/10.1038/s41598-020-69848-5

Goel2020LearningVariables
S. Goel, “Learning Ising and Potts Models with Latent Variables,” in Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, S. Chiappa and R. Calandra, eds., vol. 108 of Proceedings of Machine Learning Research, pp. 3557–3566.
PMLR, 8, 2020.
<https://proceedings.mlr.press/v108/goel20a.html>

PhysRevResearch.3.013074
J. Wang, W. Zhang, T. Hua, and T.-C. Wei, “Unsupervised learning of topological phase transitions using the Calinski-Harabaz index,” https://dx.doi.org/10.1103/PhysRevResearch.3.013074Phys. Rev. Res. 3 (Jan, 2021) 013074. https://link.aps.org/doi/10.1103/PhysRevResearch.3.013074

PhysRevResearch.3.033052
J. Arnold, F. Schäfer, M. Žonda, and A. U. J. Lode, “Interpretable and unsupervised phase classification,” https://dx.doi.org/10.1103/PhysRevResearch.3.033052Phys. Rev. Res. 3 (Jul, 2021) 033052. https://link.aps.org/doi/10.1103/PhysRevResearch.3.033052

Civitcioglu2022MachineModel
B. Çivitcioğlu, R. A. Römer, and A. Honecker, “Machine Learning the Square-Lattice Ising Model,” https://dx.doi.org/10.1088/1742-6596/2207/1/012058J. Phys: Conf. Ser. 2207 no. 1, (3, 2022) 12058. https://doi.org/10.1088/1742-6596/2207/1/012058

Basu2022MachineSpin-shuffling
P. Basu, J. Bhattacharya, D. P. S. Jakka, C. Mosomane, and V. Shukla, “Machine learning of Ising criticality with spin-shuffling,”. <http://arxiv.org/abs/2203.04012>

condmat8030083
D. W. Tola and M. Bekele, “Machine learning of nonequilibrium phase transition in an Ising model on square lattice,” https://dx.doi.org/10.3390/condmat8030083Condensed Matter 8 no. 3, (2023) 83. https://www.mdpi.com/2410-3896/8/3/83

PhysRevE.108.L032102
V. Chertenkov, E. Burovski, and L. Shchur, “Finite-size analysis in neural network classification of critical phenomena,” https://dx.doi.org/10.1103/PhysRevE.108.L032102Phys. Rev. E 108 (Sep, 2023) L032102. https://link.aps.org/doi/10.1103/PhysRevE.108.L032102

Naravane2023Semi-supervisedAutoencoders
A. Naravane and N. Mathur, “Semi-supervised learning of order parameter in 2D Ising and XY models using Conditional Variational Autoencoders,”. <http://arxiv.org/abs/2306.16822>

Pavioni2024MinimalistModels
G. L. G. Pavioni, M. Arlego, and C. A. Lamas, “Minimalist neural networks training for phase classification in diluted Ising models,” https://dx.doi.org/https://doi.org/10.1016/j.commatsci.2024.112792Comput. Mater. Sci. 235 (2024) 112792. https://www.sciencedirect.com/science/article/pii/S0927025624000132

McCoy1973TheModel
B. M. McCoy and T. T. Wu, https://dx.doi.org/10.4159/harvard.9780674180758The Two-Dimensional Ising Model.
Harvard University Press, 12, 1973.
https://www.degruyter.com/document/doi/10.4159/harvard.9780674180758/html

Rzadkowski2020DetectingLearning
W. Rza̧dkowski, N. Defenu, S. Chiacchiera, A. Trombettoni, and G. Bighin, “Detecting composite orders in layered models via machine learning,” https://dx.doi.org/10.1088/1367-2630/abae44New J. Phys. 22 no. 9, (9, 2020) 93026. https://dx.doi.org/10.1088/1367-2630/abae44

Fukushima2021CanModel
K. Fukushima and K. Sakai, “Can a CNN trained on the Ising model detect the phase transition of the q-state Potts model?,” https://dx.doi.org/10.1093/ptep/ptab057Prog. Theor. Exp. Phys. 2021 no. 6, (8, 2021) 061A01. https://doi.org/10.1093/ptep/ptab057

Giataganas2022NeuralModels
D. Giataganas, C.-Y. Huang, and F.-L. Lin, “Neural network flows of low q-state Potts and clock models,” https://dx.doi.org/10.1088/1367-2630/ac63daNew J. Phys. 24 no. 4, (4, 2022) 43040. https://dx.doi.o/Users/honecker/Downloads/IOPEXPORT_BIB.bibrg/10.1088/1367-2630/ac63da

Tirelli2022UnsupervisedModel
A. Tirelli, D. O. Carvalho, L. A. Oliveira, J. P. de Lima, N. C. Costa, and R. R. dos Santos, “Unsupervised machine learning approaches to the q-state Potts model,” https://dx.doi.org/10.1140/epjb/s10051-022-00453-3Eur. Phys. J. B 95 no. 11, (11, 2022) 189. https://doi.org/10.1140/epjb/s10051-022-00453-3

Wu1982TheModel
F. Y. Wu, “The Potts model,” https://dx.doi.org/10.1103/RevModPhys.54.235Rev. Mod. Phys. 54 no. 1, (1, 1982) 235–268. https://link.aps.org/doi/10.1103/RevModPhys.54.235

Zhang2019MachineModels
W. Zhang, J. Liu, and T.-C. Wei, “Machine learning of phase transitions in the percolation and XY models,” https://dx.doi.org/10.1103/PhysRevE.99.032142Phys. Rev. E 99 no. 3, (3, 2019) 032142. https://link.aps.org/doi/10.1103/PhysRevE.99.032142

Yu2020UnsupervisedPercolation
W. Yu and P. Lyu, “Unsupervised machine learning of phase transition in percolation,” https://dx.doi.org/10.1016/J.PHYSA.2020.125065Physica A 559 (12, 2020) 125065.

Shen2021SupervisedPercolation
J. Shen, W. Li, S. Deng, and T. Zhang, “Supervised and unsupervised learning of directed percolation,” https://dx.doi.org/10.1103/PhysRevE.103.052140Phys. Rev. E 103 no. 5, (2021) 052140. http://arxiv.org/abs/2101 http://dx.doi.org/10.1103/PhysRevE.103.052140

Bayo2022MachineModel
D. Bayo, A. Honecker, and R. A. Römer, “Machine learning the 2D percolation model,” https://dx.doi.org/10.1088/1742-6596/2207/1/012057J. Phys.: Conf. Ser.  2207 no. 1, (2022) 12057. https://iopscience.iop.org/article/10.1088/1742-6596/2207/1/012057

Bayo2023TheLearning
D. Bayo, A. Honecker, and R. A. Römer, “The percolating cluster is invisible to image recognition with deep learning,” https://dx.doi.org/10.1088/1367-2630/ad0525New J. Phys. 25 no. 11, (11, 2023) 113041. https://dx.doi.org/10.1088/1367-2630/ad0525

Patwardhan2022MachineNetworks
S. Patwardhan, U. Majumder, A. D. Sarma, M. Pal, D. Dwivedi, and P. K. Panigrahi, “Machine Learning as an Accurate Predictor for Percolation Threshold of Diverse Networks,”. <http://arxiv.org/abs/2212.14694>

Cheng2021MachineModel
S. Cheng, F. He, H. Zhang, K.-D. Zhu, and Y. Shi, “Machine Learning Percolation Model,”. <http://arxiv.org/abs/2101.08928>

Li2009TopologicalInsulator
J. Li, R. L. Chu, J. K. Jain, and S. Q. Shen, “Topological Anderson insulator,” https://dx.doi.org/10.1103/PHYSREVLETT.102.136806/FIGURES/4/MEDIUMPhys. Rev. Lett. 102 no. 13, (3, 2009) 136806. https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.102.136806

Balogh2014DiffusionAlloys
Z. Balogh and G. Schmitz, “Diffusion in Metals and Alloys,” https://dx.doi.org/10.1016/B978-0-444-53770-6.00005-8Physical Metallurgy: Fifth Edition 1 (1, 2014) 387–559.

Oswald2015a
J. Oswald and R. A. Römer, “Imaging of Condensed Quantum States in the Quantum Hall Effect Regime,” https://dx.doi.org/10.1016/j.phpro.2015.12.038Physics Procedia 75 (2015) 314–325. http://linkinghub.elsevier.com/retrieve/pii/S1875389215016776

Oswald2020
J. Oswald and R. A. Römer, “Microscopic details of stripes and bubbles in the quantum Hall regime,” https://dx.doi.org/10.1103/PhysRevB.102.121305Phys. Rev. B 102 no. 12, (9, 2020) 121305. http://arxiv.org/abs/2001.07542 https://link.aps.org/doi/10.1103/PhysRevB.102.121305

Haldane1988ModelAnomaly
F. D. Haldane, “Model for a Quantum Hall Effect without Landau Levels: Condensed-Matter Realization of the "Parity Anomaly",” https://dx.doi.org/10.1103/PhysRevLett.61.2015Phys. Rev. Lett. 61 no. 18, (10, 1988) 2015. https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.61.2015

Yu2010QuantizedInsulators
R. Yu, W. Zhang, H. J. Zhang, S. C. Zhang, X. Dai, and Z. Fang, “Quantized anomalous Hall effect in magnetic topological insulators,” https://dx.doi.org/10.1126/SCIENCE.1187485/SUPPL_FILE/YU.SOM.PDFScience 329 no. 5987, (7, 2010) 61–64. https://www.science.org/doi/10.1126/science.1187485

Pixley2015
J. Pixley, P. Goswami, and S. Das Sarma, “Anderson Localization and the Quantum Phase Diagram of Three Dimensional Disordered Dirac Semimetals,” https://dx.doi.org/10.1103/PhysRevLett.115.076601Phys. Rev. Lett. 115 no. 7, (2015) 076601. http://link.aps.org/doi/10.1103/PhysRevLett.115.076601

Meng2019
L. Meng, J. Wu, J. Zhong, and R. A. Römer, “A type of robust superlattice type-I Weyl semimetal with four Weyl nodes,” https://dx.doi.org/10.1039/C9NR04551ANanoscale 11 no. 39, (2019) 18358–18366. http://xlink.rsc.org/?DOI=C9NR04551A

Schulz-Baldes2020
    E. Prodan, H. Schulz-Baldes, “Bulk and Boundary Invariants for Complex Topological Insulators: From K-Theory to Physics,” https://doi.org/10.1007/978-3-319-29351-6Mathematical Physics Studies Part F1117, 1–201 (2015).

Hasan2010
M. Z. Hasan and C. L. Kane, “Topological Insulators,” https://dx.doi.org/10.1103/RevModPhys.82.3045Physics 82 no. 4, (2010) 23. http://arxiv.org/abs/1002.3895

Rotenberg2011
E. Rotenberg, “Topological insulators: The dirt on topology,” Nat. Phys. 7 no. 1, (2011) 8–10. <http://dx.doi.org/10.1038/nphys1869>

Rodriguez2010
A. Rodriguez, L. J. Vasquez, K. Slevin, and R. A. Römer, “Critical Parameters from a Generalized Multifractal Analysis at the Anderson Transition,” https://dx.doi.org/10.1103/PhysRevLett.105.046403Phys. Rev. Lett. 105 no. 4, (7, 2010) 046403. https://link.aps.org/doi/10.1103/PhysRevLett.105.046403

Rodriguez2011
A. Rodriguez, L. J. Vasquez, K. Slevin, and R. A. Römer, “Multifractal finite-size scaling and universality at the Anderson transition,” https://dx.doi.org/10.1103/PhysRevB.84.134209Phys. Rev. B 84 no. 13, (10, 2011) 134209. https://link.aps.org/doi/10.1103/PhysRevB.84.134209

Ohtsuki2016DeepSystems
T. Ohtsuki and T. Ohtsuki, “Deep Learning the Quantum Phase Transitions in Random Two-Dimensional Electron Systems,” https://dx.doi.org/10.7566/JPSJ.85.123706J. Phys. Soc. Jpn. 85 no. 12, (12, 2016) 123706. https://journals.jps.jp/doi/10.7566/JPSJ.85.123706

Ohtsuki2019DrawingFunctions
T. Ohtsuki and T. Mano, “Drawing Phase Diagrams for Random Quantum Systems by Deep Learning the Wave Functions,”. https://arxiv.org/abs/1909.09821

Wichert2021MachineLearning
A. Wichert and L. Sa-Couto, https://dx.doi.org/10.1142/12201Machine Learning — A Journey to Deep Learning.
World Scientific, 2, 2021.
https://www.worldscientific.com/worldscibooks/10.1142/12201

Isola2017Image-to-imageNetworks
P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” https://dx.doi.org/10.1109/CVPR.2017.632Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 2017-January (11, 2017) 5967–5976

Wang2018High-ResolutionGANs
T. C. Wang, M. Y. Liu, J. Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, “High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs,” https://dx.doi.org/10.1109/CVPR.2018.00917Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (12, 2018) 8798–8807

Kingma2013Auto-EncodingBayes
D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,”. <http://arxiv.org/abs/1312.6114>

Ede2021AdvancesLearning
J. M. Ede, “Advances in Electron Microscopy with Deep Learning,”. http://arxiv.org/abs/2101.01178 http://dx.doi.org/10.5281/zenodo.4399748

Matinyan2023MachineData
S. Matinyan, B. Demir, P. Filipcik, J. P. Abrahams, and E. van Genderen, “Machine learning for classifying narrow-beam electron diffraction data,” https://dx.doi.org/10.1107/S2053273323004680Acta Crystallographica Section A Foundations and Advances 79 no. 4, (7, 2023) 360–368. https://scripts.iucr.org/cgi-bin/paper?S2053273323004680

Friedrich2023PhaseLearning
T. Friedrich, C.-P. Yu, J. Verbeeck, and S. Van Aert, “Phase Object Reconstruction for 4D-STEM using Deep Learning,” https://dx.doi.org/10.1093/micmic/ozac002Microsc. Microanal. 29 no. 1, (2, 2023) 395–407. https://academic.oup.com/mam/article/29/1/395/6985579

Liu2021MachineStructures
X. Liu, K. Amini, A. Sanchez, B. Belsa, T. Steinle, and J. Biegert, “Machine learning for laser-induced electron diffraction imaging of molecular structures,” https://dx.doi.org/10.1038/s42004-021-00594-zCommun. Chem. 4 no. 1, (11, 2021) 154. https://www.nature.com/articles/s42004-021-00594-z

Hall2006SpecificationCIF
S. R. Hall, J. D. Westbrook, N. Spadaccini, I. D. Brown, H. J. Bernstein, and B. McMahon, https://dx.doi.org/10.1107/97809553602060000728“Specification of the Crystallographic Information File (CIF),” in International Tables for Crystallography, pp. 20–36.
International Union of Crystallography, Chester, England, 10, 2006.
https://dx.doi.org/10.1107/97809553602060000728

IgorLevin2018NISTICSD
Igor Levin, “NIST Inorganic Crystal Structure Database (ICSD).” 2018.
<https://icsd.nist.gov/>

Chou2019GeneratedVAE
J. Chou, “Generated Loss and Augmented Training of MNIST VAE,”. <http://arxiv.org/abs/1904.10937>.

IanGoodfellow2016DeepLearning
Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning.
MIT Press, 2016.
<http://www.deeplearningbook.org>

Goodfellow2014GenerativeNets
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” in Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, eds.
Curran Associates, Inc., 27 ed., 6, 2014.

https://proceedings.neurips.cc/paper_files/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf

Kullback1951OnSufficiency
S. Kullback and R. A. Leibler, “On Information and Sufficiency,” https://dx.doi.org/10.1214/aoms/1177729694Ann. Math. Statist. 22 no. 1, (3, 1951) 79–86. http://projecteuclid.org/euclid.aoms/1177729694

Pathak2016ContextInpainting
D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context Encoders: Feature Learning by Inpainting,”. <http://arxiv.org/abs/1604.07379>

Mirza2014ConditionalNets
M. Mirza and S. Osindero, “Conditional Generative Adversarial Nets,”. <http://arxiv.org/abs/1411.1784>.

Broadbent1957PercolationMazes
S. R. Broadbent and J. M. Hammersley, “Percolation processes: I. Crystals and mazes,” https://dx.doi.org/10.1017/S0305004100032680Math. Proc. Camb. Philos. Soc. 53 no. 3, (1957) 629–641. https://www.cambridge.org/core/journals/mathematical-proceedings-of-the-cambridge-philosophical-society/article/abs/percolation-processes/C00CC4943F48228F8AC8031092FE84EC

Stauffer2018
D. Stauffer and A. Aharony, https://dx.doi.org/10.1201/9781315274386Introduction To Percolation Theory.
Taylor & Francis, 12, 2018.
https://www.taylorfrancis.com/books/9781482272376

Elliott1960EquivalenceFerromagnetism
R. J. Elliott, B. R. Heap, D. J. Morgan, and G. S. Rushbrooke, “Equivalence of the Critical Concentrations in the Ising and Heisenberg Models of Ferromagnetism,” https://dx.doi.org/10.1103/PhysRevLett.5.366Phys. Rev. Lett. 5 no. 8, (10, 1960) 366–367. https://link.aps.org/doi/10.1103/PhysRevLett.5.366

Flory1953a
P. J. Flory, Principles of Polymer Chemistry.
Cornell University Press, 1953.
<https://books.google.fr/books/about/Principles_of_Polymer_Chemistry.html?id=CQ0EbEkT5R0C   redir_esc=y>

Derrida1985
B. Derrida and D. Stauffer, “Corrections To Scaling and Phenomenological Renormalization for 2-Dimensional Percolation and Lattice Animal Problems.,” https://dx.doi.org/10.1051/jphys:0198500460100162300J. Phys. (Paris) 46 no. 10, (1985) 1623–1630.
https://doi.org/10.1051/jphys:0198500460100162300

Grimmett1989
G. Grimmett, Percolation.
Springer Verlag, Berlin, 1989.
<https://link.springer.com/book/10.1007/978-3-662-03981-6>

Jacobsen2014
J. L. Jacobsen, “High-precision percolation thresholds and Potts-model critical manifolds from graph polynomials,” https://dx.doi.org/10.1088/1751-8113/47/13/135001J. Phys. A: Math. Theor. 47 no. 13, (2014) 135001.
https://dx.doi.org/10.1088/1751-8113/47/13/135001

Hoshen1976a
J. Hoshen and R. Kopelman, “Percolation and cluster distribution. I. Cluster multiple labeling technique and critical concentration algorithm,” https://dx.doi.org/10.1103/PhysRevB.14.3438Phys. Rev. B 14 no. 8, (10, 1976) 3438–3445. https://link.aps.org/doi/10.1103/PhysRevB.14.3438

Paszke2019
A. Paszke, S. Gross, et al., “PyTorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing Systems 32 no. NeurIPS, (2019). <http://arxiv.org/abs/1912.01703>

Schenk2008b
O. Schenk, M. Bollhöfer, and R. A. Römer, “On Large-Scale Diagonalization Techniques for the Anderson Model of Localization,” https://dx.doi.org/10.1137/070707002SIAM Review 50 no. 1, (1, 2008) 91–112. http://epubs.siam.org/doi/10.1137/070707002

Anderson1958a
P. W. Anderson, “Absence of Diffusion in Certain Random Lattices,” https://dx.doi.org/10.1103/PhysRev.109.1492Phys. Rev. 109 no. 5, (3, 1958) 1492–1505. https://link.aps.org/doi/10.1103/PhysRev.109.1492

Elsner1999
U. Elsner, V. Mehrmann, F. Milde, R. A. Römer, and M. Schreiber, “The Anderson Model of Localization: A Challenge for Modern Eigenvalue Methods,” https://dx.doi.org/10.1137/S1064827598332217SIAM Journal on Scientific Computing 20 no. 6, (1, 1999) 2089–2102. http://epubs.siam.org/doi/10.1137/S1064827598332217

Brandes2003AndersonRamifications
T. Brandes and S. Kettemann, https://dx.doi.org/10.1007/b13139Anderson Localization and Its Ramifications, vol. 630 of Lecture Notes in Physics.
Springer Berlin Heidelberg, Berlin, Heidelberg, 2003.
<http://link.springer.com/10.1007/b13139>

Evers2008
F. Evers and A. D. Mirlin, “Anderson transitions,” Rev. Mod. Phys. 80 no. 4, (2008) 1355–1417. <http://link.aps.org/doi/10.1103/RevModPhys.80.1355>.

Theveniaut2019NeuralLimitations
H. Théveniaut and F. Alet, “Neural network setups for a precise detection of the many-body localization transition: finite-size scaling and limitations,”https://dx.doi.org/10.1103/PhysRevB.100.224202Phys. Rev. B 100 no. 22, (2019) 224202. https://link.aps.org/doi/10.1103/PhysRevB.100.224202

Mano2017a
T. Mano and T. Ohtsuki, “Phase diagrams of three-dimensional Anderson and quantum percolation models using deep three-dimensional convolutional neural network,” https://dx.doi.org/10.7566/JPSJ.86.113704J. Phys. Soc. Jpn. 86 no. 11, (2017) . https://doi.org/10.7566/JPSJ.86.113704

Cadez2023MachinePhases
T. Cadez, B. Dietz, D. Rosa, A. Andreanov, K. Slevin, and T. Ohtsuki, “Machine learning wave functions to identify fractal phases,”. <http://arxiv.org/abs/2306.01402>.

Romer2022NumericalLocalization
R. A. Römer, https://dx.doi.org/10.1016/B978-0-323-90800-9.00099-8“Numerical methods for localization,” in Reference Module in Materials Science and Materials Engineering.
Elsevier, 7, 2022.

https://linkinghub.elsevier.com/retrieve/pii/B9780323908009000998

Bollhofer2007JADAMILU:Matrices
M. Bollhöfer and Y. Notay, “JADAMILU: a software code for computing selected eigenvalues of large sparse symmetric matrices,” https://dx.doi.org/https://doi.org/10.1016/j.cpc.2007.08.004Comput. Phys. Commun. 177 no. 12, (2007) 951–964. https://www.sciencedirect.com/science/article/pii/S0010465507003700

Swendsen1979MonteN2
R. H. Swendsen and S. Krinsky, “Monte Carlo Renormalization Group and Ising Models with n≥2,” https://dx.doi.org/10.1103/PhysRevLett.43.177Phys. Rev. Lett. 43 no. 3, (7, 1979) 177–180. <https://link.aps.org/doi/10.1103/PhysRevLett.43.177>

Landau1980PhaseInteractions
D. P. Landau, “Phase transitions in the Ising square lattice with next-nearest-neighbor interactions,” https://dx.doi.org/10.1103/PhysRevB.21.1285Phys. Rev. B 21 no. 3, (2, 1980) 1285–1297. https://link.aps.org/doi/10.1103/PhysRevB.21.1285

Binder1980PhaseInteractions
K. Binder and D. P. Landau, “Phase diagrams and critical behavior in Ising square lattices with nearest- and next-nearest-neighbor interactions,” https://dx.doi.org/10.1103/PhysRevB.21.1941Phys. Rev. B 21 no. 5, (3, 1980) 1941–1962. https://link.aps.org/doi/10.1103/PhysRevB.21.1941

Landau1985PhaseCouplings
D. P. Landau and K. Binder, “Phase diagrams and critical behavior of Ising square lattices with nearest-, next-nearest-, and third-nearest-neighbor couplings,” https://dx.doi.org/10.1103/PhysRevB.31.5946Phys. Rev. B 31 no. 9, (5, 1985) 5946–5953. https://link.aps.org/doi/10.1103/PhysRevB.31.5946

Moran-Lopez1993First-orderInteractions
J. L. Morán-López, F. Aguilera-Granja, and J. M. Sanchez, “First-order phase transitions in the Ising square lattice with first- and second-neighbor interactions,” https://dx.doi.org/10.1103/PhysRevB.48.3519Phys. Rev. B 48 no. 5, (8, 1993) 3519–3522. https://link.aps.org/doi/10.1103/PhysRevB.48.3519

Malakis2006MonteInteractions
A. Malakis, P. Kalozoumis, and N. Tyraskis, “Monte Carlo studies of the square Ising model with next-nearest-neighbor interactions,” https://dx.doi.org/10.1140/EPJB/E2006-00032-2Eur. Phys. J. B 50 no. 1, (2, 2006) 63–67. https://link.springer.com/article/10.1140/epjb/e2006-00032-2

Monroe2007PhaseModel
J. L. Monroe and S.-Y. Kim, “Phase diagram and critical exponent ν for the nearest-neighbor and next-nearest-neighbor interaction Ising model,” https://dx.doi.org/10.1103/PhysRevE.76.021123Phys. Rev. E 76 no. 2, (8, 2007) 021123. https://link.aps.org/doi/10.1103/PhysRevE.76.021123

dosAnjos2008PhaseLattice
R. A. dos Anjos, J. Roberto Viana, and J. Ricardo de Sousa, “Phase diagram of the Ising antiferromagnet with nearest-neighbor and next-nearest-neighbor interactions on a square lattice,” https://dx.doi.org/https://doi.org/10.1016/j.physleta.2007.09.059Phys. Lett. A 372 no. 8, (2008) 1180–1184. https://www.sciencedirect.com/science/article/pii/S0375960107013680

Kalz2008PhaseInteractions
A. Kalz, A. Honecker, S. Fuchs, and T. Pruschke, “Phase diagram of the Ising square lattice with competing interactions,” https://dx.doi.org/10.1140/epjb/e2008-00359-6Eur. Phys. J. B 65 no. 4, (10, 2008) 533. https://doi.org/10.1140/epjb/e2008-00359-6

Watanabe2023Non-monotonicSystems
H. Watanabe, Y. Motoyama, S. Morita, and N. Kawashima, “Non-monotonic behavior of the Binder parameter in discrete spin systems,” https://dx.doi.org/10.1093/ptep/ptad022Prog. Theor. Exp. Phys. 2023 no. 3, (8, 2023) 033A02. https://doi.org/10.1093/ptep/ptad022

Yoshiyama2023Higher-orderLattice
K. Yoshiyama and K. Hukushima, “Higher-order tensor renormalization group study of the J_1-J_2 Ising model on a square lattice,” https://dx.doi.org/10.1103/PhysRevE.108.054124Phys. Rev. E 108 no. 5, (11, 2023) 054124. https://link.aps.org/doi/10.1103/PhysRevE.108.054124

Li2024ALattice
S.-W. Li and F.-J. Jiang, “A Comprehensive Study of the Phase Transitions of the Frustrated J_1-J_2 Ising Model on the Square Lattice,” https://dx.doi.org/10.1093/ptep/ptae061Prog. Theor. Exp. Phys. 2024 no. 5, (8, 2024) 053A06. https://doi.org/10.1093/ptep/ptae061

Park2022FrustratedMachine
H. Park and H. Lee, “Frustrated Ising Model on D-wave Quantum Annealing Machine,” https://dx.doi.org/10.7566/JPSJ.91.074001J. Phys. Soc. Jpn. 91 no. 7, (6, 2022) . https://journals.jps.jp/doi/abs/10.7566/JPSJ.91.074001

Civitcioglu2024PhaseLearning
B. Çivitcioğlu, R. A. Römer, and A. Honecker, “Phase determination with and without deep learning,” <http://arxiv.org/abs/2403.09786>.

Metropolis1953EquationMachines
N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equation of State Calculations by Fast Computing Machines,” https://dx.doi.org/10.1063/1.1699114J. Chem. Phys. 21 no. 6, (1953) 1087–1092. https://doi.org/10.1063/1.1699114

Newman1999MontePhysics
M. E. J. Newman and G. T. Barkema, Monte Carlo Methods in Statistical Physics.
Oxford University Press, 1999.
<https://global.oup.com/academic/product/monte-carlo-methods-in-statistical-physics-9780198517979>

Berg2004MarkovAnalysis
B. A. Berg, https://dx.doi.org/10.1142/5602Markov Chain Monte Carlo Simulations and Their Statistical Analysis.
World Scientific, 10, 2004.
https://www.worldscientific.com/worldscibooks/10.1142/5602

Landau2014APhysics
D. P. Landau and K. Binder, https://dx.doi.org/10.1017/CBO9781139696463A Guide to Monte Carlo Simulations in Statistical Physics.
Cambridge University Press, 4 ed., 2014.
https://doi.org/10.1017/CBO9781139696463

Acevedo2021PhaseLearning
S. Acevedo, M. Arlego, and C. A. Lamas, “Phase diagram study of a two-dimensional frustrated antiferromagnet via unsupervised machine learning,” https://dx.doi.org/10.1103/PhysRevB.103.134422Phys. Rev. B 103 no. 13, (2021) 134422. https://link.aps.org/doi/10.1103/PhysRevB.103.134422

Kramers1941
H. A. Kramers and G. H. Wannier, “Statistics of the two-dimensional ferromagnet. part I,” https://dx.doi.org/10.1103/PhysRev.60.252Phys. Rev. 60 (Aug, 1941) 252–262. https://link.aps.org/doi/10.1103/PhysRev.60.252

Spence1992ElectronMicrodiffraction
J. C. H. Spence and J. M. Zuo, https://dx.doi.org/10.1007/978-1-4899-2353-0Electron Microdiffraction.
https://doi.org/10.1007/978-1-4899-2353-0
Springer US, Boston, MA, 1992.

Tanaka1994Convergent-beamDiffraction
M. Tanaka, “Convergent-beam electron diffraction,” https://dx.doi.org/10.1107/S0108767393010426Acta Crystallogr. A 50 no. 3, (5, 1994) 261–286.
https://doi.org/10.1107/S0108767393010426

Beanland2013DigitalPicture
R. Beanland, P. J. Thomas, D. I. Woodward, P. A. Thomas, and R. A. Roemer, “Digital electron diffraction – seeing the whole picture,” https://dx.doi.org/10.1107/S0108767313010143Acta Crystallogr. A 69 no. 4, (7, 2013) 427–434. https://scripts.iucr.org/cgi-bin/paper?S0108767313010143

Kossel1939ElektroneninterferenzenBundel
W. Kossel and G. Möllenstedt, “Elektroneninterferenzen im konvergenten Bündel,” https://dx.doi.org/10.1002/andp.19394280204Annalen der Physik 428 no. 2, (1, 1939) 113–140. https://onlinelibrary.wiley.com/doi/10.1002/andp.19394280204

Buxton1976ThePatterns
B. F. Buxton, J. A. Eades, J. W. Steeds, and G. M. Rackham, “The symmetry of electron diffraction zone axis patterns,” https://dx.doi.org/10.1098/rsta.1976.0024Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences 281 no. 1301, (3, 1976) 171–194. https://royalsocietypublishing.org/doi/10.1098/rsta.1976.0024

Tanaka1983Point-groupDiffraction
M. Tanaka, R. Saito, and H. Sekii, “Point-group determination by convergent-beam electron diffraction,” https://dx.doi.org/10.1107/S010876738300080XActa Crystallogr. A 39 no. 3, (5, 1983) 357–368.
https://doi.org/10.1107/S010876738300080X

Tanaka1983Space-groupDiffraction
M. Tanaka, H. Sekii, and T. Nagasawa, “Space-group determination by dynamic extinction in convergent-beam electron diffraction,” https://dx.doi.org/10.1107/S0108767383001695Acta Crystallogr. A 39 no. 6, (11, 1983) 825–837.
https://doi.org/10.1107/S0108767383001695

Saunders1999QuantitativeMeasurements
M. Saunders, A. G. Fox, and P. A. Midgley, “Quantitative zone-axis convergent-beam electron diffraction (CBED) studies of metals. II. Debye–Waller-factor measurements,” https://dx.doi.org/10.1107/S0108767398016316Acta Crystallogr. A 55 no. 3, (5, 1999) 480–488.
https://doi.org/10.1107/S0108767398016316

Zuo1998AMatching
J. M. Zuo, M. Kim, and R. Holmestad, “A new approach to lattice parameter measurements using dynamic electron diffraction and pattern matching,” https://dx.doi.org/10.1093/oxfordjournals.jmicro.a023568J. Electron Microsc. Tech. 47 no. 2, (1, 1998) 121–127.
https://doi.org/10.1093/oxfordjournals.jmicro.a023568

Kaiser1999ApplicationSubstrates
U. Kaiser, K. Saitoh, K. Tsuda, and M. Tanaka, “Application of the CBED method for the determination of lattice parameters of cubic SiC films on 6H SiC substrates,” https://dx.doi.org/10.1093/oxfordjournals.jmicro.a023672J. Electron Microsc. Tech. 48 no. 3, (1, 1999) 221–233.
https://doi.org/10.1093/oxfordjournals.jmicro.a023672

Armigliato2003ApplicationDevices
A. Armigliato, R. Balboni, G. P. Carnevale, G. Pavia, D. Piccolo, S. Frabboni, A. Benedetti, and A. G. Cullis, “Application of convergent beam electron diffraction to two-dimensional strain mapping in silicon devices,” https://dx.doi.org/10.1063/1.1565181Appl. Phys. Lett. 82 no. 13, (3, 2003) 2172–2174. https://pubs.aip.org/apl/article/82/13/2172/510371/Application-of-convergent-beam-electron

Kramer2000AnalysisCBED
S. Krämer, J. Mayer, C. Witt, A. Weickenmeier, and M. Rühle, “Analysis of local strain in aluminium interconnects by energy filtered CBED,” https://dx.doi.org/10.1016/S0304-3991(99)00191-6Ultramicroscopy 81 no. 3-4, (4, 2000) 245–262.
https://www.sciencedirect.com/science/article/pii/S0304399199001916

Cherns1989ConvergentMultilayers
D. Cherns and A. R. Preston, “Convergent beam diffraction studies of interfaces, defects, and multilayers,” https://dx.doi.org/10.1002/jemt.1060130204J. Electron Microsc. Tech. 13 no. 2, (10, 1989) 111–122.
https://onlinelibrary.wiley.com/doi/abs/10.1002/jemt.1060130204

Morniroli1996AnalysisDiffraction
J. Morniroli and D. Cherns, “Analysis of grain boundary dislocations by large angle convergent beam electron diffraction,” https://dx.doi.org/10.1016/0304-3991(95)00087-9Ultramicroscopy 62 no. 1-2, (1, 1996) 53–63.
https://www.sciencedirect.com/science/article/pii/0304399195000879

Midgley1996QuantitativeBonds
P. A. Midgley and M. Saunders, “Quantitative electron diffraction: From atoms to bonds,” https://dx.doi.org/10.1080/00107519608217535Contemp. Phys. 37 no. 6, (11, 1996) 441–456.
https://doi.org/10.1080/00107519608217535

Zuo1999DirectCu2O
J. M. Zuo, M. Kim, M. O'Keeffe, and J. C. H. Spence, “Direct observation of d-orbital holes and Cu–Cu bonding in Cu_2O,” https://dx.doi.org/10.1038/43403Nature 401 no. 6748, (9, 1999) 49–52. https://www.nature.com/articles/43403

Beanland2021RefinementDiffraction
R. Beanland, A. Hubert, and R. Roemer, “Refinement of crystal structure using ‘digital’ large angle convergent beam electron diffraction,” https://dx.doi.org/10.1017/S1431927621004803Microsc. Microanal. 27 no. S1, (8, 2021) 1282–1284. https://academic.oup.com/mam/article/27/S1/1282/6889110

Hubert2019StructurePatterns
A. Hubert, R. Römer, and R. Beanland, “Structure refinement from ‘digital’ large angle convergent beam electron diffraction patterns,” https://dx.doi.org/10.1016/j.ultramic.2018.12.007Ultramicroscopy 198 (3, 2019) 1–9.
https://www.sciencedirect.com/science/article/pii/S0304399118303498

Zuo1995OnMethod
J. Zuo and A. Weickenmeier, “On the beam selection and convergence in the Bloch-wave method,” https://dx.doi.org/10.1016/0304-3991(94)00190-XUltramicroscopy 57 no. 4, (3, 1995) 375–383.
https://www.sciencedirect.com/science/article/pii/030439919400190X

2018ElectronMicroscopy
B. G. Mendis, ed., https://dx.doi.org/10.1002/9781118696545Electron Beam‐Specimen Interactions and Simulation Methods in Microscopy.
Wiley, 5, 2018.
https://onlinelibrary.wiley.com/doi/book/10.1002/9781118696545

Cowley1957TheApproach
J. M. Cowley and A. F. Moodie, “The scattering of electrons by atoms and crystals. I. A new theoretical approach,” https://dx.doi.org/10.1107/S0365110X57002194Acta Crystallographica 10 no. 10, (10, 1957) 609–619. https://scripts.iucr.org/cgi-bin/paper?S0365110X57002194

VanDyck1984TheMicroscopy
D. Van Dyck and W. Coene, “The real space method for dynamical electron diffraction calculations in high resolution electron microscopy,” https://dx.doi.org/10.1016/0304-3991(84)90072-XUltramicroscopy 15 no. 1-2, (1, 1984) 29–40. https://linkinghub.elsevier.com/retrieve/pii/030439918490072X

Chuvilin2005OnSimulation
A. Chuvilin and U. Kaiser, “On the peculiarities of CBED pattern formation revealed by multislice simulation,” https://dx.doi.org/10.1016/j.ultramic.2005.03.003Ultramicroscopy 104 no. 1, (8, 2005) 73–82. https://linkinghub.elsevier.com/retrieve/pii/S0304399105000483

Kaiser2006ProspectsCalculation
U. Kaiser and A. Chuvilin, “Prospects of the multislice method for CBED pattern calculation,” https://dx.doi.org/10.3139/146.101319Int. J. Mater. Res. 97 no. 7, (7, 2006) 912–919. http://www.hanser-elibrary.com/doi/abs/10.3139/146.101319

Kirkland2010AdvancedMicroscopy
E. J. Kirkland, https://dx.doi.org/10.1007/978-1-4419-6533-2Advanced Computing in Electron Microscopy.
Springer US, Boston, MA, 2010.
https://link.springer.com/10.1007/978-1-4419-6533-2

BeanlandFelix:Software
R. Beanland, K. Evans, and R. A. Roemer, “Felix: Bloch wave method diffraction pattern simulation software.” 2023.
<https://github.com/WarwickMicroscopy/Felix>

2023Kaggle:Science
“Kaggle: Your Home for Data Science.” 2023.
<https://www.kaggle.com/>

Halevy2009TheData
A. Halevy, P. Norvig, and F. Pereira, “The Unreasonable Effectiveness of Data,” https://dx.doi.org/10.1109/MIS.2009.36IEEE Intelligent Systems 24 no. 2, (3, 2009) 8–12. http://ieeexplore.ieee.org/document/4804817/

Spasic2020ClinicalReview
I. Spasic and G. Nenadic, “Clinical Text Data in Machine Learning: Systematic Review,” https://dx.doi.org/10.2196/17984JMIR Medical Informatics 8 no. 3, (3, 2020) e17984. http://medinform.jmir.org/2020/3/e17984/

Webb2024GitHubWephy/ai-diffraction
J. J. Webb, “GitHub - wephy/ai-diffraction.” 2, 2024.
<https://github.com/wephy/ai-diffraction>

Webb2023FelixPatterns
J. J. Webb and R. Beanland, “Felix Diffraction Patterns.” V2, 2023.
<https://www.kaggle.com/datasets/wephys/felix-diffraction-patterns>

Saini2023TacklingReview
M. Saini and S. Susan, “Tackling class imbalance in computer vision: a contemporary review,” https://dx.doi.org/10.1007/s10462-023-10557-6Artif. Intell. Rev. 56 no. S1, (10, 2023) 1279–1335. https://link.springer.com/10.1007/s10462-023-10557-6

Edwards1975TheoryGlasses
S. F. Edwards and P. W. Anderson, “Theory of spin glasses,” https://dx.doi.org/10.1088/0305-4608/5/5/017J. Phys. F: Met. Phys. 5 no. 5, (5, 1975) 965. 
https://iopscience.iop.org/article/10.1088/0305-4608/5/5/017/meta

Carleo2016
G. Carleo and M. Troyer, “Solving the quantum many-body problem with artificial neural networks”, https://dx.doi.org/10.1126/science.aag2302Science 355 no. 6325, (2, 2017) 602–606. 

http://dx.doi.org/10.1126/science.aag2302 http://0-science.sciencemag.org/content/355/6325/602

Zhu2023HubbardNet:Networks
Z. Zhu, M. Mattheakis, W. Pan, and E. Kaxiras, “HubbardNet: Efficient predictions of the Bose-Hubbard model spectrum with deep neural networks,” https://dx.doi.org/10.1103/PhysRevResearch.5.043084Phys. Rev. Res. 5 no. 4, (10, 2023) 043084. https://link.aps.org/doi/10.1103/PhysRevResearch.5.043084

Pfau2024AccurateNetworks
D. Pfau, S. Axelrod, H. Sutterud, I. v. Glehn, and J. S. Spencer, “Accurate computation of quantum excited states with neural networks,” https://dx.doi.org/10.1126/SCIENCE.ADN0137Science 385 no. 6711, (8, 2024) . https://0-www-science-org/doi/10.1126/science.adn0137

PhDBayo24
D. Bayo, Phase Determination by Deep Learning for Disordered Systems.
PhD thesis, University of Warwick (home) & CY Cergy Paris Université (host), 2024.

PhDCivitcioglu24
B. Çivitcioğlu, Phase Determination with and without Deep Learning for the J_1-J_2 Ising Model.
PhD thesis, CY Cergy Paris Université, 2024.



