 case has no impact on execution time, as the autoencoder architectures and learning hyperparameter settings are identical in both cases. 
Moreover, as a consequence of allowing the minibatch size to proportionally vary with the training-validation sample size $N$, we found that the execution time depended very little on $N$. Thus, we report each execution time as an average over all sample sizes $N$ and over the ferromagnetic vs.\ antiferromagnetic cases. 
We find that the GE-autoencoder method is significantly faster than the baseline-autoencoder. 
Moreover, multiscale training gives an additional boost in efficiency in the computation time needed to extrapolate estimates to infinite lattice size, as the multiscale GE-autoencoder needs to be trained only once across the four finite lattice sizes. 
We therefore conclude that our GE-autoencoder method is indeed more efficient than the baseline-autoencoder method.

\begin{figure}
\centering
\includegraphics[width=1.0\columnwidth]{figs/plots/time.pdf}
\caption{\label{fig:times} %
Computation times to generate data (red) and train (not red) the autoencoders for each lattice size $L$. 
Note that the time to measure magnetization is reported as equivalent to data generation time as these measurements were taken on-the-fly during the MC simulation. 
Note also that each training time is a \emph{sum} over all eight training folds and RNG seeds. 
The training times were then averaged over all training-validation sample sizes $N$ and over both the ferromagnetic and antiferromagnetic cases, 
as these factors did not contribute to significant variation in execution time. 
The GE-autoencoder is significantly more time-efficient (lower time) than the baseline-autoencoder. 
The times reported for infinite lattice size are just sums of the times for the finite lattice sizes, 
although the multiscale GE-autoencoder exhibits greater efficiency as it only requires a single network to be trained across all finite lattice sizes.
}
\end{figure}


\subsection{Detecting an external magnetic field}
\label{sec:field}

Finally, we investigate if the baseline-encoder and GE-encoder order observables can be used to detect the presence of a weak external magnetic field $h$ in the ferromagnetic Ising model by adding a term to Eq.~\eqref{Eq:HIsing}:
\begin{equation}
    \mathcal{H}(\mathbf{x}) \to \mathcal{H}(\mathbf{x}) - h\sum_{\bf i} {x}_{\bf i}\,. 
\end{equation}
We assume the magnetic field is uniform and note that it breaks the internal symmetry of the Ising model.
We consider two temperatures---$2.0$ and $2.5$---slightly below and above the critical temperature, and we consider three field strengths $0.001J$, $0.01J$, and $0.1J$ (where $J=1$ is the coupling constant in the Ising Hamiltonian). 
For each case, as well as the case of no external field at all, we use the Wolff algorithm with a ``ghost site''~\cite{coniglio1989exact} to simulate the Ising model with lattice size $L=128$ in an external magnetic field; 
we generate $N=2,000$ sample lattice configurations for each temperature and field strength. 
If $x^0_1,\ldots,x^0_N$ (resp. $x_1,\ldots,x_N$) are the sample lattice configurations in the absence (resp. presence) of an external magnetic field, then for each encoder $\mathcal{O}$, we compute the statistic
\begin{equation}
D = \left \lvert \frac{1}{N}\sum_{n=1}^N\mathcal{O}(x_n) - \frac{1}{N}\sum_{n=1}^N\mathcal{O}(x^0_n)\right\rvert. 
\end{equation}
Using the baseline-encoder and GE-encoder already fitted to data as described in previous sections, we obtain $24$ measurements of $D$ for each encoder (eight training data folds and three initialization seeds). 
We then define the ``confidence score''
\begin{equation} \label{eq:xi}
\xi = \frac{\langle D\rangle}{\sqrt{\langle D^2\rangle-\langle D\rangle^2}}.
\end{equation}
Intuitively, a value sufficiently far from zero indicates that the order parameter has shifted and hence there is an external symmetry-breaking field. 
We normalize by the standard deviation of $D$ so that $\xi$ is independent of the arbitrary scale learned by each encoder; 
it also boosts the score when the measurement of $D$ is robust across the $24$ trials. 
Figure~\ref{fig:field} shows the $\xi$ scores for the baseline-encoder and GE-encoder at each temperature and external field strength. 
In all cases, the GE-encoder attains a significantly higher score than does the baseline-encoder, meaning that it is more sensitive to and detects with greater confidence the presence of a weak external magnetic field. 
At temperature $2.5$, we note that the baseline-encoder does become more confident (increasing score) in its detection as the external field becomes stronger, which is to be expected; 
however, the GE-encoder remains confident even in its detection of the weakest field.

\begin{figure}
\centering
\includegraphics[width=1.0\columnwidth]{figs/plots/anomaly_detection.pdf}
\caption{\label{fig:field} %
``Confidence scores'' $\xi$ (see Eq.~\ref{eq:xi}) of the baseline-encoder (green) and GE-encoder (blue) at detecting the presence of an external magnetic field at two different temperatures and for three different field strengths. 
The error bars represent standard deviations estimated using $10,000$ bootstrap draws from the $24$ trials used to calculate $\xi$. 
The GE-encoder consistently detects the external field with greater confidence than does the baseline-encoder.
}
\end{figure}


\section{On vector order observables}
\label{sec:vector}

\subsection{A 2D order observable}
\label{sec:2D}

We have thus far assumed that the order observable is scalar-valued or equivalently that a sufficient choice for the latent dimension of the autoencoder networks is $1$. 
In this section, we justify this assumption by instead assuming a 2D vector order observable; 
if the real 2D orthogonal representation $\psi:G\mapsto\mathrm{O}(2, \mathbb{R})$ by which the order observable transforms---as learned by the GE-autoencoder---is equivalent to a 1D representation, then we may conclude that a 1D order observable is sufficient to describe the phase transition. 
The exposition in this section will also help to illustrate how our GE-autoencoder method can be extended to a somewhat more complex scenario.

We construct the 2D GE-autoencoder in direct analogy to the 1D case. 
First, we calculate the subgroup of never-broken symmetries and find it to be $H = \langle\alpha^2, \beta^2\rangle$, 
which is the subgroup generated by the even horizontal translations and even vertical translations (see Prop.~\ref{prop:never_broken_2D} in Appendix~\ref{appendix:2D}). 
This subgroup has ${L^2}/{4}$ symmetries for even lattice size $L$, 
and the corresponding quotient group $G/H$ of possibly-broken symmetries has $16L^2/(L^2/4) = 64$ elements. 
Compare this to the 1D case, in which there are $L^2$ never-broken symmetries and only $16$ possibly-broken symmetries.

Next, we constrain the GE-autoencoder so that its encoder is invariant to the subgroup $H$ of never-broken symmetries. 
Instead of constraining each weight matrix $\mathbf{w}_k$ (associated to the $k^\text{th}$ hidden neuron) to have a checkerboard pattern as in the 1D case, 
we constrain it to be a tiling of a $2\times 2$ submatrix; 
the resulting weight matrix is then invariant to the action of the subgroup $H$, and hence so is the encoder. 
Applying this constrained encoder to an input $L\times L$ lattice configuration $\mathbf{x}$ is then equivalent to the following procedure: 
First, we construct a reduced 4D representation $\check{x} = (\check{x}_1,\check{x}_2,\check{x}_3,\check{x}_4)$ of $\mathbf{x}$ as follows:
\begin{align*}
\check{x}_1 &= \frac{4}{L^2}\sum_{i_x \mbox{ even, } i_y \mbox{ even}} x_{\bf i} \\
\check{x}_2 &= \frac{4}{L^2}\sum_{i_x \mbox{ even, } i_y \mbox{ odd}} x_{\bf i} \\
\check{x}_3 &= \frac{4}{L^2}\sum_{i