# MarginMatch: Improving Semi-Supervised Learning with Pseudo-Margins

Tiberiu Sosea Cornelia Caragea
University of Illinois Chicago
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We introduce MarginMatch, a new SSL approach combining consistency
regularization and pseudo-labeling, with its main novelty arising from the use
of unlabeled data training dynamics to measure pseudo-label quality. Instead
of using only the model’s confidence on an unlabeled example at an arbitrary
iteration to decide if the example should be masked or not, MarginMatch also
analyzes the behavior of the model on the pseudo-labeled examples as the
training progresses, to ensure low quality predictions are masked out.
MarginMatch brings substantial improvements on four vision benchmarks in low
data regimes and on two large-scale datasets, emphasizing the importance of
enforcing high-quality pseudo-labels. Notably, we obtain an improvement in
error rate over the state-of-the-art of $3.25\%$ on CIFAR-100 with only $25$
labels per class and of $3.78\%$ on STL-10 using as few as $4$ labels per
class. We make our code available at https://github.com/tsosea2/MarginMatch.

##  1 Introduction

Deep learning models have seen tremendous success in many vision tasks [22,
43, 42, 14, 27]. This success can be attributed to their scalability, being
able to produce better results when they are trained on large datasets in a
supervised fashion [27, 43, 15, 35, 47, 34]. Unfortunately, large labeled
datasets annotated for various tasks and domains are difficult to acquire and
demand considerable annotation effort or domain expertise. Semi-supervised
learning (SSL) is a powerful approach that mitigates the requirement for large
labeled datasets by effectively making use of information from unlabeled data,
and thus, has been studied extensively in vision [30, 39, 23, 44, 5, 4, 46,
25, 38, 36, 45].

Recent SSL approaches integrate two important components: consistency
regularization [46, 49] and pseudo-labeling [25]. Consistency regularization
works on the assumption that a model should output similar predictions when
fed perturbed versions of the same image, whereas pseudo-labeling uses the
model’s predictions of unlabeled examples as labels to train against. For
example, Sohn et al. [41] introduced FixMatch that combines consistency
regularization on weak and strong augmentations with pseudo-labeling. FixMatch
relies heavily on a high-confidence threshold to compute the unsupervised
loss, disregarding any pseudo-labels whose confidence falls below this
threshold. While training using only high-confidence pseudo-labels has shown
to consistently reduce the confirmation bias [1], this rigid threshold allows
access only to a small amount of unlabeled data for training, and thus,
ignores a considerable amount of unlabeled examples for which the model’s
predictions do not exceed the confidence threshold. More recently, Zhang et
al. [49] introduced FlexMatch that relaxes the rigid confidence threshold in
FixMatch to account for the model’s learning status of each class in that it
adaptively scales down the threshold for a class to encourage the model to
learn from more examples from that class. The flexible thresholds in FlexMatch
allow the model to have access to a much larger and diverse set of unlabeled
data to learn from, but lowering the thresholds can lead to the introduction
of wrong pseudo-labels, which are extremely harmful for generalization.
Interestingly, even when the high-confidence threshold is used in FixMatch can
result in wrong pseudo-labels. See Figure 1 for incorrect pseudo-labels
detected in the training set after we apply FixMatch and FlexMatch on
ImageNet. We posit that a drawback of FixMatch and FlexMatch and in general of
any pseudo-labeling approach is that they use the confidence of the model only
at the current iteration to enforce quality of pseudo-labels and completely
ignore model’s predictions at prior iterations.

FixMatch Predicted: onion elephant fossa green pepper pop art crowd
firefighter horse crowd
Actual: bell pepper camel cougar handrail poncho uniform voleyball bison meat
market FlexMatch Predicted: screen pyramid decoration scale computer carpet
cabbage tower screen
Actual: stopwatch obelisk socks parking meter heater teddy bear cauliflower
torch ipod Figure 1: Incorrect pseudo-labels propagated until the end of the
training process for FixMatch and FlexMatch on ImageNet.

In this paper, we propose MarginMatch, a new SSL approach that monitors the
behavior of the model on the unlabeled examples as the training progresses,
from the beginning of training until the current iteration, instead of using
only the model’s current _belief_ about an unlabeled example (i.e., its
confidence at the current iteration) to decide if the example should be masked
or not.

We estimate a pseudo-label’s contribution to learning and generalization by
introducing pseudo-margins of unlabeled examples averaged across training
iterations. Pseudo-margins of unlabeled examples extend the margins from
machine learning [3, 33, 11, 18] which provide a measure of confidence of the
outputs of the model and capture the difference between the output for the
correct (gold) label and the other labels. In our case, the pseudo-margins
capture how much larger the assigned logit (the logit corresponding to the
argmax of the model’s prediction) is compared with all other logits at
iteration $t$. Similar to FlexMatch, in MarginMatch we take advantage of the
flexible confidence thresholds to allow the model to learn from larger and
more diverse sets of unlabeled examples, but unlike FlexMatch, we train the
model itself to identify the characteristics of mislabeled pseudo-labels
simply by monitoring the model’s training dynamics on unlabeled data over the
iterations.

We carry out comprehensive experiments using established SSL experimental
setups on CIFAR-10, CIFAR-100 [21], SVHN [31], STL-10 [8], ImageNet [10], and
WebVision [26]. Despite its simplicity, our findings indicate that MarginMatch
produces improvements in performance over strong baselines and prior works on
all datasets at no additional computational cost. Notably, compared to current
state-of-the-art, on CIFAR-100 we see $3.02\%$ improvement in error rate using
only $4$ labels per class and $3.78\%$ improvement on STL-10 using the same
extremely label-scarce setting of $4$ labels per class. In addition, on
ImageNet [10] and WebVision [26] we find that MarginMatch pushes the state-of-
the-art error rates by $0.97\%$ on ImageNet and by $0.79\%$ on WebVision.

Our contributions are as follows:

  1. 1.

We introduce a new SSL approach which we call MarginMatch that enforces high
pseudo-label quality during training. Our approach allows access to a large
set of unlabeled data to learn from (thus, incorporating more information from
unlabeled data) and, at the same time, monitors the training dynamics of
unlabeled data as training progresses to detect and filter out potentially
incorrect pseudo-labels.

  2. 2.

We show that MarginMatch outperforms existing works on six well-established
computer vision benchmarks showing larger improvements in error rates
especially on challenging datasets, while achieving similar convergence
performance (or better) than prior works.

  3. 3.

We perform a comprehensive analysis of our approach and indicate potential
insights into why our MarginMatch substantially outperforms other SSL
techniques.

##  2 MarginMatch

#### Notation

Let $L=\\{(x_{1},y_{1}),...,(x_{B},y_{B})\\}$ be a batch of size $B$ of
labeled examples and $U=\\{\hat{x}_{1},...,\hat{x}_{\nu B}\\}$ be a batch of
size $\nu B$ of unlabeled examples, where $\nu$ is the batch-wise ratio of
unlabeled to labeled examples. Let $p_{\theta}(y|x)$ denote the class
distribution produced by model $\theta$ on input image $x$ and
$\hat{p}_{\theta}(y|x)$ denote the argmax of this distribution as a one-hot
label. Let also $H(p,q)$ denote the cross-entropy between two probability
distributions $p$ and $q$.

###  2.1 Background

Consistency regularization [39] is an important component in recent semi-
supervised learning approaches and relies on the continuity assumption [2, 23]
that the model should output similar predictions on multiple perturbed
versions of the same input $x$. As mentioned above, examples of two such
approaches are FixMatch [41] and FlexMatch [49] that use consistency
regularization at their core combined with psedo-labeling. In psedo-labeling
[25], a model itself is used to assign artificial labels for unlabeled data
and only artificial labels whose largest class probability is above a
predefined confidence threshold are used during training.

Specifically, FixMatch [41] predicts artificial labels for unlabeled examples
using a weakly-augmented version of each unlabeled example and then employs
the artificial labels as pseudo-labels to train against but this time using a
strongly-augmented version of each unlabeled example. That is, FixMatch
minimizes the following batch-wise consistency loss on unlabeled data:

$\mathcal{L}_{u}=\sum_{i=1}^{\nu
B}\mathbbm{1}(\max(p_{\theta}(y|\pi(\hat{x}_{i})))>\tau)\quad\times\\\
H(\hat{p}_{\theta}(y|\pi(\hat{x}_{i})),p_{\theta}(y|\Pi(\hat{x}_{i})))$ (1)

where $\tau$ is a confidence threshold, $\pi$ and $\Pi$ are weak and strong
augmentations, respectively, and $\mathbbm{1}$ is the indicator function.
Therefore, the low-confidence examples (lower than $\tau$) are completely
ignored despite containing potentially useful information for model training.

FlexMatch [49] argues that using a fixed threshold $\tau$ to filter the
unlabeled data ignores the learning difficulties of different classes, and
thus, introduces class-dependent thresholds, which are obtained by adaptively
scaling $\tau$ depending on the learning status of each class. FlexMatch
assumes that a class with fewer examples above the fixed threshold $\tau$ has
a greater learning difficulty, and hence, it adaptively lowers the threshold
$\tau$ to encourage more training examples from this class to be learned. The
learning status $\alpha_{c}$ for a class $c$ is simply computed as the number
of unlabeled examples that are predicted in class $c$ and pass the fixed
threshold $\tau$:

$\alpha_{c}=\sum_{i=1}^{n}\mathbbm{1}(\max(p_{\theta}(y|\pi(\hat{x}_{i})))>\tau)\mathbbm{1}(\hat{p}_{\theta}(y|\pi(\hat{x}_{i}))=c)$
(2)

where $n$ is the total number of unlabeled examples. This learning effect is
then normalized and used to obtain the class-dependent threshold for each
class $c$:

$\mathcal{T}_{c}=\frac{\alpha_{c}}{\underset{c}{\max}(\alpha_{c})}\times\tau$
(3)

In practice, FlexMatch iteratively computes new thresholds after each complete
pass through unlabeled data, hence we can parameterize $\mathcal{T}_{c}$ as
$\mathcal{T}_{c}^{t}$, denoting the threshold obtained at iteration $t$. The
unlabeled loss is then obtained by plugging in the adaptive threshold
$\mathcal{T}_{c}^{t}$ in Eq. 1:

$\mathcal{L}_{u}=\sum_{i=1}^{\nu
B}\mathbbm{1}(\max(p_{\theta}(y|\pi(\hat{x}_{i})))>\mathcal{T}_{\hat{p}_{\theta}(y|\pi(\hat{x}_{i}))}^{t})\quad\times\\\
H(\hat{p}_{\theta}(y|\pi(\hat{x}_{i})),p_{\theta}(y|\Pi(\hat{x}_{i})))$ (4)

The aforementioned works use the confidence of the model solely at the current
iteration to enforce quality of pseudo-labels. We believe this is not
sufficient as it provides only a myopic view of the model’s behavior (i.e.,
its confidence) on unlabeled data (at a single iteration) and may result in
wrong pseudo-labels even when the confidence threshold is high enough (e.g.,
if the model is miscalibrated or overly-confident [13]). Figure 1 shows
examples of images that are added to the training set with a wrong pseudo-
label for both FixMatch and FlexMatch. These types of unlabeled examples,
which are incorrectly pseudo-labeled and used during training are particularly
harmful for deep neural networks, which can attain zero training error on any
dataset, even on randomly assigned labels [50], resulting in poor
generalization capabilities.

###  2.2 Proposed Approach: MarginMatch

We now introduce MarginMatch, our new SSL approach that uses the model’s
training dynamics on unlabeled data to improve pseudo-label data quality. Our
approach leverages consistency regularization with weak and strong
augmentations and pseudo-labeling, but instead of using only the model’s
current _belief_ (i.e., its confidence at the current iteration) to decide if
an unlabeled example should be used for training or not, our MarginMatch
monitors the training dynamics of unlabeled data over the iterations by
investigating the margins (a measure of confidence) of the outputs of the
model [3]. The margin of a training example is a well established metric in
machine learning [3, 33, 11, 18] that quantifies the difference between the
logit corresponding to the assigned ground truth label and the largest other
logit.

1:Labeled data $L$; unlabeled data $U$; erroneous examples $E$; maximum number
of iterations $T$; number of classes $C+1$ ($C$ original  classes plus one
virtual class of erroneous examples); $\theta$ model; $\pi$ weak
augmentations; $\Pi$ strong augmentations.

2:Initialize the Average Pseudo-Margin ($APM$) threshold $\gamma^{1}$ at the
first iteration to a small value (e.g., $\gamma^{1}=-\infty$).

3:for  _t_ = $1$ to $T$ do

4: Estimate learning status $\alpha_{c}$ (using Eq. 2) and calculate the
class-wise flexible thresholds $\mathcal{T}^{t}_{c}$ (using Eq. 3) for each
class $c$.

5: while $U$ not exhausted do

6: Labeled batch $L_{b}=\\{(x_{1},y_{1}),...,(x_{B},y_{B})\\}$, unlabeled
batch $U_{b}=\\{\hat{x}_{1},...,\hat{x}_{\nu B}\\}$, erroneous (or mislabeled)
batch $E_{b}=\mbox{ }\mbox{ }\mbox{ }\mbox{ }\mbox{ }\mbox{ }\mbox{ }\mbox{
}\mbox{ }\mbox{ }\mbox{ }\\{(\tilde{x}_{1},C+1),...,(\tilde{x}_{B},C+1)\\}$

7: for $x\in U_{b}\cup E_{b}$ do

8: Compute logits $z_{c}$ for each class $c$ after applying weak augmentations
when $x\in U_{b}$ and strong augmentations when $x\in E_{b}$.

9: Calculate pseudo-margin $PM^{t}_{c}$ (using Eq. 5) and update Average
$PM^{t}_{c}$ (using Eq. 6) for each $c=1$ to $C+1$.

10: end for

11: Minimize
$\mathcal{L}=\mathcal{L}_{s}+\lambda(\mathcal{L}_{u}+\mathcal{L}_{e})$

12:
$\mathcal{L}_{s}=\frac{1}{B}\sum_{i=1}^{B}H(y_{i},p_{\theta}(y|\pi(x_{i})))$

13: $\mathcal{L}_{u}=\sum_{i=1}^{\nu B}$
$\mathbbm{1}(\mbox{APM}^{t}_{\hat{p}_{\theta}(y|\pi(\hat{x}_{i}))}(\hat{x}_{i})>\gamma^{t})$
$\times\mathbbm{1}(\max(p_{\theta}(y|\pi(\hat{x}_{i})))>\mathcal{T}^{t}_{\hat{p}_{\theta}(y|\pi(\hat{x}_{i}))})\times
H(\hat{p}_{\theta}(y|\pi(\hat{x}_{i})),p_{\theta}(y|\Pi(\hat{x}_{i})))$

14: $\mathcal{L}_{e}=\sum_{i=1}^{B}H(C+1,p_{\theta}(y|\Pi(\tilde{x}_{i})))$

15: end while

16: Update $\gamma^{t+1}$ as the $95^{th}$ percentile erroneous sample
$APM_{C+1}^{t}$.

17:end for

Algorithm 1 MarginMatch

In our SSL formulation, we redefine the concept of margins to pseudo-margins
of unlabeled examples since no ground truth labels are available for the
unlabeled data. Let $c$ be the pseudo-label (or the argmax of the prediction,
i.e., $\hat{p}_{\theta}(y|\pi(\hat{x}))$) at iteration $t$ on unlabeled
example $\hat{x}$ after applying weak augmentations. We define the pseudo-
margin (PM) of $\hat{x}$ with respect to pseudo-label $c$ at iteration $t$ as
follows:

$\mbox{PM}_{c}^{t}(\hat{x})=z_{c}-max_{c!=i}(z_{i})\\\ $ (5)

where $z_{c}$ is the logit corresponding to the assigned pseudo-label $c$ and
$\max_{c!=i}(z_{i})$ is the largest other logit corresponding to a label $i$
different from $c$. To monitor the model’s predictions on $\hat{x}$ with
respect to pseudo-label $c$ from the beginning of training to iteration $t$,
we average all the margins with respect to $c$ from the first iteration until
$t$ and obtain the average pseudo-margin (APM) as follows:

$\mbox{APM}_{c}^{t}(\hat{x})=\frac{1}{t}\sum_{j=1}^{t}\mbox{PM}_{c}^{j}(\hat{x})$
(6)

Here $c$ acts as the “ground truth” label for the APM calculation. Note that
if at a prior iteration $t^{\prime}$, the assigned pseudo-label is different
from $c$ (say $c^{\prime}$), then the APM calculation at iteration
$t^{\prime}$ is done with respect to $c^{\prime}$ (by averaging all margins
with respect to $c^{\prime}$ from 1 to $t^{\prime}$). In practice, we maintain
a vector of pseudo-margins for all classes accumulated over the training
iterations and dynamically retrieve the accumulated pseudo-margin value of the
argmax class $c$ to obtain the $\mbox{APM}_{c}^{t}$ at iteration $t$.

Intuitively, if $c$ is the pseudo-label of $\hat{x}$ at iteration $t$, then
$\mbox{PM}_{c}^{t}$ with respect to class $c$ at iteration $t$ will be
positive. In contrast, if the argmax of the model prediction on $\hat{x}$ at a
previous iteration $t^{\prime}<t$ is different from $c$, then
$PM_{c}^{t^{\prime}}$ at $t^{\prime}$ with respect to $c$ will be negative.
Therefore, if over the iterations, the model predictions do not agree
frequently with the pseudo-label $c$ from iteration $t$ and the model
fluctuates significantly between iterations on the predicted label, the APM
for class $c$ will have a low, likely negative value. Similarly, if the model
is highly uncertain of the class of $\hat{x}$ (reflected in a high entropy of
the class probability distribution), the APM for class $c$ will have a low
value. These capture the characteristics of mislabeled examples or of those
harmful for training. Motivated by these observations, MarginMatch leverages
the APM of the assigned pseudo-label $c$ and compares it with an APM threshold
to mask out pseudo-labeled examples with low APMs. Formally, the unlabeled
loss in MarginMatch is:

$\vspace{-2mm}\mathcal{L}_{u}=\sum_{i=1}^{\nu
B}\mathbbm{1}(\mbox{APM}^{t}_{\hat{p}_{\theta}(y|\pi(\hat{x}_{i}))}(\hat{x}_{i})>\gamma^{t})\quad\times\\\
\quad\quad\quad\mathbbm{1}(\max(p_{\theta}(y|\pi(\hat{x}_{i})))>\mathcal{T}^{t}_{\hat{p}_{\theta}(y|\pi(\hat{x}_{i}))})\quad\times\\\
H(\hat{p}_{\theta}(y|\pi(\hat{x}_{i})),p_{\theta}(y|\Pi(\hat{x}_{i})))$ (7)

where $\gamma^{t}$ is the APM threshold at iteration $t$, estimated as
explained below, and $\mathcal{T}^{t}_{\hat{p}_{\theta}(y|\pi(\hat{x}_{i}))}$
is the flexible threshold estimated as in FlexMatch [49]. To train our model,
we adopt the best practices [49, 41] and optimize the weighted combination of
the supervised and unsupervised losses:

$\mathcal{L}=\mathcal{L}_{s}+\lambda\mathcal{L}_{u}$ (8)

where the supervised loss is given by:

$\mathcal{L}_{s}=\frac{1}{B}\sum_{i=1}^{B}H(y_{i},p_{\theta}(y|\pi(x_{i})))$
(9)

#### Average Pseudo-Margin Threshold Estimation

Inspired by Pleiss et al. [33], we propose to estimate the average pseudo-
margin threshold $\gamma^{t}$ by analyzing the training dynamics of a special
category of unlabeled examples, which we force to be _erroneous_ or mislabeled
examples. That is, to create the sample of _erroneous_ examples $E$, we
randomly sample a subset of unlabeled examples from $U$ that we assign to an
inexistent (or virtual) class $C+1$ at the beginning of the training process
and remove them from $U$. The purpose of these erroneous examples is to mimic
the training dynamics of incorrectly pseudo-labeled (unlabeled) examples and
use them as proxy to estimate the cutoff of (potentially) mislabeled pseudo-
labels. Since the examples in $E$ should belong to one of the $C$ original
classes, assigning them to the inexistent class $C+1$ makes them by definition
mislabeled (see Appendix A for additional insights into this virtual class).
As with all unlabeled examples from $U$, we compute $APM^{t}_{C+1}$ for the
special category of erroneous examples from $E$, but unlike the unlabeled
examples from $U$, the erroneous ones from $E$ have a fixed class $C+1$. To
mimic the training dynamics of unlabeled examples from $U$, we use strong
augmentations to compute the loss of the erroneous examples from $E$. That is,
given a batch $E_{b}$ of $B$ erroneous examples, the erroneous sample loss
becomes:

$\mathcal{L}_{e}=\sum_{i=1}^{B}H(C+1,p_{\theta}(y|\Pi(\tilde{x}_{i})))$ (10)

At iteration $t$, we use the APMs of the erroneous examples to choose the APM
threshold $\gamma^{t}$. We set $\gamma^{t}$ as the APM of the $95^{th}$
percentile erroneous sample. The total loss becomes:

$\mathcal{L}=\mathcal{L}_{s}+\lambda(\mathcal{L}_{u}+\mathcal{L}_{e})$ (11)

Our full MarginMatch algorithm is shown in Algorithm 1.

Dataset | CIFAR-10 | CIFAR-100 | SVHN | STL-10
---|---|---|---|---
#Labels/Class | $4$ | $25$ | $400$ | $4$ | $25$ | $100$ | $4$ | $25$ | $100$ | $4$ | $25$ | $100$
Pseuso-Labeling | $74.61_{0.26}$ | $46.49_{2.20}$ | $15.08_{0.19}$ | $87.45_{0.85}$ | $57.74_{0.28}$ | $36.55_{0.24}$ | $64.61_{5.60}$ | $25.21_{2.03}$ | $9.40_{0.32}$ | $74.68_{0.99}$ | $55.45_{2.43}$ | $32.64_{0.71}$
UDA | $10.79_{3.75}$ | $5.32_{0.06}$ | $4.41_{0.07}$ | $48.95_{1.59}$ | $29.43_{0.21}$ | $23.87_{0.23}$ | $5.34_{4.27}$ | $4.26_{0.39}$ | $1.95_{0.01}$ | $37.82_{8.44}$ | $9.81_{1.15}$ | $6.81_{0.17}$
MixMatch | $45.24_{2.15}$ | $12.76_{1.14}$ | $7.13_{0.34}$ | $62.15_{2.17}$ | $41.51_{1.19}$ | $28.16_{0.24}$ | $46.18_{1.78}$ | $3.98_{0.17}$ | $3.5_{0.13}$ | $34.15_{1.54}$ | $8.95_{0.32}$ | $10.41_{0.73}$
ReMixMatch | $5.27_{0.19}$ | $4.85_{0.13}$ | $4.04_{0.12}$ | $47.15_{0.76}$ | $27.14_{0.23}$ | $23.78_{0.12}$ | $4.23_{0.31}$ | $3.18_{0.04}$ | $1.94_{0.06}$ | $31.51_{0.75}$ | $8.54_{0.48}$ | $6.19_{0.24}$
FixMatch | $7.8_{0.28}$ | $4.91_{0.05}$ | $4.25_{0.08}$ | $48.21_{0.82}$ | $29.45_{0.16}$ | $22.89_{0.12}$ | $3.97_{1.18}$ | $\boldsymbol{3.13_{1.03}}$ | $1.97_{0.03}$ | $38.43_{4.14}$ | $10.45_{1.04}$ | $6.43_{0.33}$
FlexMatch | $5.04_{0.06}$ | $5.04_{0.09}$ | $4.19_{0.01}$ | $39.99_{1.62}$ | $26.96_{0.08}$ | $22.44_{0.15}$ | $8.19_{3.20}$ | $7.78_{2.55}$ | $6.72_{0.30}$ | $29.15_{1.32}$ | $8.23_{0.13}$ | $5.77_{0.12}$
MarginMatch | $\boldsymbol{4.91_{0.07}}$ | $\boldsymbol{4.73_{0.12}}$ | $\boldsymbol{3.98_{0.02}}$ | $\boldsymbol{36.97_{1.32}}$ | $\boldsymbol{23.71_{0.13}}$ | $\boldsymbol{21.39_{0.12}}$ | $\boldsymbol{3.75_{1.20}}$ | $3.14_{1.17}$ | $\boldsymbol{1.93_{0.01}}$ | $\boldsymbol{25.37_{3.58}}$ | $\boldsymbol{7.31_{0.35}}$ | $\boldsymbol{5.52_{0.15}}$

Table 1: Test error rates on CIFAR-10, CIFAR-100, SVHN, and STL-10 datasets.
Best results are shown in blue.

#### Exponential Moving Average of Pseudo-Margins

The current definition of APM weighs the pseudo-margin at iteration $t$
identical to the pseudo-margin at a much earlier iteration $p$ ($t>>p$). This
is problematic since very old pseudo-margins eventually become deprecated
(especially due to the large number of iterations through unlabeled data in
consistency training ($\sim 9K$)), and hence, the old margins are no longer
indicative of the current learning status of the model. To this end, instead
of averaging all pseudo-margins (from the beginning of training to the current
iteration), we propose to use an exponential moving average to place more
importance on recent iterations. Formally, APM becomes:

$\mbox{APM}^{t}_{c}(\hat{x})=\mbox{PM}_{c}^{t}(\hat{x})*\frac{\delta}{1+t}+\mbox{APM}^{t-1}_{c}(\hat{x})*(1-\frac{\delta}{1+t})$
(12)

We set the smoothing parameter $\delta$ to $0.997$ in experiments.

##  3 Experiments

We evaluate the performance of our MarginMatch on a wide range of SSL
benchmark datasets. Specifically, we perform experiments with various numbers
of labeled examples on CIFAR-10, CIFAR-100 [21], SVHN [31], STL-10 [8],
ImageNet [10], and WebVision [26]. For smaller scale datasets such as
CIFAR-10, CIFAR-100, SVHN, and STL-10 we randomly sample a small number of
labeled examples per class (ranging from $4$ labels per class to $400$ labels
per class) and treat them as labeled data, whereas the remaining labeled
examples are treated as unlabeled data, except for STL-10 [8], which provides
its own set of unlabeled examples. On ImageNet and WebVision, we use $\sim
10\%$ of the available labeled examples as labeled data, with the remaining
($90\%$) being treated as unlabeled data. In all our experiments, we sample
$5\%$ of the unlabeled data and place it in the set of erroneous examples.

We report the mean and standard deviation of error rates from five runs with
different parameter initializations. Similar to FixMatch [41], we use Wide
Residual Networks [48]: WRN-28-2 for CIFAR-10 and SVHN; WRN-28-8 for
CIFAR-100; and WRN-37-2 for STL-10. We use ResNet-50 [14] for both ImageNet
and WebVision.

In our experiments, we adopt the same hyperparameters as FixMatch [41].
Specifically, we use stochastic gradient descent (SGD) with a momentum of
$0.9$. We start with a learning rate of $0.03$ and employ a cosine learning
rate; at iteration $k$, our learning rate is $\eta(k)=cos(\frac{7k\pi}{16K})$,
where $K$ is the maximum number of iterations and is set to $2^{20}$. We also
leverage the same data augmentations as in FlexMatch [49]. Specifically, for
weak augmentations we employ a standard flip-and-shift augmentation and use
RandAugment [9] for strong augmentations. We set the batch size $B=64$, ratio
of unlabeled to labeled data in a batch to $\nu=7$ and weigh the supervised
and unsupervised losses equally (i.e., $\lambda=1$). We set our initial
confidence threshold to $\tau=0.95$ and our average pseudo-margin (APM)
threshold to the APM of the $95^{th}$ percentile threshold sample. To report
the error rates, we compare all the approaches using the model at the end of
training as in FixMatch [41].

01K2K3K4K5K6K7K8K9K$10$$20$$30$$40$$50$$60$$70$$80$$\\#Epochs$$ErrorRate(\%)$CIFAR-10FixMatchFlexMatchMarginMatch

01K2K3K4K5K6K7K8K9K$30$$40$$50$$60$$70$$80$$90$$100$$\\#Epochs$$ErrorRate(\%)$CIFAR-100FixMatchFlexMatchMarginMatch

01K2K3K4K5K6K7K8K9K$10$$20$$30$$40$$50$$60$$70$$\\#Epochs$$ErrorRate(\%)$SVHNFixMatchFlexMatchMarginMatch

01K2K3K4K5K6K7K8K9K$30$$40$$50$$60$$70$$80$$90$$100$$\\#Epochs$$ErrorRate(\%)$STL-10FixMatchFlexMatchMarginMatch

Figure 2: Convergence speed of MarginMatch against FixMatch and FlexMatch with
4 labels per class.

###  3.1 CIFAR-10, CIFAR-100, SVHN, and STL-10

We compare MarginMatch against strong baselines and prior works: Pseudo-
Labeling [1], Unsupervised Data Augmentation (UDA) [46], MixMatch [5],
ReMixMatch [4], FixMatch [41], and FlexMatch [49]. We show in Table 1 the
error rates obtained by our MarginMatch and the baselines on the CIFAR-10,
CIFAR-100, SVHN and STL-10 datasets. First, we observe that our approach
improves the performance on both CIFAR-10 and CIFAR-100. On CIFAR-10,
MarginMatch improves performance in all data regimes upon FlexMatch [49],
which is the current state-of-the-art, while mantaining a good error rate
standard deviation. On CIFAR-100, which is significantly more challenging than
CIFAR-10, we observe that MarginMatch bring substantially larger improvements.
Notably, we see $3.02\%$ improvement over FlexMatch in error rate using only
$4$ labels per class, and $3.25\%$ improvement using $25$ examples per class.
These results on CIFAR-100 emphasize the effectiveness of MarginMatch, which
performs well on a more challenging dataset.

On SVHN, our approach performs better than FixMatch using $4$ labels per class
and performs similarly with FixMatch using $25$ and $100$ labels per class.
However, on this dataset, MarginMatch performs much better compared with
FlexMatch. For example, MarginMatch achieves $3.75\%$ error rate using $4$
labels per class, whereas FlexMatch obtains an error rate of $8.19\%$ with the
same labels per class, yielding an improvement of MarginMatch of $4.44\%$ over
FlexMatch. We hypothesize that the low performance of FlexMatch is due to its
limitation in handling unbalanced class distributions [49]. On STL-10,
MarginMatch as well outperforms all the other approaches both in error rates
and error rate standard deviation. Notably, on this dataset, our approach
pushes the performance of FlexMatch by $3.78\%$ in error rate using only $4$
labels per class and by $0.92\%$ using $25$ labels per class.

Next, we compare MarginMatch with FixMatch and FlexMatch in terms of
convergence speed in the extremely label-scarce setting of 4 labels per class
and show these results in Figure 2. Notably, we observe that MarginMatch has a
similar convergence speed (or even better on CIFAR-100) compared with
FlexMatch while achieving a lower test error rate than FlexMatch on all
datasets with 4 labels per class (see Table 1). Even more strikingly, compared
with FixMatch, MarginMatch has a much superior convergence speed for a much
better test error rate with 4 labels per class. This is because the rigid
thresholds in FixMatch allow access only to a small amount of unlabeled data
for training at each iteration and it takes a lot longer for the model to
train.

Dataset | ImageNet | WebVision
---|---|---
| top-1 | top-5 | top-1 | top-5
Supervised | $48.39$ | $25.49$ | $49.58$ | $26.78$
FixMatch | $43.66$ | $21.80$ | $44.76$ | $22.65$
FlexMatch | $42.02$ | $19.49$ | $43.87$ | $22.07$
MarginMatch | $\boldsymbol{41.05}$ | $\boldsymbol{18.28}$ | $\boldsymbol{43.08}$ | $\boldsymbol{21.13}$

Table 2: Test error rates on the ImageNet and WebVision datasets. Best results
are shown in blue.

###  3.2 ImageNet and WebVision

To showcase the effectiveness of our approach in a large-scale setup, we test
our MarginMatch on ImageNet [10] and WebVision [26] using $10\%$ labeled
examples in total. We show the results obtained in Table 2. We observe that
our MarginMatch outperforms FixMatch and FlexMatch on both datasets. It is
worth noting that large-scale self-supervised approaches such as SimCLR [7]
achieve high performance on ImageNet but at a much higher computational cost.
MarginMatch outperforms other SSL methods using the same ResNet-50
architecture at the same computational cost. We emphasize MarginMatch is most
successful and relevant in low data regimes on smaller datasets.

##  4 Ablation Study

Exponential Moving Average Smoothing for APM Computation In our approach, we
employ an exponential moving average (EMA) of the pseudo-margin values with a
smoothing value of $\delta=0.997$ to compute the APM. We now analyze how our
approach performs with different EMA smoothing values or with no EMA at all.
Table 3 shows these results on CIFAR-100 with 4 labels per class. First, we
observe that employing a simple average of pseudo-margin values for the APM
computation (i.e., $\delta=1$) performs extremely poorly, obtaining a $39.72$%
error rate. This result emphasizes that margins eventually become deprecated
and it is essential to scale them down in time. Using a low smoothing factor
of $\delta=0.95$ is not effective either, denoting that abruptly forgetting
margin values does not work either. Our chosen $\delta=0.997$ strikes a
balance between the two by eliminating the harmful effects of very old margins
while keeping track of a good amount of previous estimates (e.g., a margin
value computed $200$ epochs previously is scaled down by $0.55$, while a
margin value computed $1000$ epochs previously is scaled by $0.05$).

$\delta$ | $0.95$ | $0.99$ | $0.995$ | $0.997$ | $0.999$ | $1$
---|---|---|---|---|---|---
err rate | $38.13$ | $38.05$ | $37.92$ | $\bf 37.91$ | $39.12$ | $39.72$

Table 3: Error rates obtained on CIFAR-100 with four examples per class and
various smoothing values $\delta$. Best result is in blue.

Dataset | CIFAR-10 | CIFAR-100 | SVHN | STL-10
---|---|---|---|---
#Labels/Class | $4$ | $25$ | $400$ | $4$ | $25$ | $100$ | $4$ | $25$ | $100$ | $4$ | $25$ | $100$
Avg Confidence | $23.87_{2.73}$ | $14.21_{1.37}$ | $7.54_{0.78}$ | $41.23_{2.15}$ | $31.49_{1.48}$ | $24.11_{2.36}$ | $8.99_{4.27}$ | $6.54_{0.39}$ | $4.73_{0.01}$ | $31.67_{8.44}$ | $14.87_{1.15}$ | $7.59_{0.17}$
Avg Entropy | $8.58_{0.41}$ | $6.18_{0.15}$ | $5.85_{0.12}$ | $45.10_{0.91}$ | $26.02_{1.11}$ | $22.13_{0.25}$ | $15.69_{1.25}$ | $12.74_{0.78}$ | $9.33_{0.05}$ | $29.54_{3.51}$ | $10.63_{1.35}$ | $10.84_{0.47}$
Avg Margin | $7.25_{0.29}$ | $5.38_{0.76}$ | $4.73_{0.09}$ | $39.72_{1.52}$ | $25.21_{0.52}$ | $23.18_{0.17}$ | $18.45_{1.36}$ | $11.29_{0.93}$ | $8.40_{0.04}$ | $28.45_{4.28}$ | $9.34_{1.34}$ | $7.59_{0.21}$
EMA Confidence | ${4.91_{0.45}}$ | $4.74_{0.09}$ | $3.99_{0.06}$ | $38.67_{0.74}$ | $25.61_{0.12}$ | $21.48_{0.17}$ | $3.84_{0.23}$ | $3.25_{0.03}$ | $1.93_{0.09}$ | $25.9_{0.81}$ | $7.6_{0.42}$ | $5.74_{0.57}$
EMA Entropy | $6.4_{0.43}$ | $8.34_{0.12}$ | $4.21_{0.09}$ | $41.63_{0.76}$ | $36.84_{0.13}$ | $22.52_{0.07}$ | $3.81_{1.26}$ | $3.17_{0.87}$ | $2.14_{0.04}$ | $27.21_{4.05}$ | $8.28_{1.01}$ | $6.79_{0.27}$
EMA Margin | $\boldsymbol{4.91_{0.07}}$ | $\boldsymbol{4.73_{0.12}}$ | $\boldsymbol{3.98_{0.02}}$ | $\boldsymbol{36.97_{1.32}}$ | $\boldsymbol{23.71_{0.13}}$ | $\boldsymbol{21.39_{0.12}}$ | $\boldsymbol{3.75_{1.20}}$ | $\boldsymbol{3.14_{1.17}}$ | $\boldsymbol{1.93_{0.01}}$ | $\boldsymbol{25.37_{3.58}}$ | $\boldsymbol{7.31_{0.35}}$ | $\boldsymbol{5.52_{0.15}}$

Table 4: Test error rates comparing pseudo-margin with confidence and entropy.
Best results are shown in blue.

Pseudo-Margin vs. Other Measures for Pseudo-Label Correctness Our MarginMatch
monitors the pseudo-margins of a model’s predictions across training
iterations to ensure the quality of pseudo-labels. However, other measures
such as confidence or entropy exist that can assess the pseudo-label
correctness. Hence, we perform an ablation where we replace the pseudo-margins
in our MarginMatch with average confidence and entropy and compare their
performance. Specifically, we design the following approaches: 1) Avg
Confidence monitors the confidence of the prediction for each unlabeled
example and takes the average over the training iterations; 2) Avg Entropy
monitors the entropy of the class probability distribution for each unlabeled
example and takes the average across the training iterations. In addition, we
also consider 3) EMA Confidence and 4) EMA Entropy which are similar to Avg
Confidence and Avg Entropy, respectively, but use an exponential moving
average (EMA) instead of the simple averaging. The estimation of the threshold
for each of these approaches is done in a similar manner as for pseudo-
margins, using erroneous samples and considering the value of the 95th
percentile erroneous sample as the threshold.

We show the results obtained using these approaches in Table 4. First, we
observe that all measures (pseudo-margin, confidence and entropy) with EMA
perform better than their counterpart with simple averaging. Second, EMA
Margin achieves the lowest test error rates compared with EMA Confidence and
EMA Entropy. Thus, we conclude that pseudo-margins provide an excellent
measure for pseudo-label correctness. See Appendix B for some additional
insights into why EMA Margin outperforms EMA confidence and entropy.

4.5K5K5.5K6K6.5K7K7.5K8K8.5K$20$$30$$\\#Epochs$$MaskRate(\%)$CIFAR-100FixMatchFlexMatchMarginMatch

4.5K5K5.5K6K6.5K7K7.5K8K8.5K$11$$12$$13$$14$$15$$16$$17$$18$$19$$20$$\\#Epochs$$Impurity(\%)$CIFAR-100FixMatchFlexMatchMarginMatch

Figure 3: Mask rate and impurity on CIFAR-100 with $4$ labeled examples per
class.

##  5 Analysis

###  5.1 Mask Rate and Impurity

We now contrast MarginMatch with FixMatch and FlexMatch in terms of the
quality of pseudo-labels using two metrics: mask rate and impurity and show
these results in Figure 3, respectively, using CIFAR-100 with $4$ labels per
class. Mask rate is defined as the fraction of pseudo-labeled examples that do
not participate in the training at epoch t due to confidence masking or
pseudo-margin masking (or both). Impurity in contrast is defined as the
fraction of pseudo-labeled examples that do participate in the training at
epoch t but with a wrong label. An effective SSL model minimizes both metrics:
a low mask rate indicates that the model has access to more unlabeled examples
during training (otherwise a low percentage and less diverse set of unlabeled
examples are seen during training) while low impurity indicates that the
pseudo-labels of these examples are of high quality. Note that we can compute
impurity on these two datasets because our unlabeled data comes from the
labeled training set of each of these datasets (thus we compare the pseudo-
labels against the gold labels of each dataset).

As can be seen from the figures, FixMatch has a significantly larger mask rate
due to the rigid confidence threshold set to a high value of $0.95$. In
contrast, FlexMatch lowers the mask rate by $5\%$ with the introduction of
flexible thresholds, but has a much higher impurity compared with FixMatch.
Notably, our MarginMatch has only a slightly higher mask rate compared with
FlexMatch and at the same time achieves a much lower impurity than FlexMatch
and even FixMatch despite that FixMatch employs a very high confidence
threshold. These results show that MarginMatch that enforces an additional
measure for pseudo-labeled data quality maintains a low mask rate without
compromising the quality of the pseudo-labels (i.e., low mask rate and low
impurity).

###  5.2 Anecdotal Evidence

We show in Figure 4 anecdotal evidence of the effectiveness of MarginMatch. To
this end, we extract two _bird_ images from our unlabeled portion of CIFAR-10
[21] of various learning difficulties that resemble characteristics of plane
images (e.g., the background). The top part of the figure illustrates the
progression over the training iterations of the confidence and the confidence
thresholds of FlexMatch for the classes _bird_ and _plane_ , whereas the
bottom part of the figure illustrates the progression of the APM threshold of
MarginMatch along with its APMs of _bird_ and _plane_ classes over the
training iterations. In the rightmost image, for MarginMatch we can observe
that the APM of the bird class becomes stronger and stronger as the training
progresses and eventually exceeds the APM threshold of MarginMatch, and hence,
the image is included in the training until the end with the correct bird
label. Interestingly, for the same image, in FlexMatch the confidence for the
bird class is very close to the bird confidence threshold and eventually falls
below this threshold and exits the training set.

In contrast, the leftmost image is significantly more challenging than the
rightmost image since it is more similar to _plane_ images, which makes it an
easily confusable example. Here, we observe that the confidence of FlexMatch
exceeds the flexible threshold with the incorrect argmax class _plane_
starting from iteration $3000$. Moreover, Flexmatch continues to use this
image with the wrong _plane_ label for the remaining of the training process.
Critically, in MarginMatch the APM value for the _plane_ class does not exceed
the APM threshold, and the model eventually learns to classify this image
correctly and includes it in training with the correct _bird_ pseudo-label.

01K2K3K4K5K6K7K8K9K$-1$$-0.5$$0$$0.5$$1$$\\#Epochs$$APM$‘Bird‘ and ‘Plane‘ APM
and APM threshold‘Plane‘ APM‘Bird‘ APMAPM
Threshold01K2K3K4K5K6K7K8K9K$-1$$-0.5$$0$$0.5$$1$$\\#Epochs$$APM$‘Bird‘ and
‘Plane‘ APM and APM threshold‘Plane‘ APM‘Bird‘ APMAPM
Threshold01K2K3K4K5K6K7K8K9K$0.25$$0.5$$0.75$$1$$\\#Epochs$$Confidence$‘Bird‘
and ‘Plane‘ confidence and confidence thresholds‘Plane‘ Confidence‘Bird‘
Confidence‘Bird‘ Confidence Thrs‘Plane‘ Confidence
Thrs01K2K3K4K5K6K7K8K9K$0.25$$0.5$$0.75$$1$$\\#Epochs$$Confidence$‘Bird‘ and
‘Plane‘ confidence and confidence thresholds‘Plane‘ Confidence‘Bird‘
Confidence‘Bird‘ Confidence Thrs‘Plane‘ Confidence Thrs FlexMatch MarginMatch
Figure 4: Confidence thresholding vs. APM Thresholding on two images from the
CIFAR-10 dataset.

##  6 Related Work

Here, we focus on various SSL approaches that our MarginMatch directly builds
upon although there are other SSL techniques that are not presented in this
review, such as approaches based on generative models [24, 17], graph-based
approaches [19, 20] and robust SSL [32, 37] (see Appendix C for a comparison
between MarginMatch and Robust SSL approaches).

Self-training [47, 40, 28, 36] is a popular SSL method where the predictions
of a model on unlabeled data are used as artificial labels to train against.
Noisy student training [47] is a popular self-training approach that also
leverages knowledge distillation [16] and iteratively jointly trains two
models in a teacher-student framework. Noisy student uses a larger model size
and noised inputs, exposing the student to more difficult learning
environments, leading to an increased performance compared to the teacher.

Pseudo-labeling is a variant of self-training where these predictions are
sharpened to obtain hard labels [25]. The use of hard labels can be seen as a
means of entropy minimization [12] and nowadays is a valuable component in
most successful SSL approaches [5, 4, 49]. These hard labels are usually used
along a confidence threshold, where unconfident unlabeled examples are
completely disregarded (e.g., [5]) to avoid using noisy pseudo-labels.
Recently, approaches such as Curriculum Labeling (CL) [6] or FlexMatch [49]
started to explore curriculum learning in the SSL context. CL proposes a self-
pacing strategy of identifying easy and hard examples to ensure that the model
first uses easy and progressively moves towards harder examples. Similarly,
MarginMatch uses curriculum learning and pseudo-labeling, but the focus of our
approach is placed on producing better thresholds for assessing the quality of
pseudo-labels.

Consistency regularization [2] is a method that applies random perturbations
when generating the artificial label, such as data augmentation [5, 4, 41],
dropout [39], or adversarial perturbations [29]. Current state-of-the-art
approaches [41, 49] exploit a combination of weak and strong data
augmentations, which were shown to be extremely beneficial in SSL. The most
popular strong augmentations used in the SSL literature are RandAugment [9]
and CTAugment [4]. The approaches based on these methods first generate a hard
label using pseudo-labeling on a weakly augmented image (i.e., using a low
noise transformation such as a flip-and-shift augmentation), then optimize the
predictions of the model on a strongly augmented version of the same image
towards this hard label. Similar to these approaches, MarginMatch uses the
same combination of weak and strong data augmentations.

##  7 Conclusion

In this paper, we proposed a novel semi-supervised learning method that
improves the pseudo-label quality using training dynamics. Our new method is
lightweight and achieves state-of-the-art performance on four computer vision
SSL datasets in low data regimes and on two large-scale benchmarks.
MarginMatch takes into consideration not only a flexible confidence threshold
to account for the difficulty of each class, but also a measure of quality for
each unlabeled example using training dynamics. In addition, MarginMatch is a
general approach that can be leveraged in most SSL frameworks and we hope that
it can attract future research in analyzing the effectiveness of SSL
approaches focused on data quality. As future work, we aim to further explore
our method in settings when there is a mismatch between the labeled and
unlabeled data distributions (i.e., making use of out-of-domain unlabeled
data).

## References

  * [1] Eric Arazo, Diego Ortego, Paul Albert, Noel E. O’Connor, and Kevin McGuinness.  Pseudo-labeling and confirmation bias in deep semi-supervised learning.  CoRR, abs/1908.02983, 2019.
  * [2] Philip Bachman, Ouais Alsharif, and Doina Precup.  Learning with pseudo-ensembles.  In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
  * [3] Peter L. Bartlett, Dylan J. Foster, and Matus Telgarsky.  Spectrally-normalized margin bounds for neural networks.  CoRR, abs/1706.08498, 2017.
  * [4] David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel.  Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring.  arXiv preprint arXiv:1911.09785, 2019.
  * [5] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel.  Mixmatch: A holistic approach to semi-supervised learning.  arXiv preprint arXiv:1905.02249, 2019.
  * [6] Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi, and Vicente Ordonez.  Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning.  In AAAI, 2021.
  * [7] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In International conference on machine learning, pages 1597–1607. PMLR, 2020.
  * [8] Adam Coates, Andrew Ng, and Honglak Lee.  An analysis of single-layer networks in unsupervised feature learning.  In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011.
  * [9] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le.  Randaugment: Practical automated data augmentation with a reduced search space.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 702–703, 2020.
  * [10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [11] Gamaleldin Fathy Elsayed, Dilip Krishnan, Hossein Mobahi, Kevin Regan, and Samy Bengio.  Large margin deep networks for classification.  2018\.
  * [12] Yves Grandvalet and Yoshua Bengio.  Semi-supervised learning by entropy minimization.  In L. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems, volume 17. MIT Press, 2004.
  * [13] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger.  On calibration of modern neural networks.  In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1321–1330. PMLR, 06–11 Aug 2017\.
  * [14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [15] Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Patwary, Mostofa Ali, Yang Yang, and Yanqi Zhou.  Deep learning scaling is predictable, empirically.  arXiv preprint arXiv:1712.00409, 2017.
  * [16] Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean.  Distilling the knowledge in a neural network.  In NIPS Deep Learning and Representation Learning Workshop, 2015\.
  * [17] Geoffrey E Hinton and Russ R Salakhutdinov.  Using deep belief nets to learn covariance kernels for gaussian processes.  In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007.
  * [18] Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio.  Predicting the generalization gap in deep networks with margin distributions, 2018.
  * [19] Thorsten Joachims.  Transductive inference for text classification using support vector machines.  In Proceedings of the Sixteenth International Conference on Machine Learning, ICML ’99, page 200–209, San Francisco, CA, USA, 1999. Morgan Kaufmann Publishers Inc.
  * [20] Thorsten Joachims.  Transductive learning via spectral graph partitioning.  In Proceedings of the Twentieth International Conference on International Conference on Machine Learning, ICML’03, page 290–297. AAAI Press, 2003.
  * [21] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [22] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton.  Imagenet classification with deep convolutional neural networks.  In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, NIPS’12, page 1097–1105, Red Hook, NY, USA, 2012. Curran Associates Inc.
  * [23] Samuli Laine and Timo Aila.  Temporal ensembling for semi-supervised learning.  In ICLR (Poster). OpenReview.net, 2017.
  * [24] J.A. Lasserre, C.M. Bishop, and T.P. Minka.  Principled hybrids of generative and discriminative models.  In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 1, pages 87–94, 2006.
  * [25] Dong-Hyun Lee et al.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.  In Workshop on challenges in representation learning, ICML, volume 3, page 896, 2013.
  * [26] Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool.  Webvision database: Visual learning and understanding from web data.  CoRR, abs/1708.02862, 2017.
  * [27] Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten.  Exploring the limits of weakly supervised pretraining.  In Proceedings of the European conference on computer vision (ECCV), pages 181–196, 2018.
  * [28] David McClosky, Eugene Charniak, and Mark Johnson.  Effective self-training for parsing.  In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 152–159, New York City, USA, June 2006. Association for Computational Linguistics.
  * [29] Takeru Miyato, Andrew M Dai, and Ian Goodfellow.  Adversarial training methods for semi-supervised text classification.  arXiv preprint arXiv:1605.07725, 2016.
  * [30] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii.  Virtual adversarial training: a regularization method for supervised and semi-supervised learning.  IEEE transactions on pattern analysis and machine intelligence, 41(8):1979–1993, 2018.
  * [31] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng.  Reading digits in natural images with unsupervised feature learning.  2011\.
  * [32] Jongjin Park, Sukmin Yun, Jongheon Jeong, and Jinwoo Shin.  Opencos: Contrastive semi-supervised learning for handling open-set unlabeled data.  In ECCV Workshops, 2021.
  * [33] Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinberger.  Identifying mislabeled data using the area under the margin ranking.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 17044–17056. Curran Associates, Inc., 2020.
  * [34] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.  Language models are unsupervised multitask learners.  OpenAI blog, 1(8):9, 2019.
  * [35] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  arXiv preprint arXiv:1910.10683, 2019.
  * [36] Chuck Rosenberg, Martial Hebert, and Henry Schneiderman.  Semi-supervised self-training of object detection models.  In 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION’05) - Volume 1, volume 1, pages 29–36, 2005.
  * [37] Kuniaki Saito, Donghyun Kim, and Kate Saenko.  Openmatch: Open-set semi-supervised learning with open-set consistency regularization.  In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 25956–25967. Curran Associates, Inc., 2021.
  * [38] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen.  Mutual exclusivity loss for semi-supervised deep learning.  In 2016 IEEE International Conference on Image Processing (ICIP), pages 1908–1912. IEEE, 2016.
  * [39] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen.  Regularization with stochastic transformations and perturbations for deep semi-supervised learning.  Advances in neural information processing systems, 29:1163–1171, 2016.
  * [40] H. Scudder.  Probability of error of some adaptive pattern-recognition machines.  IEEE Transactions on Information Theory, 11(3):363–371, 1965.
  * [41] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li.  Fixmatch: Simplifying semi-supervised learning with consistency and confidence.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 596–608. Curran Associates, Inc., 2020.
  * [42] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015.
  * [43] Mingxing Tan and Quoc Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In International Conference on Machine Learning, pages 6105–6114. PMLR, 2019.
  * [44] Antti Tarvainen and Harri Valpola.  Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results.  In Advances in neural information processing systems, pages 1195–1204, 2017.
  * [45] Vikas Verma, Kenji Kawaguchi, Alex Lamb, Juho Kannala, Arno Solin, Yoshua Bengio, and David Lopez-Paz.  Interpolation consistency training for semi-supervised learning.  Neural Networks, 2021.
  * [46] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le.  Unsupervised data augmentation for consistency training.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6256–6268. Curran Associates, Inc., 2020.
  * [47] Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le.  Self-training with noisy student improves imagenet classification.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10687–10698, 2020.
  * [48] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks, 2017.
  * [49] Bowen Zhang, Yidong Wang, Wenxin Hou, HAO WU, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki.  Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling.  In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 18408–18419. Curran Associates, Inc., 2021.
  * [50] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals.  Understanding deep learning requires rethinking generalization.  CoRR, abs/1611.03530, 2016.


# The families of Hamiltonians sharing common symmetry structure

C. Gonera e-mail<EMAIL_ADDRESS>Faculty of Physics and Applied
Informatics, University of Lodz, Pomorska 149/153, 90-236 Łódź, Poland. J.
Gonera e-mail<EMAIL_ADDRESS>Faculty of Physics and Applied
Informatics, University of Lodz, Pomorska 149/153, 90-236 Łódź, Poland. A.
Jasiński  Faculty of Physics and Applied Informatics, University of Lodz,
Pomorska 149/153, 90-236 Łódź, Poland. P. Kosiński e-mail:
<EMAIL_ADDRESS>Faculty of Physics and Applied Informatics,
University of Lodz, Pomorska 149/153, 90-236 Łódź, Poland.

###### Abstract

We describe a general procedure which allows to construct, starting from a
given Hamiltonian, the whole family of new ones sharing the same set of
unparameterized trajectories in phase space. The symmetry structure of this
family can be completely characterized provided the symmetries of initial
Hamiltonian are known. Our approach covers numerous examples considered in
literature. It provides a simple proof of Darboux theorem and Hietarinta et
al. coupling-constant metamorphosis method.

##  I Introduction

Hamiltonian dynamical systems play an important role both in physics and
mathematics. In the literature one encounters many examples which are
interesting due to their relevance to physics and/or intriguing mathematical
properties. It happens quite often that the systems defined by different
Hamiltonians share the same unparameterized trajectories in phase space. They
are, however, related in rather nontrivial way since the relevant time
reparameterization varies, in general, from trajectory to trajectory, i.e. is
a function of canonical variables. The Hamiltonians sharing unparameterized
phase space trajectories admit the same set of integrals of motion which do
not depend explicitly on time. This implies that they are to the same extent
integrable. In particular, they are (or not) simultaneously (super)integrable.
The classical example of such systems is provided by Darboux theorem [1], [2].
Other very interesting examples have been constructed by the related method
proposed by Hietarinta et.al. [3] under the name of coupling-constant
metamorphosis. It allowed, for example, to relate the Henon-Heiles [4] and
Holt [5, 6] Hamiltonians.

In the present paper we propose a fairly general method of constructing,
starting from a given Hamiltonian, a whole family of related Hamiltonians
sharing the same set of unparameterized phase space trajectories. It provides
also an explicit relation between the Poisson algebras of integrals of motion.
One can say that our approach yields a complete description of the symmetry
structure of the resulting Hamiltonians provided the symmetry of the initial
one is known. The disadvantage of the method is that, in general, the final
Hamiltonian doesn’t have the natural form even though we start with the
natural one (the natural Hamiltonian is a sum of a non-degenerate quadratic
form in momenta plus a function of generalized coordinates only). However, in
many cases one obtains the Hamiltonian corresponding to the metric in
configuration space conformally equivalent to that of the initial Hamiltonian.
The method proposed generalizes the Darboux and Hietarinta et al. approaches
as well as it allows to describe and generalize many examples of dynamical
systems encountered in the literature.

The paper is organized as follows. In Section II the general formalism is
presented while Section III is devoted to some examples discussed recently in
the literature [7, 8, 9, 10, 11, 12, 13, 14]. Darboux and Hietarinta
approaches are considered in Sec. IV. Finally, Sec. V is devoted to some
conclusions.

##  II General scheme

Assume we have a Hamiltonian system defined in 2N-dimensional phase space. The
relevant Hamiltonian reads:

$H=H({\bf q},{\bf p},{\boldsymbol{\lambda}})$ (1)

where ${\bf q}\equiv(q_{1},...,q_{N})$, ${\bf p}\equiv(p_{1},...,p_{N})$ are
canonical variables while
${\boldsymbol{\lambda}}\equiv(\lambda_{1},...,\lambda_{M})$ \- the set of
parameters (masses, coupling constants, frequencies etc.) characterizing our
system. One defines the new Hamiltonian $\tilde{H}({\bf{q},{p}})$ as follows.
Let $\tilde{H}$ be a new variable; replace the parameters $\lambda_{\alpha}$
by the functions

$\lambda_{\alpha}=\lambda_{\alpha}(\tilde{H}).$ (2)

Assume that they are chosen in such a way that the equation

$\tilde{H}({\bf{q},p})=H({\bf{q},p},{\boldsymbol{\lambda}}(\tilde{H}({\bf{q},p}))$
(3)

has the unique solution $\tilde{H}(\bf{q},\bf{p})$. In particular,

$\Omega({\bf q,p})\equiv 1-\frac{\partial
H}{\partial\lambda_{\alpha}}\frac{\partial\lambda_{\alpha}}{\partial\tilde{H}}\neq
0$ (4)

$\tilde{H}({\bf q,p})$ defines new dynamics on our phase space. Obviously, we
may assume that the functions $\lambda_{\alpha}$, eq.(2), still depend on some
parameters, $\lambda_{\alpha}=\lambda_{\alpha}(\tilde{H},\mu)$. Both
Hamiltonian dynamics, generated by $H$ and $\tilde{H}$, are closely related.
One finds:

$\begin{split}&\frac{\partial H}{\partial
q_{i}}=\Omega({\bf{q},p})\frac{\partial\tilde{H}}{\partial q_{i}}\\\
&\frac{\partial H}{\partial
p_{i}}=\Omega({\bf{q},p})\frac{\partial\tilde{H}}{\partial p_{i}}\end{split}$
(5)

Therefore,

$\begin{split}&\odv{q_{i}}{t}=\pdv{H}{p_{i}}=\Omega({\bf
q,p})\pdv{\tilde{H}}{p_{i}}=\Omega({\bf q,p})\odv{q_{i}}{\tilde{t}}\\\
&\odv{p_{i}}{t}=-\pdv{H}{q_{i}}=-\Omega({\bf
q,p})\pdv{\tilde{H}}{q_{i}}=\Omega({\bf
q,p})\odv{p_{i}}{\tilde{t}};\end{split}$ (6)

here $t\left(\tilde{t}\right)$ is the time parameter for the dynamics
generated by $H(\tilde{H})$. Let ${\bf q=\tilde{q}}\left(\tilde{t}\right)$,
${\bf p=\tilde{p}}\left(\tilde{t}\right)$ be any particular solution to the
Hamiltonian equations for $\tilde{H}$, corresponding to some initial data (in
particular, energy $\tilde{E}$). Then $\Omega$ becomes a function of time
$\tilde{t}$ only, $\Omega=\Omega(\tilde{t})$. Let us relate $t$ to $\tilde{t}$
by the equation

$\odv{\tilde{t}}{t}=\Omega.$ (7)

By comparing eqs.(6) and(7) we conclude that

$\begin{split}&q_{i}=\tilde{q}_{i}(\tilde{t}(t))\\\
&p_{i}=\tilde{p}_{i}(\tilde{t}(t)),\quad\quad i=1,2,...,N\end{split}$ (8)

are the solutions to the Hamiltonian equations for the Hamiltonian

$H=H({\bf q,p},{\boldsymbol{\lambda}}(\tilde{E}))$ (9)

Obviously, this works in the reverse direction as well.

It follows then that the set of all trajectories in phase space generated by
$\tilde{H}({\bf q,p})$ coincides with that for $H({\bf
q,p},{\boldsymbol{\lambda}}(\tilde{E}))$; they differ only by time
parameterization (this difference varies from trajectory to trajectory).
Therefore, the (super)integrability of $H$ implies the (super)integrability of
$\tilde{H}$ and reverse.
Moreover, one can easily relate the algebras of integrals of motion for both
dynamical systems. For any function on phase space,

$F\equiv F({\bf q,p};{\boldsymbol{\lambda}})$ (10)

define

$\tilde{F}\equiv F({\bf q,p};{\boldsymbol{\lambda}}(\tilde{H}({\bf q,p}))).$
(11)

Then

$\begin{split}&\pdv{\tilde{F}}{q_{i}}=\pdv{F}{q_{i}}+\pdv{F}{\lambda_{\alpha}}\pdv{\lambda_{\alpha}}{\tilde{H}}\pdv{\tilde{H}}{q_{i}}\\\
&\pdv{\tilde{F}}{p_{i}}=\pdv{F}{p_{i}}+\pdv{F}{\lambda_{\alpha}}\pdv{\lambda_{\alpha}}{\tilde{H}}\pdv{\tilde{H}}{p_{i}}.\end{split}$
(12)

In particular, let

$C=C({\bf q,p},{\boldsymbol{\lambda}})$ (13)

be any integral of motion of the dynamics generated by H. Then one easily
finds

$\\{\tilde{C},\tilde{H}\\}=0$ (14)

implying that $\tilde{C}$ is also an integral of motion; this follows also
directly from the coincidence of trajectories for both Hamiltonians.

Moreover, for any two integrals of motion we have

$\\{\tilde{C}_{a},\tilde{C}_{b}\\}=\widetilde{\\{C_{a},C_{b}\\}}$ (15)

If $\\{C_{a}\\}$ is the set of all functionally independent integrals of
motion for H, then

$\\{C_{a},C_{b}\\}=F_{ab}({\bf C},{\boldsymbol{\lambda}})$ (16)

with $F_{a,b}$ being some functions. Equations (15) implies

$\\{\tilde{C}_{a},\tilde{C}_{b}\\}=F_{ab}({\bf\tilde{C}},\tilde{\boldsymbol{\lambda}}(\tilde{H}))$
(17)

Concluding, the algebras of symmetries for both systems are closely related.

##  III Examples

Consider the three-dimensional isotropic harmonic oscillator

$H=\frac{1}{2}\left(\vec{p}\,^{2}+\omega^{2}\vec{q}\,^{2}\right).$ (18)

It is superintegrable. To reveal the structure of the algebra of integrals of
motion define the classical counterparts of creation/annihilation operators:

$a_{i}\equiv\frac{1}{\sqrt{2\omega}}\left(p_{i}-\imath\omega
q_{i}\right),\qquad i=1,2,3$ (19)

Then

$\\{a_{i},\bar{a}_{j}\\}=-\imath\delta_{ij}.$ (20)

Further, let $\Lambda_{a}$ , $a=1,2,...,8$ be the Gell-Mann matrices obeying
$SU(3)$ Lie algebra commutation rules

$[\Lambda_{a},\Lambda_{b}]=\imath{d_{ab}}^{c}\Lambda_{c}$ (21)

Then the functions

$C_{a}\equiv\frac{1}{2}a_{i}(\Lambda_{a})_{ij}\overline{a}_{j}$ (22)

are real integrals of motion obeying

$\\{C_{a},C_{b}\\}={d_{ab}}^{c}C_{c}.$ (23)

Let us now modify the parameter $\omega^{2}$:

$\omega^{2}(\tilde{H})\equiv\omega^{2}-2\mu\tilde{H},\qquad\mu\in R$ (24)

Following the prescription of the previous section we find the Hamiltonian [7,
8]

$\tilde{H}=\frac{1}{2}\frac{\vec{p}\,^{2}+\omega^{2}\vec{q}\,^{2}}{1+\mu\vec{q}\,^{2}}$
(25)

For $\mu\geq 0$ $\tilde{H}$ is non-singular everywhere; for $\mu<0$ some care
must be exercised. The function $\Omega({\bf q,p})$, eq.(4), takes the form

$\Omega({\bf q,p})=1+\mu{\vec{q}}\,^{2}$ (26)

For simplicity consider the case $\mu\geq 0$. Assuming
$\tilde{E}<\displaystyle{\frac{\omega^{2}}{2\mu}}$ we have
$\omega^{2}(\tilde{H})>0$ (bounded motion). Eqs.(19), (21) are replaced by

$\tilde{a}_{i}=\frac{1}{\sqrt{2\omega(\tilde{H})}}\left(p_{i}-\imath\omega(\tilde{H})q_{i}\right),\quad
i=1,2,3$ (27)

$\tilde{C}_{a}=\frac{1}{2}\tilde{a}_{i}\left(\Lambda_{a}\right)_{ij}\overline{\tilde{a}}_{j}$
(28)

According to the discussion of Section III $\tilde{C}_{a}$ are integrals of
motion obeying $SU(3)$ algebra

$\\{\tilde{C}_{a},\tilde{C}_{b}\\}={d_{ab}}^{c}\tilde{C}_{c}.$ (29)

Let us note that eq.(20) is no longer valid.

For $\tilde{E}=\displaystyle{\frac{\omega^{2}}{2\mu}}$ (
$\tilde{E}>\displaystyle{\frac{\omega^{2}}{2\mu}}$) the relevant symmetry
algebra gets modified appropriately.

Let us now add the linear term to harmonic potential,

$H=\frac{1}{2}\left(\vec{p}\,^{2}+\omega^{2}\vec{q}\,^{2}\right)+\vec{k}\cdot\vec{q}$
(30)

with $\vec{k}$ being a constant vector.
By making the canonical transformation

$\begin{split}&\vec{Q}=\vec{q}+\frac{\vec{k}}{\omega^{2}}\\\
&\vec{P}=\vec{p}\end{split}$ (31)

$H$ reduces to harmonic oscillator:

$H=\frac{1}{2}\left(\vec{P}\,^{2}+\omega^{2}\vec{Q}\,^{2}\right)+\text{const}$
(32)

Therefore, the superintegrability is preserved together with the structure of
the symmetry algebra.
Making the substitution (24) one obtains the Hamiltonian studied in [9]

$\tilde{H}=\frac{1}{1+\mu\vec{q}\,^{2}}\left(\frac{1}{2}\left(\vec{p}\,^{2}+\omega^{2}\vec{q}\,^{2}\right)+\vec{k}\cdot\vec{q}\right)$
(33)

Again, assuming $\tilde{E}<\displaystyle{\frac{\omega^{2}}{2\mu}}$ and
defining

$a_{i}=\frac{1}{\sqrt{2\omega(\tilde{H})}}\left(p_{i}-\imath\omega(\tilde{H})q_{i}-\frac{\imath
k_{i}}{\omega(\tilde{H})}\right),\quad i=1,2,3$ (34)

we conclude that the integrals $\tilde{C}_{a}$, eq.(28), obey $SU(3)$ algebra.
The price one has to pay for having $SU(3)$ algebra as the symmetry algebra is
that the integrals $\tilde{C}_{a}$ are not polynomial in momenta. However,
when multiplied by $\omega^{3}(\tilde{H})$ they become polynomials; the
modified integrals of motion obey then the Poisson commutation rules with
$\tilde{H}$-dependent structure constants.

As the next example consider the generalization of Smorodinsky-Winternitz
system[15, 16, 17, 18, 19, 20, 21, 10, 11]. The relevant Hamiltonian reads

$H=\frac{1}{2}\left(\vec{p}\,^{2}+\sum_{i=1}^{3}\omega^{2}_{i}q^{2}_{i}\right)+\sum_{i=1}^{3}\frac{k_{i}}{q^{2}_{i}}$
(35)

with $k_{i}>0$, $i=1,2,3$. $H$ is obviously integrable being separable in
Cartesian coordinates. It is straightforward to find the action variables
$I_{k}$, $k=1,2,3$; in terms of them the Hamiltonian takes the form

$H=2\sum_{k=1}^{3}\omega_{k}I_{k}+\sum_{k=1}^{3}\sqrt{2k}\omega_{k}$ (36)

Assuming all ratios $\displaystyle{\frac{\omega_{i}}{\omega_{j}}}$ rational we
conclude that it is maximally superintegrable [22]. Putting
$\omega_{k}=n_{k}\omega$, with $n_{k}$ natural, one can write

$H=\frac{1}{2}\left(\vec{p}\,^{2}+\omega^{2}\sum_{k=1}^{3}n^{2}_{k}q^{2}_{k}\right)+\sum_{i=1}^{3}\frac{k_{i}}{q^{2}_{i}}$
(37)

The integrals of motion can be chosen to obey Lie algebra commutation rules
with respect to Poisson brackets [23]; it is again $SU(3)$ algebra.
Let us now replace $\omega^{2}\rightarrow\omega^{2}-2\mu\tilde{H}$. Then

$\tilde{H}=\frac{1}{\left(1+\mu\displaystyle{\sum_{k=1}^{3}n_{k}^{2}q_{k}^{2}}\right)}\left(\frac{1}{2}\vec{p}\,^{2}+\frac{\omega^{2}}{2}\sum_{k=1}^{3}n_{k}^{2}q^{2}_{k}+\sum_{i=1}^{3}\frac{k_{i}}{q^{2}_{i}}\right)$
(38)

For regularity we assume $\mu>0$. According to the discussion of the previous
section $\tilde{H}$ is superintegrable. Assuming $\omega^{2}-2\mu\tilde{E}>0$
we find also that the integrals of motion can be chosen in such a way as to
obtain $SU(3)$ algebra. The particular case of $\tilde{H}$, $n_{1}=n_{2}=1$,
$n_{3}=2$ has been considered in [9].

It is instructive to analyze in more detail the integrals of motion for the
Hamiltonian (37) (and, consequently, the Hamiltonian(38)). The angle variables
for the dynamics (37) read

$\begin{split}\cos{\varphi_{i}}=\frac{E_{i}-n^{2}_{i}\omega^{2}q_{i}^{2}}{\sqrt{E_{i}^{2}-2k_{i}n_{i}^{2}\omega^{2}}}\\\
\sin{\varphi_{i}}=\frac{E_{i}-n^{2}_{i}\omega^{2}p_{i}^{2}}{\sqrt{E_{i}^{2}-2k_{i}n_{i}^{2}\omega^{2}}}\end{split}$
(39)

with $E_{i}=2\omega n_{i}I_{i}+\sqrt{2k_{i}}\omega n_{i}$ being partial
energies. It is easy to write out the general form of the integral of motion
[14]

$C=\sum_{\\{m\\}}F_{\\{m\\}}({\bf{I}})\prod_{k=1}^{3}\left(e^{\imath\varphi_{k}}\right)^{m_{k}}$
(40)

with $\\{m\\}\equiv\\{m_{1},m_{2},m_{3}\\}$ being arbitrary triples of
integers obeying

$m_{1}n_{1}+m_{2}n_{2}+m_{3}n_{3}=0$ (41)

and $F_{\\{m\\}}(\bf{I})$ \- arbitrary functions. Obviously, only 5 of these
integrals can be chosen as functionally independent. It is also clearly seen
from eqs. (39),(40) that one can choose the integrals as polynomials in
momenta. For example, for $n_{1}=n_{2}=1$, $n_{3}=2$, the quadratic integral
$K_{4}$ in [9] may be written as

$\omega^{2}K_{4}=E_{1}\cdot
E_{2}-\sqrt{E^{2}_{1}-2k_{1}\omega^{2}}\sqrt{E^{2}_{2}-2k_{2}\omega^{2}}\cos{(\varphi_{1}-\varphi_{2})}$
(42)

Now, for $\omega^{2}-2\mu\tilde{E}>0$ the relevant integrals $\tilde{C}$ for
Hamiltonian $\tilde{H}$ can be obtained along the lines sketched in previous
section.

As a next example consider Kepler-related Hamiltonians. The initial
Hamiltonian for the textbook Kepler problem reads

$H=\frac{1}{2}\vec{p}\,^{2}+\frac{\alpha}{|\vec{q}\,^{2}|},\quad\alpha<0$ (43)

We make the replacement $\alpha\rightarrow\alpha+\beta\tilde{H}$ which yields

$\tilde{H}=\left(\frac{|\vec{q}\,|}{|\vec{q}\,|-\beta}\right)\left(\frac{1}{2}\vec{p}\,^{2}+\frac{\alpha}{|\vec{q}\,|}\right)$
(44)

For regularity we assume $\beta<0$. $\tilde{H}$ is superintegrable. Except
energy and angular momentum it admits further integrals of motion in the form
of generalized Runge-Lenz vector

$\tilde{\vec{A}}=\vec{A}-\frac{m\beta\tilde{H}\vec{q}}{|\vec{q}\,|^{2}}$ (45)

where $\vec{A}$ is a standard Runge-Lenz vector.

The Kepler dynamics can be generalized in a way still preserving
superintegrability [10],[11],[12]:

$H=\frac{1}{2}\vec{p}\,^{2}+\frac{\alpha}{|\vec{q}\,|}+\sum_{i=1}^{3}\frac{k_{i}}{q^{2}_{i}}$
(46)

Under the same substitution $\alpha\rightarrow\alpha+\beta\tilde{H}$, one
finds

$\tilde{H}=\left(\frac{|\vec{q}\,|}{|\vec{q}\,|-\beta}\right)\left(\frac{1}{2}\vec{p}\,^{2}+\frac{\alpha}{|\vec{q}\,|}+\sum_{i=1}^{3}\frac{k_{i}}{q^{2}_{i}}\right)$
(47)

It defines, according to our findings, the superintegrable dynamics. For a
more detailed discussion of its properties we refer the reader to [9]. The
integrals of motion described there can be derived using the technique
introduced in the present paper.

Both harmonic oscillator and Kepler system with additional term
$\displaystyle{\sum_{i=1}^{3}\frac{k_{i}}{q^{2}_{i}}}$ (eqs. (37) and (46))
can be generalized to the case of configuration space of constant curvature
[13]. Our procedure works also in this case . For example, the curvature-
dependent version of Smorodinsky-Winternitz system leads to the following
Hamiltonian

$\tilde{H}=\frac{1}{1-\mu
T^{2}_{\kappa}(r)}\left(\frac{1}{2}\left(p_{r}^{2}+\frac{1}{s^{2}_{\kappa}(r)}\left(p^{2}_{\theta}+\frac{p^{2}_{\varphi}}{\sin^{2}(\theta)}\right)\right)+\frac{1}{2}\alpha^{2}T^{2}_{\kappa}(r)+\sum_{i=1}^{3}\frac{k_{i}}{q^{2}_{\kappa_{i}}}\right)$
(48)

where

$s_{\kappa}(r)=\begin{cases}\frac{1}{\sqrt{\kappa}}\sin(\sqrt{\kappa}r),&\quad\kappa>0\\\
r,&\quad\kappa=0\\\
\frac{1}{\sqrt{-\kappa}}\sinh(\sqrt{-\kappa}r),&\quad\kappa<0\\\ \end{cases}$
(49)
$T_{\kappa}(r)=\begin{cases}\frac{1}{\sqrt{\kappa}}\tan(\sqrt{\kappa}r),&\quad\kappa>0\\\
r,&\quad\kappa=0\\\
\frac{1}{\sqrt{-\kappa}}\tanh(\sqrt{-\kappa}r),&\quad\kappa<0\\\ \end{cases}$
(50)

and

$\begin{split}q_{\kappa_{1}}&=s_{\kappa}(r)\sin\theta\cos\varphi\\\
q_{\kappa_{2}}&=s_{\kappa}(r)\sin\theta\sin\varphi\\\
q_{\kappa_{3}}&=s_{\kappa}(r)\cos\theta\end{split}$ (51)

These systems are discussed in detailed in [13]. It follows immediately that
they are superintegrable and their integrals of motion can be obtained
following the procedure described in the present paper.

Let us come back to the Hamiltonian (30). Adding a constant $h$ to the
Hamiltonian and making the substitutions
$\omega^{2}\rightarrow\omega^{2}-2\mu\tilde{H}$,
$\vec{k}\rightarrow\vec{k}-\tilde{H}\vec{l}$, $h\rightarrow h-h_{0}\tilde{H}$
we find

$\tilde{H}=\frac{1}{\mu\vec{q}\,^{2}+\vec{l}\cdot\vec{q}+h_{0}+1}\left(\frac{\vec{p}\,^{2}}{2}+\frac{\omega^{2}}{2}\vec{q}\,^{2}+\vec{k}\cdot\vec{q}+h\right)$
(52)

Considering two dimensions and putting $l_{i}=\delta_{i1}$,
$k_{i}=\delta_{i2}$, $\mu=0$, $h_{0}=-1$ we arrive at the one of the
Hamiltonians considered in [14] (eq.(64) therein).
Further, let us start with the two-dimensional Hamiltonian

$H=\frac{1}{2}\vec{p}\,^{2}+\frac{\omega^{2}}{2}\left(4q_{1}^{2}+q_{2}^{2}\right)+k_{1}q_{1}+\frac{k_{2}}{q_{2}^{2}}+h$
(53)

which is superintegrable. We put $h\rightarrow h_{0}+\tilde{H}$,
$k_{1}\rightarrow-\tilde{H}$; then

$\tilde{H}=\frac{\vec{p}\,^{2}}{2q_{1}}+\frac{\omega^{2}(4q_{1}^{2}+q_{2}^{2})}{2q_{1}}+\frac{h_{0}}{q_{1}}+\frac{k_{2}}{q_{1}q_{2}^{2}}$
(54)

which is the Hamiltonian (59) from [14].

##  IV Darboux and Hietarinta methods

In the eighties Hietarinta et al. introduced a very nice method, known as
coupling-constant metamorphosis, of constructing new Hamiltonians out of the
given ones. It consists in exchanging the roles of a coupling constant and
energy while preserving the form of unparameterized trajectories in phase
space. This method is somehow related to the theorem proven by Darboux in 1889
[1] (see also [2]). Both Darboux theorem and Hietarinta method nicely fit into
general framework described in Section II.

Let us start with the natural Hamiltonian of the form

$H=\frac{1}{2}g^{ij}(q)p_{i}p_{j}-\lambda U(q);$ (55)

here $\lambda$ is an arbitrary coupling constant and we follow the nonstandard
convention denoting by $-\lambda U$ the potential energy. Adding to the right
hand side the constant $\lambda-1$ and making the replacement
$\lambda\rightarrow\tilde{H}$ we arrive at the new Hamiltonian (cf. eq.(3))

$\tilde{H}=\frac{1}{U}\left(\frac{1}{2}g^{ij}p_{i}p_{j}-1\right)$ (56)

Let

$q^{i}=\tilde{q}\,^{i}\left(\tilde{t},\tilde{E}\right),\quad
p_{i}=\tilde{p}_{i}\left(\tilde{t},\tilde{E}\right)$ (57)

where

$\tilde{E}=\frac{1}{U}\left(\frac{1}{2}g^{ij}p_{i}p_{j}-1\right),$ (58)

denote the solutions to the canonical equations for the Hamiltonian (56). Then
the solutions to the canonical equations for $H$ can be written in the form
(cf. eqs.(4), (6), (7), (8))

$q^{i}=q^{i}\left(t;\lambda=\tilde{E}\right)=\tilde{q}\,^{i}\left(\tilde{t}(t),\tilde{E}\right),\quad
p_{i}=p_{i}\left(t;\lambda=\tilde{E}\right)=\tilde{p}_{i}\left(\tilde{t}(t),\tilde{E}\right)$
(59)

where

$\odv{\tilde{t}}{t}\equiv\Omega=U.$ (60)

The Lagrangians corresponding to $H$ and $\tilde{H}$ read, respectively (up to
irrelevant additive constants)

$L=\frac{1}{2}g_{ij}\dot{q}\,^{i}\dot{q}\,^{j}+\lambda U$ (61)

$\tilde{L}=\frac{U}{2}g_{ij}\dot{q}\,^{i}\dot{q}\,^{j}+\frac{1}{U}$ (62)

Now, according to the Darboux theorem [1]: the unparametrized trajectories in
configuration space for the Lagrangians $\tilde{L}$ (cf. eq.(62)) and
$L^{\prime}$ defined by the formula

$L^{\prime}=\frac{1}{2}g_{ij}\dot{q}\,^{i}\dot{q}\,^{j}+U$ (63)

coincide and the energies $\tilde{E}$ and $E^{\prime}$ corresponding to
$\tilde{L}$ and $L^{\prime}$, respectively satisfy the relation

$E^{\prime}\cdot\tilde{E}=1.$ (64)

Both conclusions follow from our approach. In fact, the solution
${q^{\prime}}\,^{i}={q^{\prime}}\,^{i}(t)$ to the Lagrange equations for
$L^{\prime}$ can be written in terms of the solutions $q^{i}=q^{i}(t;\lambda)$
to the Lagrange equations for $L$ as follows

$q^{\prime}\,{}^{i}(t)=q^{i}\left(\frac{1}{\sqrt{\lambda}}t;\lambda\right)$
(65)

Putting $\lambda=\tilde{E}$ and taking into account the first of equations
(59) we conclude

${q^{\prime}}\,^{i}(t)=\tilde{q}\,^{i}\left(\tilde{t}\left(\frac{t}{\sqrt{\tilde{E}}}\right),\tilde{E}\right)$
(66)

The energy $E^{\prime}$ corresponding to $L^{\prime}$

$E^{\prime}=\frac{1}{2}g_{ij}\odv{qi^{\prime}\,{}^{i}}{t}\odv{q^{\prime}\,{}^{j}}{t}-U$
(67)

when expressed in term of $\tilde{q}\,^{i}$ (cf. eq.(60)) reads

$E^{\prime}=\frac{U^{2}}{\tilde{E}}\frac{1}{2}g_{ij}\odv{\tilde{q}\,^{i}}{\tilde{t}}\odv{\tilde{q}\,^{j}}{\tilde{t}}-U$
(68)

By comparing eq.(68) with the energy corresponding to $\tilde{L}$

$\tilde{E}=\frac{U}{2}g_{ij}\odv{\tilde{q}\,^{i}}{\tilde{t}}\odv{\tilde{q}\,^{j}}{\tilde{t}}-\frac{1}{U}$
(69)

we find the relation (64).
As far as the Hietarinta et al. method is concerned one rewrites the
Hamiltonian (1) from Ref.[3] by adding the constant

$H=H_{0}-gF+(g-h)$ (70)

Letting $g=\tilde{H}\equiv G$ one arrives at the formula (2) from [3]

$G=\frac{H_{0}}{F}-\frac{h}{F}$ (71)

##  V Conclusions

We proposed a general framework for constructing, for a given Hamiltonian, the
whole family of new Hamiltonians sharing the same set of unparameterized
trajectories in phase space. Once the symmetry of initial Hamiltonian is known
our approach provides the complete characterization of the symmetry structure
of new Hamiltonian systems. In fact, eqs.(11) and (14)-(17) imply that the
integrals of motion for the latter can be immediately constructed provided
those corresponding to the initial Hamiltonian are known.

We have shown that our approach covers many examples considered recently in
the literature. We presented only some of them. It is interesting to note that
further, more complicated, examples can be obtained by applying the algorithm
iteratively.

Our method provides also an alternative proof of Darboux theorem as well as
Hietarinta et al. coupling-constant metamorphosis method.

## References

  * [1] Darboux, G.  Comptes Rendus Acad. Sci. Paris 108, 449–450 (1889).
  * [2] Albouy, A. and Zhao, L.  Regular and Chaotic Dynamics 27(3), 253–280 (2022).
  * [3] Hietarinta, J., Grammaticos, B., Dorizzi, B., and Ramani, A.  Phys. Rev. Lett. 53, 1707–1710 Oct (1984).
  * [4] Henon, M. and Heiles, C.  Astronomical Journal 69(1), 73–& (1964).
  * [5] Grammaticos, B., Dorizzi, B., and Ramani, A.  Journal of Mathematical Physics 25(12), 3470–3473 (1984).
  * [6] Holt, C. R.  Journal of Mathematical Physics 23(6), 1037–1046 (1982).
  * [7] Anco, S. C., Ballesteros, A., and Gandarias, M. L.  Physics Letters A 383(9), 801–807 (2019).
  * [8] Gonera, J., Jasiński, A., and Kosiński, P.  Annals of Physics 427, 168428 (2021).
  * [9] Cariñena, J. F., Rañada, M. F., and Santander, M.  Journal of Physics A: Mathematical and Theoretical 54(36), 365201 aug (2021).
  * [10] Rodríguez, M. A., Tempesta, P., and Winternitz, P.  Physical Review E 78(4), 046608 (2008).
  * [11] Rodríguez, M., Tempesta, P., and Winternitz, P.  In Journal of Physics: Conference Series, volume 175, 012013. IOP Publishing, (2009).
  * [12] Verrier, P. and Evans, N.  Journal of mathematical physics 49(2), 022902 (2008).
  * [13] Cariñena, J. F., Rañada, M. F., and Santander, M.  Journal of Physics A: Mathematical and Theoretical 54(36), 365201 (2021).
  * [14] Kalnins, E. G., Kress, J. M., and Winternitz, P.  Journal of Mathematical Physics 43(2), 970–983 (2002).
  * [15] Friš, J., Mandrosov, V., Smorodinsky, Y., Uhlíř, M., and Winternitz, P.  Physics Letters 16(3), 354–356 (1965).
  * [16] Evans, N.  Journal of mathematical physics 32(12), 3369–3375 (1991).
  * [17] Evans, N. W. and Verrier, P. E.  Journal of Mathematical Physics 49(9), 092902 (2008).
  * [18] Kalnins, E. G., Miller Jr, W., et al.  SIGMA. Symmetry, Integrability and Geometry: Methods and Applications 8, 034 (2012).
  * [19] Miller, W., Post, S., and Winternitz, P.  Journal of Physics A: Mathematical and Theoretical 46(42), 423001 (2013).
  * [20] Chanu, C., Degiovanni, L., Rastelli, G., et al.  SIGMA. Symmetry, Integrability and Geometry: Methods and Applications 7, 038 (2011).
  * [21] Gubbiotti, G. and Latini, D.  Journal of Physics A: Mathematical and Theoretical 51(28), 285201 (2018).
  * [22] Landau, L. and Lifshitz.  Mechanics: Volume 1.  Course of theoretical physics. Elsevier Science, (1976).
  * [23] Gonera, C. and Majewski, M.  Acta Physica Polonica. Series B 32(4), 1167–1171 (2001).