11institutetext: Korea University, Seoul, Korea
11email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>22institutetext: NAVER AI Lab, Seongnam, Korea
33institutetext: Samsung Electro-Mechanics, Suwon, Korea
33email<EMAIL_ADDRESS>
# ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency
Regularization

Jiwon Kim Work done in Korea UniversityEqual contribution1122 Youngjo
Min††footnotemark:  11 Daehwan Kim ††footnotemark:  33 Gyuseong Lee 11
Junyoung Seo 11 Kwangrok Ryoo 11 Seungryong Kim 11

###### Abstract

We present a novel semi-supervised learning framework that intelligently
leverages the consistency regularization between the model’s predictions from
two strongly-augmented views of an image, weighted by a confidence of pseudo-
label, dubbed ConMatch. While the latest semi-supervised learning methods use
weakly- and strongly-augmented views of an image to define a directional
consistency loss, how to define such direction for the consistency
regularization between two strongly-augmented views remains unexplored. To
account for this, we present novel confidence measures for pseudo-labels from
strongly-augmented views by means of weakly-augmented view as an anchor in
non-parametric and parametric approaches. Especially, in parametric approach,
we present, for the first time, to learn the confidence of pseudo-label within
the networks, which is learned with backbone model in an end-to-end manner. In
addition, we also present a stage-wise training to boost the convergence of
training. When incorporated in existing semi-supervised learners, ConMatch
consistently boosts the performance. We conduct experiments to demonstrate the
effectiveness of our ConMatch over the latest methods and provide extensive
ablation studies. Code has been made publicly available at
https://github.com/JiwonCocoder/ConMatch

##  1 Introduction

Semi-supervised learning has emerged as an attractive solution to mitigate the
reliance on large labeled data, which is often laborious to obtain, and
intelligently leverage a large amount of unlabeled data, to the point of being
deployed in many computer vision applications, especially image classification
[55, 53, 40]. Generally, this task have adopted pseudo-labeling [19, 30, 46,
51, 1, 61, 40] or consistency regularization [29, 48, 17, 36, 24, 53]. Some
methods [5, 4, 52, 47, 58, 54, 42] proposed to integrate both approaches in a
unified framework, which is often called holistic approach. As one of
pioneering works, FixMatch [47] first generates a pseudo-label from the
model’s prediction on the weakly-augmented instance and then encourages the
prediction from the strongly-augmented instance to follow the pseudo-label.
Their success inspired many variants that use, e.g., curriculum learning [58,
54].

On the other hand, concurrent to the race for better semi-supervised learning
methods [47, 58, 54], substantial progress has been made in self-supervised
representation learning, especially with contrastive learning [3, 8, 22, 20,
6, 10], aiming at learning a task-agnostic feature representation without any
supervision, which can be well transferred to the downstream tasks. Formally,
they encourage the features extracted from two differently-augmented images to
be pulled against each other, which injects some invariance or robustness into
the models. Not surprisingly, semi-supervised learning frameworks can
definitely benefit from self-supervised representation learning [33, 25, 34]
in that good representation from the feature encoder yields better performance
with semi-supervised learning, and thus, some methods [33, 25] attempt to
combine the aforementioned two paradigms to boost the performance by achieving
the better feature encoder.

Extending techniques presented in existing self-supervised representation
learning [3, 8, 22, 20, 6, 10], which only focus on learning feature encoder,
to further consider the model’s prediction itself would be an appealing
solution to effectively combine the two paradigms, which allows for boosting
not only feature encoder but also classifier. However, compared to feature
representation learning [3, 8, 22, 20, 6, 10], the consistency between the
model’s predictions from two different augmentations should be defined by
considering which direction is better to achieve not only invariance but also
high accuracy in image classification. Without this, simply pulling the
model’s predictions as done in [3, 8, 22, 20, 6, 10] may hinder the classifier
output, thereby decreasing the accuracy.

In this paper, we present a novel framework for semi-supervised learning,
dubbed ConMatch, that intelligently leverages the confidence-guided
consistency regularization between the model’s predictions from two strongly-
augmented images. Built upon conventional frameworks [47, 58], we consider two
strongly-augmented images and one weakly-augmented image, and define the
consistency between the model’s predictions from two strongly-augmented
images, while still using an unsupervised loss between the model’s predictions
from one of the strongly-augmented images and the weakly-augmented image, as
done in [47, 58]. Since defining the direction of consistency regularization
between two strongly-augmented images is of prime importance, rather than
selecting in a deterministic manner, we present a probabilistic technique by
measuring the confidence of pseudo-labels from each strongly-augmented image,
and weighting the consistency loss with this confidence. To measure the
confidence of pseudo-labels, we present two techniques, including non-
parametric and parametric approaches. With this confidence-guided consistency
regularization, our framework dramatically boosts the performance of existing
semi-supervised learners [47, 58]. In addition, we also present a stage-wise
training scheme to boost the convergence of training. Our framework is a plug-
and-play module, and thus various semi-supervised learners [4, 52, 47, 58, 54,
33, 25, 34] can benefit from our framework. We briefly summarize our method
with other highly relevant works in semi-supervised learning in Table 1.
Experimental results and ablation studies show that the proposed framework not
only boosts the convergence but also achieves the state-of-the-art performance
on most standard benchmarks [28, 37, 12].

Table 1:  Comparison of our ConMatch to other relevant works which have a form
of consistency regularization combining pseudo-labeling [5, 47, 58, 54, 25,
33, 34]

| MixMatch | FixMatch | FlexMatch | Dash | SelfMatch | CoMatch | LESS | ConMatch
---|---|---|---|---|---|---|---|---
|  [5] |  [47] |  [58] |  [54] |  [25] |  [33] |  [34] | (Ours)
Using pseudo-labeling | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
Using two strong branches | ✗ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✓
Learning confidence measure | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓
Using stage-wise training | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓

##  2 Related Works

####  2.0.1 Semi-supervised Learning.

Semi-supervised learning has been an effective paradigm for leveraging an
abundance of unlabeled data along with limited labeled data. For this task,
various methods such as pseudo-labeling [30, 19] and consistency
regularization [44, 29, 48] have been proposed. In pseudo-labeling [30], a
model uses unlabeled samples with high confidence as training targets, which
reduces the density of data points at the decision boundary [19, 43].
Consistency regularization has been first introduced by $\pi$-model [44],
which is further improved by numerous following works [29, 48, 17, 36, 24,
53]. In the consistency regularization, the model should minimize the distance
between the model’s predictions when fed perturbed versions of the input [29,
48, 36, 39, 24, 51, 53] or the model [29, 48, 39, 24, 53, 59]. Very recently,
advanced consistency regularization methods [4, 52, 47] have been introduced
by combining with pseudo-labeling. These methods show high accuracy,
comparable to supervised learning in a fully-labeled setting, e.g., ICT [50],
MixMatch [5], UDA [52], ReMixMatch [4], and FixMatch [47]. The aforementioned
methods can be highly boosted by simultaneously considering the techniques
proposed in recent self-supervised representation learning methods [3, 8, 22,
20, 6].

####  2.0.2 Self-supervised Representation Learning.

Self-supervised representation learning has recently attracted much attention
[16, 60, 38, 18, 3, 8, 22, 20, 6] due to its competitive performance.
Specifically, contrastive learning [3, 8, 22, 20, 6] becomes a dominant
framework. It formally maximizes the agreement between different augmented
views of the same image [16, 60, 38, 18]. Most previous methods benefit from a
large amount of negative pairs to preclude constant outputs and avoid a
collapse problem [8]. An alternative to approximate the loss is to use
cluster-based approach by discriminating between groups of images with similar
features [6]. Some methods [20, 10] mitigated to use negative samples by using
a momentum encoder [20] and a stop-gradient technique [10]. The aforementioned
methods applied the consistency loss at the feature-level, unlike recent semi-
supervised learning methods [47, 58] that consider the consistency loss in the
logit-level, which may not be optimal to be incorporated with semi-supervised
learners. Formulating the consistency loss in the logit-level as self-
supervision is challenging because a direction between two augmented views
should be determined. Without this, simply pulling the model’s predictions as
done in [3, 8, 22, 20, 6, 10] may hinder the classifier output, thereby
decreasing the accuracy.

(a) Semi-sup. (b) Self-sup. (c) Semi-sup. with self-sup.

Figure 1: Conceptual illustration of existing methods that leverage unlabeled
data: (a) semi-supervised learning- the model uses the model’s prediction
itself to produce a pseudo-label for unlabeled data [30, 44, 29, 48, 17, 36,
50, 5, 4, 52, 47], (b) self-supervised representation learning- the model is
learned to generate the same feature embedding for two augmented views from
unlabeled data [3, 8, 22, 20, 6, 20, 10], and (c) semi-supervised learning
with self-supervision representation learning- the model simultaneously learns
a feature representation with self-supervised representation loss, while
learning all the networks with semi-supervised learning [25, 33, 34].

####  2.0.3 Self-supervision in Semi-supervised Learning.

Many recent state-of-the-art semi-supervised learning methods adopt the self-
supervised representation learning methods [57, 9] to jointly learn good
feature representation. Self-supervised pre-training, followed by supervised
fine-tuning, has shown strong performance on semi-supervised learning
settings. Specifically, SelfMatch [25] adopted SimCLR [8] for self-supervised
pre-training and FixMatch [47] for semi-supervised fine-tuning. However, it
may learn sub-optimal representation for the image classification task due to
the task-agnostic learning. On the other hand, some methods [33, 31] unify
pseudo-labeling and self-supervised learning. [32] alternates between self-
and semi-supervised learning. There lacks a study to effectively use self-
supervision, rather than simply adopting this.

####  2.0.4 Confidence Estimation in Semi-supervised Learning.

In semi-supervised learning, a confidence-based strategy has been widely used
along with pseudo labeling so that the unlabeled data are used only when the
predictions are sufficiently confident. Such confidence in pseudo-labeling has
been often measured by the peak values of the predicted probability
distribution [52, 47, 58, 54, 42]. Although the selection of unlabeled samples
with high confidence predictions moves decision boundaries to low density
regions [7], many of these selected predictions are incorrect due to the poor
calibration of neural networks [21], which has the discrepancy between the
confidence level of a network’s individual predictions and its overall
accuracy and leads to noisy training and poor generalization [14, 15].
However, there was no study how to learn the confidence of pseudo-labels,
which is the topic of this paper.

##  3 Methodology

###  3.1 Preliminaries

Let us define a batch of labeled instances as
$\mathcal{X}=\left\\{\left(x_{b},y_{b}\right)\right\\}_{b=1}^{B}$, where
$x_{b}$ is an instance and $y_{b}$ is a label representing one of $Y$ labels.
In addition, let us define a batch of unlabeled instances as
$\mathcal{U}=\left\\{u_{b}\right\\}_{b=1}^{\mu B}$, where $\mu$ is a hyper-
parameter that determines the size of $\mathcal{U}$ relative to $\mathcal{X}$.
The objective of semi-supervised learning is to use both $\mathcal{X}$ and
$\mathcal{U}$ to train a model with parameters $\theta$ taking an instance
$r\in\mathcal{X}\cup\mathcal{U}$ as input and outputting a distribution over
class labels $y$ such that $p_{\mathrm{model}}(y|r;\theta)$. The model
generally consists of an feature encoder $f(\cdot)$ and a classifier
$g(\cdot)$, and thus, $p_{\mathrm{model}}(y|r;\theta)=g(f(r))$.

For semi-supervised learning, most state-of-the-art methods are based on
consistency regularization approaches [2, 44, 29] that rely on the assumption
that the model should generate similar predictions when perturbed versions of
the same instance are fed, e.g., using data augmentation [36], or model
perturbation [29, 48]. These methods formally extract a pseudo-label from one
branch, filtered by confidence, and use this as a target for another branch.
For instance, FixMatch [47] utilizes two types of augmentations such as weak
and strong, denoted by $\alpha(\cdot)$ and $\mathcal{A}(\cdot)$, and a pseudo-
label from weakly-augmented version of an image is used as a target for
strongly-augmented version of the same image. This loss function is formally
defined such that

$\mathcal{L}_{\mathrm{un}}=c(r)\mathcal{H}\left(q(r),p_{\mathrm{model}}\left(y|\mathcal{A}(r);\theta\right)\right),$
(1)

where $c(r)$ denotes a confidence of $q(r)$, and $q(r)$ denotes a pseudo-label
generated from $p_{\mathrm{model}}(y|\alpha(r);\theta)$, which can be either
an one-hot label [47, 58, 42, 54] or a sharpened one [52, 5, 4], and
$\mathcal{H}(\cdot,\cdot)$ is often defined as a cross-entropy loss. In this
framework, measuring confidence $c(r)$ is of prime importance, but
conventional methods simply measure this, e.g., by the peak value of the
softmax predictions [52, 47, 58, 42, 54].

On the other hands, semi-supervised learning framework can definitely benefit
from existing self-supervised representation learning [33, 25, 34] in that
good representation from the feature encoder $f(\cdot)$ yields better
performance with semi-supervised learner. In this light, some methods
attempted to combine semi-supervised learning and self-supervised
representation learning to achieve the better feature encoder [33, 25].
Concurrent to the race for better semi-supervised learning methods,
substantial progress has been made in self-supervised representation learning,
especially with contrastive learning [3, 8, 22, 20, 6, 10]. The loss function
for this task can also be defined as a consistency regularization loss,
similar to [52, 47, 58, 42, 54] but in the feature-level, such that

$\begin{split}\mathcal{L}_{\mathrm{self}}=\mathcal{D}(F_{i}(r),F_{j}(r)),\end{split}$
(2)

where $F_{i}(r)=f(\mathcal{A}_{i}(r))$ and $F_{j}(r)=f(\mathcal{A}_{j}(r))$
extracted from images with two different strongly-augmented images
$\mathcal{A}_{i}(\cdot)$ and $\mathcal{A}_{j}(\cdot)$, respectively.
$\mathcal{D}(\cdot,\cdot)$ can be defined as contrastive loss [22] or negative
cosine similarity [10]. Even though this loss helps to boost learning the
feature encoder $f(\cdot)$, the mechanism that simply pulls the features
$F_{i}(r)$ and $F_{j}(r)$ may not be optimal to boost a semi-supervised
learner and break the latent feature space, without considering a direction
representing which branch is better.

Figure 2: Network configuration of ConMatch. A semi-supervised learning
framework built upon consistency loss with an additional strong branch to
leverage confidence loss between two strong branches. In the parametric
approach, the confidence estimator block takes a concatenated heterogeneous
feature as an input and produces the estimated confidence of pseudo label.

###  3.2 Formulation

To combine the semi- and self-supervised learning paradigm in a boosting
fashion, unlike [33, 25, 34], we present to effectively exploit a self-
supervision between two strong branches tailored for boosting semi-supervised
learning, called ConMatch. Unlike existing self-supervised representation
learning methods, e.g., SimSiam [10], we formulate the consistency
regularization loss at class logit-level111In the paper, class logit means the
output of the network, i.e., $p_{\mathrm{model}}(y|r;\theta)$ for $r$., as
done in semi-supervised learning methods [47, 58], and estimate the
confidences of each pseudo-label from two strongly-augmented images,
$\mathcal{A}_{i}(r)$ and $\mathcal{A}_{j}(r)$ for $r$, and use them to
consider the probability of each direction between them. Since measuring such
confidences is notoriously challenging, we present novel confidence estimators
by using the output from weak-augmented image $\alpha(r)$ as an anchor in non-
parametric and parametric approaches.

An overview of our ConMatch is illustrated in Fig. 2. Specifically, there
exist two branches for strongly-augmented images (called strong branches) and
one branch for weakly-augmented image (called weak branch). Similar to
existing semi-supervised representation learning methods [47, 58, 54], we
attempt to apply the consistency loss between a pair of each strong branch and
weak branch. But, tailored to semi-supervised learning, we present a
confidence-guided consistency regularization loss $\mathcal{L}_{\mathrm{ccr}}$
between two strong branches such that

$\begin{split}\mathcal{L}_{\mathrm{ccr}}=c_{i}(r)\mathcal{H}(q_{i}(r),p_{\mathrm{model}}(y|\mathcal{A}_{j}(r);\theta))+c_{j}(r)\mathcal{H}(q_{j}(r),p_{\mathrm{model}}(y|\mathcal{A}_{i}(r);\theta)),\end{split}$
(3)

where $q_{i}(r)$ and $q_{j}(r)$ denote the pseudo-labels generated from
$p_{\mathrm{model}}(y|\mathcal{A}_{i}(r);\theta)$ and
$p_{\mathrm{model}}(y|\mathcal{A}_{j}(r);\theta)$, respectively. $c_{i}(r)$
and $c_{j}(r)$ denote estimated confidences of $q_{i}(r)$ and $q_{j}(r)$. Our
proposed loss function is different from conventional self-supervised
representation learning loss $\mathcal{L}_{\mathrm{self}}$ in that the
consistency is applied in the logit-level (not feature-level) similar to [47,
58], and adjusted by the estimated confidence. However, unlike [52, 47, 58,
42, 54], we can learn the better feature representation by considering two
strongly-augmented views, while improving semi-supervised learning performance
at the same time. It should be noted that this simple loss function can be
incorporated with any semi-supervised learners, e.g., FixMatch [47] or
FlexMatch [58].

To measure the confidences $c_{i}(r)$ and $c_{j}(r)$, we present two kinds of
confidence estimators, based on non-parametric and parametric approaches. In
the following, we explain how to measure these confidences in detail.

###  3.3 Measuring Confidence: Non-parametric Approach

Existing semi-supervised learning methods [30, 44, 47] have selected unlabeled
samples with high confidence as training targets (i.e., pseudo-labels) in a
straightforward way; which can be viewed as a form of entropy minimization
[19]. It has been well known that it is non-trivial to set an appropriate
threshold for such handcrafted confidence estimation, and thus, confidence-
based strategies commonly suffer from a dilemma between pseudo-label
exploration and accuracy depending on the threshold [1, 34].

In our framework, estimating the confidence of pseudo-labels from strong
branches may suffer from similar limitations if the conventional handcrafted
methods [30, 44, 47] are simply used. To overcome this, we present a novel way
to measure the confidences, $c_{i}(r)$ and $c_{j}(r)$, based on the similarity
between outputs of strongly-augmented images and weakly-augmented images.
Based on the hypothesis that the similarity between the logits or
probabilities from strongly-augmented images and weakly-augmented images can
be directly used as a confidence estimator, we present to measure confidence
of each strong branch loss by the cross-entropy loss value itself between
strongly-augmented and weakly-augmented images. Specifically, we measure such
a confidence with the following:

$s_{i}(r)=\frac{1}{\mathcal{H}(p_{\mathrm{model}}(y|\alpha(r);\theta),p_{\mathrm{model}}(y|\mathcal{A}_{i}(r);\theta))},$
(4)

where the smaller
$\mathcal{H}(p_{\mathrm{model}}(y|\alpha(r);\theta),p_{\mathrm{model}}(y|\mathcal{A}_{i}(r);\theta))$,
the higher $s_{i}(r)$ is. $s_{j}(r)$ can be similarly defined with $\alpha(r)$
and $\mathcal{A}_{j}(r)$. Finally, $c_{i}(r)$ is computed such that
$c_{i}(r)=s_{i}(r)/(s_{i}(r)+s_{j}(r))$, and $c_{j}(r)$ is similarly computed.

In this case, the total loss for the non-parametric approach is as follows:

$\mathcal{L}^{\mathrm{np}}_{\mathrm{total}}=\lambda_{\mathrm{sup}}\mathcal{L}_{\mathrm{sup}}+\lambda_{\mathrm{un}}\mathcal{L}_{\mathrm{un}}+\lambda_{\mathrm{ccr}}\mathcal{L}_{\mathrm{ccr}},$
(5)

where $\lambda_{\mathrm{sup}}$, $\lambda_{\mathrm{un}}$, and
$\lambda_{\mathrm{ccr}}$ are weights for $\mathcal{L}_{\mathrm{sup}}$,
$\mathcal{L}_{\mathrm{un}}$, and $\mathcal{L}_{\mathrm{ccr}}$, respectively.
Note that for weakly-augmented labeled images $\alpha(x_{b})$ with labels
$y_{b}$, a simple classification loss $\mathcal{L}_{\mathrm{sup}}$ is applied
as $\mathcal{H}(y_{b},p_{\mathrm{model}}(y|\alpha(x_{b});\theta))$, as done in
[47].

1: Notation: strong augmentation $\mathcal{A}$, weak augmentation $\alpha$,
model $p_{\mathrm{model}}(\cdot;\theta)$ consisting of feature encoder $f$ and
classifier $g$, confidence estimator $h(\cdot;\theta_{\mathrm{conf}})$, pseudo
label $q$, leranable confidence $c$

2: Input:
$\mathcal{X}=\\{(x_{b},y_{b}):b\in(1,\dots,B)\\},\;\;\mathcal{U}=\\{u_{b}:b\in(1,\dots,\mu
B)\\}$

3: for $b=1$ to $B$ do

4: $F(\alpha(x_{b})),L(\alpha(x_{b}))=f(\alpha(x_{b})),g(f(\alpha(x_{b}))$)

5:
$c(\alpha(x_{b}))=h(F(\alpha(x_{b})),L(\alpha(x_{b}));\theta_{\mathrm{conf}})$

6: if
$y_{b}==\mathrm{argmax}_{y}{p_{\mathrm{model}}(y|\alpha(x_{b});\theta))}$ then

7: $c_{\mathrm{GT}}(\alpha(x_{b}))=1$

8: else

9: $c_{\mathrm{GT}}(\alpha(x_{b}))=0$

10: end if

11: end for

12:
$\mathcal{L}_{\mathrm{sup}}=\sum_{b=1}^{B}\mathcal{H}(y_{b},p_{\mathrm{model}}(y|\alpha(x_{b});\theta))$

13: $\mathcal{L}_{\mathrm{conf-
sup}}=\mathcal{H}(c_{\mathrm{GT}}(\alpha{(x_{b})}),h(F(\alpha(x_{b})),L(\alpha(x_{b}));\theta_{\mathrm{conf}}))$

14: for $b=1$ to $\mu B$ do

15:
$(F_{i},F_{j}),(L_{i},L_{j})=f(\mathcal{A}_{i}(u_{b}),\mathcal{A}_{j}(u_{b})),g(f(\mathcal{A}_{i}(u_{b}),\mathcal{A}_{j}(u_{b})))$

16:
$c_{i},c_{j}=h(F_{i},L_{i};\theta_{\mathrm{conf}}),h(F_{j},L_{j};\theta_{\mathrm{conf}})$

17: Generate pseudo labels for differently augmented versions
$\alpha,\mathcal{A}_{i},\mathcal{A}_{j}$

18: end for

19: Calculate $\mathcal{L}_{\mathrm{un}}$ using $c,q$ from $\alpha({u_{b}})$,
and $p_{\mathrm{model}}(y|\mathcal{A}(u_{b});\theta)$ via Eq. 2

20: Calculate $\mathcal{L}_{\mathrm{ccr}}$ using $c,q$ from
$\mathcal{A}(u_{b})$ and $p_{\mathrm{model}}(y|\mathcal{A}(u_{b});\theta)$ via
Eq. 3

21: Calculate $\mathcal{L}_{\mathrm{conf}}$ using $c$ from
$\mathcal{A}(u_{b})$ and $p_{\mathrm{model}}(y|\alpha(u_{b});\theta)$ via Eq.
6

22: Update $\theta$ by minimizing
$\mathcal{L}_{\mathrm{sup}},\mathcal{L}_{\mathrm{un}}$ and
$\mathcal{L}_{\mathrm{ccr}}$

23: Update $\theta_{\mathrm{conf}}$ by minimizing $\mathcal{L}_{\mathrm{conf-
sup}}$ and $\mathcal{L}_{\mathrm{conf}}$

24: Return: Model parameters {$\theta$, $\theta_{\mathrm{conf}}$}

Algorithm 1 ConMatch-P (Parametric Approach)

###  3.4 Measuring Confidence: Parametric Approach

Even though the above confidence estimator with non-parametric approach yields
comparable performance to some extent (which will be discussed in
experiments), it solely depends on each image, and thus it may be sensitive to
outliers or errors without any modules to learn a prior from the dataset. To
overcome this, we present an additional parametric approach for confidence
estimation. Motivated by stereo confidence estimation [41, 45, 49, 11],
obtaining a confidence measure from the networks by extracting the confidence
features from input and predicting the confidence with a classifier, we also
introduce a learnable confidence measure for pseudo-labels. Unlike existing
methods that simply use the model output as confidence [52, 47, 58, 54, 42],
such learned confidence can intelligently select a subset of pseudo-labels
that are less noisy, which helps the network to converge significantly faster
and achieve improved performance by utilizing the false negative samples
excluded from training by high threshold at early training iterations.

Specifically, we define an additional network for learnable confidence
estimation such that $c(r)=h(F(r),L(r);\theta_{\mathrm{conf}})$, where
$h(\cdot)$ is a confidence estimator with model parameters
$\theta_{\mathrm{conf}}$, $F(r)$ is a feature, and $L(r)$ is a logit from an
instance $r$, as shown in Fig. 2. For the network architecture, the
concatenation of feature $F(r)$ and logit $L(r)$ transformed by individual
non-linear projection heads is used, based on the intuition that a direct
concatenation of two their heterogeneous confidence features does not provide
an optimal performance [26], followed by the final classifier for confidence
estimation. The detailed network architecture is described in the
supplementary material.

The confidence estimator is learned with the following loss function:

$\mathcal{L}_{\mathrm{conf}}=c_{i}(r)\mathcal{H}(p_{\mathrm{model}}(y|\alpha(r);\theta_{\mathrm{freeze}}),p_{\mathrm{model}}(y|\mathcal{A}_{i}(r);\theta_{\mathrm{freeze}}))+\mathrm{log}(1/c_{i}(r)),$
(6)

where $\theta_{\mathrm{freeze}}$ is a freezed network parameter with a stop
gradient. The intuition behind is that during the confidence network training,
we just want to make the network learn the confidence itself, rather than
collapsing to trivial solution to learn the feature encoder simultaneously. In
addition, we also use the supervised loss for confidence estimator
$\mathcal{L}_{\mathrm{conf-
sup}}=\mathcal{H}(c_{\mathrm{GT}},h(F(\alpha(x_{b})),L(\alpha(x_{b}));\theta_{\mathrm{conf}}))$;
$c_{\mathrm{GT}}$ = 1 if $y_{b}$ is equal to
$\mathrm{argmax}_{y}\,p_{\mathrm{model}}(y|\alpha(x_{b});\theta)$, and
$c_{\mathrm{GT}}=0$ otherwise.

The total loss for the parametric case can be written as

$\mathcal{L}^{\mathrm{param}}_{\mathrm{total}}=\lambda_{\mathrm{sup}}\mathcal{L}_{\mathrm{sup}}+\lambda_{\mathrm{un}}\mathcal{L}_{\mathrm{un}}+\lambda_{\mathrm{conf}}\mathcal{L}_{\mathrm{conf}}+\lambda_{\mathrm{conf-
sup}}\mathcal{L}_{\mathrm{conf-
sup}}+\lambda_{\mathrm{ccr}}\mathcal{L}_{\mathrm{ccr}}$ (7)

where $\lambda_{\mathrm{conf}}$ and $\lambda_{\mathrm{conf-sup}}$ are the
weights for $\mathcal{L}_{\mathrm{conf}}$ and $\mathcal{L}_{\mathrm{conf-
sup}}$, respectively. We explain an algorithm for ConMatch of parametric
approach in Alg. 1.

###  3.5 Stage-Wise Training

Even though our framework can be trained in an end-to-end manner, we further
propose a stage-wise training strategy to boost the convergence of training.
This stage-wise training consists of three stages, 1) pre-training for the
feature encoder, 2) pre-training for the confidence estimator (for parametric
approach only), and 3) fine-tuning for both feature encoder and confidence
estimator (for parametric approach only). Specifically, we first warm up the
feature encoder by solely using the standard semi-supervised loss functions
with $\mathcal{L}_{\mathrm{sup}}$ and $\mathcal{L}_{\mathrm{un}}$. We then
train the confidence estimator based on the outputs of the pre-trained feature
encoder in the parametric approach. As mentioned in [27], this kind of simple
technique highly boosts the convergence to discriminate between confident and
unconfident outputs from the networks. Finally, we fine-tune all the networks
with the proposed confidence-guided self-supervised loss
$\mathcal{L}_{\mathrm{ccr}}$. We empirically demonstrate the effectiveness of
the stage-wise training by achieving state-of-the-art results on standard
benchmark datasets [28, 37, 12].

##  4 Experiments

###  4.1 Experimental Settings

In experiments, we extensively evaluate the performance of our ConMatch on
various standard datasets [28, 37, 12] with various label fraction settings in
comparison to state-of-the-art algorithms, such as UDA [52], FixMatch [47],
FlexMatch [58], SelfMatch [25], LESS [34] and Dash [54]. Our proposed methods
have two variants; ConMatch-NP (non-parametric approach), and ConMatch-P
(parametric approach) integrated to FlexMatch [58], which is the state-of-the-
art semi-supervised learner, even though it can be easily integrated to others
[51, 47, 33].

####  4.1.1 Datasets.

We consider four standard benchmarks, including CIFAR-10/100 [28], SVHN [37],
and STL-10 [12]. CIFAR-10 [28] contains 50,000 training images and 10,000 test
images, which have resolution 32$\times$32 with ten classes. Similar to
CIFAR-10, CIFAR-100 [28] has the same number of training/test images and image
size, but it differently classifies as 100 fine-grained classes. SVHN [37]
consists of 73,257 training images with 26,032 test images, having also
32$\times$32 resolution images, belonging to ten different classes of numeric
digits. STL-10 [12] contains 5,000 labeled images with size of 96$\times$96
from 10 classes and 100,000 unlabeled images with size of 96$\times$96.

####  4.1.2 Evaluation Metrics.

For quantitative evaluation, we compute the mean and standard deviation of
error rates, when trained on 3 different folds for labeled data, based on the
standard evaluation protocol of selecting a subset of the training data while
keeping the remainder unlabeled. In addition, as in [33, 58], we evaluate the
quality of pseudo labels by training curves of precision, recall, and F1
values.

###  4.2 Implementation Details

For a fair comparison, we generally follow the same hyperparameters with
FixMatch [47]. Specifically, we use Wide ResNet (WRN)[56] as a feature encoder
for the experiments, especially WRN-28-2 for CIFAR-10 [28] and SVHN [37],
WRN-28-8 for CIFAR-100[28], and WRN-37-2 for STL-10 [12]. We use a batch size
of labeled data $B=64$, the ratio of unlabeled data $\mu=7$, and SGD optimizer
with a learning rate starting from 0.03, The detailed hyperparameter settings
are described in the supplementary material. For a weakly-augmented sample, we
use a crop-and-flip, and for a strongly-augmented sample, we use RandAugmnet
[13].

###  4.3 Comparison to State-Of-The-Art Methods

On standard semi-supervised learning benchmarks, we evaluate the performance
of our frameworks, ConMatch-P and ConMatch-NP, compared to various state-of-
the-art methods, as shown in Table 2 and Table 3. We observe that the
performance difference between ConMatch-NP and ConMatch-P is not large, except
in the label-scare setting. This may be explained by the fact that non-
parametric method highly depends on baseline performance since it does not
consider other samples which can be modeled as a prior. We show our
superiority on most benchmarks with extensive label setting, but we mainly
focus the label-scare setting, since it corresponds to the central goal of
semi-supervised learning, reducing the need for labeled data. We achieves
4.43% and 38.89% error rate for CIFAR-10 and CIFAR-100 settings [28] with only
4 labels per class respectively. Compared to the results of SelfMatch [25] and
CoMatch [33], closely related to ours, adopting self-supervised methods, we
can prove the competitiveness of our method by achieving 2.38% and 2.48%
improvements at CIFAR-10 with 40 labels. On the other datasets, CIFAR-100 [37]
and STL-10 [12], we record the lowest error rate of 38.89% and 25.39% with 400
and 2500 labels setting, and also slightly better than baseline [58] by
recording 5.26% in STL-10 dataset.

Table 2:  Comparison on error rates on CIFAR-10 [28] and CIFAR-100 [28]
benchmarks on 3 different folds.

| CIFAR-10 | CIFAR-100
---|---|---
Methods | 40 | 250 | 4,000 | 400 | 2,500
UDA [52] | 29.05$\pm$5.93 | 8.82$\pm$1.08 | 4.88$\pm$0.18 | 59.28$\pm$0.88 | 33.13$\pm$0.22
FixMatch (RA) [47] | 13.81$\pm$3.37 | 5.07$\pm$0.65 | 4.26$\pm$0.06 | 48.85$\pm$1.75 | 28.29$\pm$0.11
FlexMatch [58] | 4.97$\pm$0.06 | 4.98$\pm$0.09 | 4.19$\pm$0.01 | 39.94$\pm$1.62 | 26.49$\pm$0.20
SelfMatch [25] | 6.81$\pm$1.08 | 4.87$\pm$0.26 | 4.06$\pm$0.08 | - | -
CoMatch [33] | 6.91$\pm$8.47 | 4.91$\pm$0.33 | - | - | -
LESS [34] | 6.80$\pm$1.10 | 4.90$\pm$0.80 | - | 48.70$\pm$12.40 | -
Dash (RA) [54] | 13.22$\pm$3.75 | 4.56$\pm$0.13 | 4.08$\pm$0.06 | 44.76$\pm$0.96 | 27.18$\pm$0.21
ConMatch-NP | 4.89$\pm$0.07 | 5.00$\pm$0.37 | 4.36$\pm$0.42 | 44.90$\pm$1.34 | 26.91$\pm$1.35
ConMatch-P | 4.43$\pm$0.13 | 4.70$\pm$0.25 | 3.92$\pm$0.08 | 38.89$\pm$2.18 | 25.39$\pm$0.20

Table 3:  Comparison on error rates on SVHN [37] and STL-10 [12] benchamarks
on 3 different folds.

| SVHN | STL-10
---|---|---
Method | 40 | 250 | 1,000
UDA [52] | 52.63$\pm$20.51 | 5.69$\pm$2.76 | 7.66$\pm$0.56
FixMatch (RA) [47] | 3.96$\pm$2.17 | 2.48$\pm$0.38 | 7.98$\pm$1.50
FlexMatch [58] | 4.97$\pm$0.06 | 4.98$\pm$0.09 | 5.77$\pm$0.18
SelfMatch [25] | 3.42$\pm$1.02 | 2.63$\pm$0.43 | -
CoMatch [33] | 6.91$\pm$8.47 | 4.91$\pm$0.33 | 20.20$\pm$0.38
Dash (RA) [54] | 3.03$\pm$1.59 | 2.17$\pm$0.10 | 7.26$\pm$0.40
ConMatch-NP | 6.20$\pm$3.44 | 5.80$\pm$0.74 | 6.02$\pm$0.08
ConMatch-P | 3.14$\pm$0.57 | 3.13$\pm$0.72 | 5.26$\pm$0.04

Table 4:  Ablation study of different semi-supervised baselines. We evaluate
non-parametric (ConMatch-NP) and parametric (ConMatch-P) approaches with
different baselines, Fixmatch [47] and FlexMatch [58].

| CIFAR-10 | CIFAR-100
---|---|---
Methods | 40 | 250 | 400
FixMatch [47] | 13.81 | 5.07 | 48.85
ConMatch-NP w/ [47] | 6.83 | 4.73 | 48.73
ConMatch-P w/ [47] | 5.13 | 4.64 | 48.00
FlexMatch [58] | 4.97 | 4.98 | 39.94
ConMatch-NP w/ [58] | 4.84 | 4.74 | 44.90
ConMatch-P w/ [58] | 4.68 | 4.70 | 38.89

Table 5:  Ablation study of training schemes. E means end-to-end training and
S means stage-wise training.

| Status | CIFAR-10 | CIFAR-100
---|---|---|---
Methods | 40 | 250 | 400
ConMatch w/ FixMatch [47] | E | 4.85 | 4.77 | 47.81
S | 5.13 | 4.60 | 48.00
ConMatch w/ FlexMatch [58] | E | 4.68 | 4.70 | 57.16
S | 4.43 | 4.70 | 38.89

Table 6:  Ablation study of our component on CIFAR-10 [28] with 40 labels.

|  Three |  Logit-level | Confidence net. input |  Error rate
---|---|---|---|---
|  branches |  self-sup. |  logits |  features |
(I) |  ✓ |  ✗ |  ✗ |  ✗ |  18.11
(II) |  ✓ |  ✓ |  ✗ |  ✗ |  77.50
(III) |  ✓ |  ✓ |  ✓ |  ✗ |  7.05
(IV) |  ✓ |  ✓ |  ✓ |  ✓ |  5.13

###  4.4 Ablation Study

####  4.4.1 Effects of Different Baseline.

We first evaluate our ConMatch with two baselines, FixMatch [47] and FlexMatch
[58], in both parametric (ConMatch-P) and non-parametric (ConMatch-NP)
approaches as shown in Table 4. ConMatch-P w/[47] boosts the performance
significantly on CIFAR-10 with 40 labels from 13.81% to 5.13%, achieving the
state-of-the-art result. The performance gains of ConMatch-P w/[47] is
relatively higher than one w/[58] on most setting since [47] does not
adaptively adjust the threshold depending on the difficulty level of samples.
Note that the thresholds of FixMatch [47] and FlexMatch [58] are used only for
$\mathcal{L}_{\mathrm{un}}$.

####  4.4.2 Effectiveness of Confidence Measure.

In Table 4, we evaluate two confidence measures in non-parametric and
parametric approach. In extremely label-scare setting, such as CIFAR-10 with 4
labels per class, the non-parametric approach achieves relatively lower
performance, 1.70% and 0.16%, in both FixMatch and FlexMatch baseline, while
the parametric approach (ConMatch-P w/[58]) reaches the state-of-the-art
performance. But, as the number of labels increases, the gap between non-
parametric and parametric approach decreases, indicating that a certain number
of labeled samples are required to measure the confidence without the
confidence estimator.

####  4.4.3 Effectiveness of Stage-Wise Training.

In Table 5, we report the performance difference between end-to-end training
and stage-wise training. We can observe that ConMatch-P has obtained
meaningful enhancements in both training schemes, but stage-wise training
shows more larger gap between baseline.

####  4.4.4 Architecture.

Here we analyze the key components of ConMatch, the confidence estimator and
guided consistency regularization as shown in Table 6. For the fair
comparison, we construct three branches on FixMatch [47] as baseline (I), one
branch for a weakly-augmented sample and two branches for strongly-augmented
samples. (II) uses logit-level self-supervised loss, but not weighted by
confidence, i.e., $\mathcal{L}_{\mathrm{ccr}}$ with $c_{i}(r),c_{j}(r)=1/2$.
(III) and (IV) weight confidences of strongly-augmented instances to logit-
level self-supervised loss. (III) only takes logits as an input of confidence
estimator while both logits and features are fed into (IV). The result of this
ablation study shows that logit-level self-supervised loss without confidence
guidance causes network collapse. The collapse occurred in (II) is one of the
reasons why other semi-supervised methods [33] could not use self-supervision
at logit-level and should use negative pairs. (III) and (IV) show a
significant performance improvement compared to (I) without such collapse.

(a) Precision  (b) Recall  (c) F1-score

Figure 3: Plots of evolution of pseudo-labeling between ours and baselines
[47, 58] as training progresses on CIFAR-10 [28] with 40 labels: in terms of
(a) Precision, (b) Recall, and (c) F1-Score.

####  4.4.5 Evaluating Confidence Estimation.

To evaluate the effectiveness of our confidence estimator, we measure
precision, recall, and F1-score of ConMatch and FixMatch [47] as evolving the
training iterations on CIFAR-10 [28] with 40 labels as shown in Fig. 3. The
confident sample is defined as an unlabeled sample having max probability over
than threshold in the baseline and confidence measures over than 0.5 in
ConMatch. The quality of the confident sample is important to determine
precisely to prevent the confirmation bias problem, significantly degrading
the performances. The three classification metric, precision, recall and
F1-score, are effective to evaluate the quality of the confidence. By Fig. 3,
we can observe that ConMatch, starting from the scratch for the fair
comparison, shows higher values in all metric compared to the baseline.

(a) (b) (c) (d)

Figure 4: Convergence analysis of baselines [47, 58] and ConMatch: A
comparison of top-1-accuracy and loss between FixMatch [47] and ConMatch w/
[47] are shown at (a) and (b). A comparison between FlexMatch [58] and
ConMatch w/[58] is shown at (c) and (d). Evaluations are done every 200K
iterations on CIFAR-10 with 40 labels.

###  4.5 Analysis

####  4.5.1 Convergence Speed.

One of the advantages of our ConMatch is its superior convergence speed. Based
on the results as shown in Fig. 4 (b) and (d), the loss of ConMatch decreases
much faster and smoother than corresponding baseline [47], demonstrating our
superior convergence speed. Furthermore, the result of the accuracy in Fig. 4
(a) also proves that the global optimum is quickly reached. We also prove our
effectiveness of our method by comparing the another baseline, FlexMatch [58].
The convergence speed gap is relatively smaller than FixMatch since it
dynamically adjust class-wise thresholds at each time step, leading to the
stable training, but ConMatch achieves fast convergence at all time step from
the early phase where the predictions of the model are still unstable. It is
manifest that the introduction of ConMatch successfully encourages the model
to proactively improve the overall learning effect.

##  5 Conclusion

In this paper, we have proposed a novel semi-supervised learning framework
built upon conventional consistency regularization frameworks with an
additional strong branch to define the proposed confidence-guided consistency
loss between two strong branches. To account for the direction of such
consistency loss, we present confidence measures in non-parametric and
parametric approaches. Also, we also presented a stage-wise training to boost
the convergence of training. Our experiments have shown that our framework
boosts the performance of base semi-supervised learners, and is clearly state-
of-the-art on several benchmarks.

####  5.0.1 Acknowledgements.

This research was supported by the MSIT, Korea (IITP-2022-2020-0-01819, ICT
Creative Consilience program), and National Research Foundation of Korea
(NRF-2021R1C1C1006897).

Appendix

In this supplementary document, we provide additional experimental results and
implementation details to complement the main paper. The source code and a
pre-trained model will be released in the near future.

## Appendix A. Implementation Details - Hyper-parameters

For a fair comparison, we basically followed the default hyperparameters of
our baselines, i.e., FixMatch [47] and FlexMatch [58] as demonstrated in the
Sec 4.2 of the main paper. In addition to this, we also provide the best
hyperparameters for the specific loss functions and each training stage, as
shown in Table 1.

Table 1: Additional hyper-parameter list of ConMatch Stage | hyper-parameters
---|---
Feature encoder pre-training | $\lambda_{\textrm{{sup}}}$ | 1.0
$\lambda_{\textrm{{un}}}$ | 1.0
Confidence estimator pre-training | $\lambda_{\textrm{{conf}}}$ | 0.1
$\lambda_{\textrm{{conf-sup}}}$ | 1.0
Fine-tuning | $\lambda_{\textrm{{sup}}}$ | 1.0
$\lambda_{\textrm{{un}}}$ | 1.0
$\lambda_{\textrm{{ccr}}}$ | 1.0
$\lambda_{\textrm{{conf}}}$ | 1.0
$\lambda_{\textrm{{conf-sup}}}$ | 1.0

## Appendix B. Additional Experimental Results

####  5.0.2 Results of ConMatch with FixMatch Baseline

Since our framework basically works as a plug-and-play module, it can be
combined with various semi-supervised learners. We experimented FlexMatch
[58]-based ConMatch (ConMatch with FlexMatch) on general semi-supervised
learning benchmarks in Table 1 and Table 2 of the main paper. In this
supplementary material, we additionally provide the benchmark results of
FixMatch [47]-based ConMatch (ConMatch with FixMatch) on SVHN and STL-10
datasets. On SVHN with 40 labels and 250 labels, ConMatch with FixMatch
records the state-of-the-art accuracy of 96.75% and 98.14% respectively as
displayed in Table 2. We could observe that ConMatch with FixMatch shows
similar or even better results than ConMatch with FlexMatch on SVHN dataset.
This is mainly due to the weakness of FlexMatch [58] on the unbalanced
dataset. While training feature encoder of FlexMatch on SVHN, class-wise
imbalance of SVHN leads the classes with fewer samples to have low thresholds.
Such low thresholds allow noisy pseudo-labeled samples to be learned
throughout the training process and eventually reduce the accuracy of model’s
prediction. A feature encoder of FixMatch, on the other hand, is not affected
by class-wise imbalance of dataset, since it fixes its threshold at 0.95 to
filter out noisy samples. Consequently, this performance gap between two
encoders caused by class-wise imbalance of dataset allows ConMatch with
FixMatch to achieve similar or even higher accuracy score than ConMatch with
FlexMatch.

Table 2:  Comparison on error rates on SVHN [37] and STL-10 [12] benchamarks
on 3 different folds.

| SVHN | STL-10
---|---|---
Method | 40 | 250 | 1,000
FixMatch [47] | 3.96$\pm$2.17 | 2.48$\pm$0.38 | 7.98$\pm$1.50
FlexMatch [58] | 4.97$\pm$0.06 | 4.98$\pm$0.09 | 5.77$\pm$0.18
ConMatch w/FixMatch [47] | 3.25$\pm$1.06 | 1.86$\pm$0.06 | 5.90$\pm$0.17
ConMatch w/FlexMatch [58] | 3.14$\pm$0.57 | 3.13$\pm$0.72 | 5.26$\pm$0.04

####  5.0.3 Class-wise Results with Other SSL Techniques

In Table 2 and Table 3 of the main paper, we demonstrated that semi-supervised
learners [47, 58] combined with ConMatch outperform their baselines by a
significant margin in most SSL benchmark settings. Additionally, to provide a
detailed analysis for our performance results on CIFAR-10, we also performed a
per-class quantitative evaluation as shown in Table 3. ConMatch w/ [47]
achieves performance improvement over FixMatch in all the classes except for
the dog class. Although FixMatch seems to predict dog class better than any
other models, its prediction for the cat class, which is very similar to dog,
obtains significantly low accuracy score. This result shows that Fixmatch
suffers from a confirmation bias, and its prediction accuracy for dog class is
a distorted value. ConMatch w/ [47] on the other hand, records high accuracy
score for all classes including cat class. This verifies that ConMatch w/ [47]
effectively alleviate the confirmation bias by making use of confidence
estimator. Furthermore, ConMatch w/ [58] outperforms its baseline [58] on all
classes. Based on all these class-wise evaluation and comparison results, we
could confirm the effectiveness of combining ConMatch with existing semi-
supervised learners.

Table 3: Per-class quantitative evaluation on ConMatch and base semi-supervised learners [47, 58] on CIFAR-10 with 40 labels. The best results are in bold. Methods | air. | auto. | bird | cat | deer | dog | frog | horse | ship | truck | total
---|---|---|---|---|---|---|---|---|---|---|---
FixMatch [47] | 94.3 | 97.6 | 72.6 | 28.1 | 96.6 | 94.1 | 98.1 | 95.6 | 97.5 | 96.6 | 87.11
FlexMatch [58] | 96.8 | 98.1 | 92.3 | 88.0 | 95.8 | 88.7 | 98.3 | 97.1 | 98.4 | 97.2 | 95.07
ConMatch w/[47] | 97.2 | 97.7 | 90.9 | 86.0 | 96.9 | 88.6 | 98.7 | 97.2 | 97.6 | 97.9 | 94.87
ConMatch w/[58] | 97.1 | 98.5 | 92.9 | 88.1 | 96.0 | 91.8 | 98.9 | 97.6 | 98.7 | 97.4 | 95.70

####  5.0.4 Convergence Time.

In this section, we analyze the effect of performance-boosting of ConMatch on
end-to-end training. Specifically, we analyze the time taken to achieve the
best accuracy of FixMatch [47], 86.19% in CIFAR-10 40 labels for a fair
comparison between FixMatch [47], FlexMatch [58], and ConMatch due to their
different convergence speeds. As seen from Table 4, FixMatch [47] takes 261.9k
iters to converge, 1,699 minutes, while ConMatch w/FixMatch [47] converges at
27.1k iters, 117 minutes, which means that ConMatch can boost convergence
about 14.5 times faster. Additionally, ConMatch w/FlexMatch [58] converges at
20.5k iters, 89 minutes, which is 1.5 times faster than FlexMatch [58] which
converges at 44.6k iters, 138 minutes.

Table 4:  Comparison of Convergence Time between FixMatch and ConMatch
w/FixMatch Convergence time means the time taken to reach the highest accuracy
of FixMatch, 86.19%, in CIFAR-10 40 labels. Runtime is measured in the same
setting using RTX 3090 Ti. The best results are in bold.

Method | Iter | Convergence Time
---|---|---
FixMatch | 261.9k | 1,699 mins
ConMatch w/FixMatch | 27.1k | 117 mins
FlexMatch | 44.6k | 138 mins
ConMatch w/FlexMatch | 20.3k | 89 mins

Table 5:  Ablation study on warm_up iterations for feature encoder. The best
results are in bold.

Warm_up iter. | CIFAR-10
---|---
40
0 | 93.24
1000 | 93.78
4000 | 94.87
5000 | 93.29
10000 | 93.10
20000 | 92.34

## Appendix C. Ablation Studies

####  5.0.5 Warm-up Iteration

As mentioned in Sec.3.5 of the main paper, we adopt the warm-up stage when
training, to stabilize the confidence estimator at the early stage of training
and to boost the convergence of training. Therefore, it can be the question
that how many iterations of pre-training are we needed, so we perform an
ablation study of these warm-up iterations at CIFAR-10 40 labels setting, and
the results are shown in Table 5. After warm-up iteration, we perform the pre-
training stage for confidence estimator for 10,000 iterations equally for all
ablation settings except for the end-to-end training strategy. While without
warm-up stage training results the accuracy of 93.24%, pre-training the
feature extractor 10,000 iterations and the confidence estimator 10,000
iterations results 94.87% accuracy, showing 1.63% improvement compared to the
end-to-end training strategy.

(a) Basic. (b) Reduced. (c) Top-k. (d) BatchNorm.

Figure 1: Network Architecture Variants for Confidence estimator: (a) Basic-
The number of parameters of linear layers is always constant. (b) Reduced-The
number of parameters decreases to a multiplier of 2 according to the depth of
the linear layers. (c)Top-k - The top-k values of the classifier outputs are
only used for confidence estimator, by masking class values that do not belong
to the top-k. (d)BatchNorm.-Batch normalization layer is followed by the
linear layer.

####  5.0.6 Designing Confidence Estimator Network

The overall confidence estimation network, consisting of two sub-networks, is
built as the collection of linear layer and ReLU activations. To input the
logit values and features into the confidence estimator, it includes feature
projection embedding and logit projection embedding. For the first variant of
the confidence estimator, as shown in Fig. 1(b), we reduce the channel
dimension of each linear layer of the confidence estimator by a factor of two,
which is called reduced version. It shows 93.26% accuracy with the same
evaluation setting, which is 0.91% increased results compared to the basic
architecture as shown in Table 7 (I) and (II). Given the distributions of all
classes from the classifier, we find that passing a part of it, especially
masking the lower values of logit makes the estimator more robust and working
well, shown in Fig. 1(c). The performance gap between (II) and (III) proves
that the top-k approach is effective for the confidence estimator. We also
find that the best top-k value is set by k=5 and that masking top-k, logit
dimension is same as the number of class but masking class value, not
belonging to top-k, is better than reducing top-k, logit dimension is same as
top-k. For the final variant of the confidence estimator, we test the effect
of BatchNorm [23] that came after the linear layer and a detailed architecture
for this version is shown in Fig. 1(d). As shown in Table 7, (IV) record the
best accuracy, 2.52%, 1.61%, and 0.93% better than (I), (II) and (III),
respectively.

####  5.0.7 Similarity Functions in Non-parametric Approach

In this section, we analyze the similarity function of the confidence
estimation module in a non-parametric approach. We assume that the confidence
of a strongly-augmented sample can be represented by the similarity between a
weakly-augmented sample $\alpha(r)$ and a strongly-augmented sample
$\mathcal{A}(r)$ in a probabilistic way. As seen in Eq.(4), we define a
similarity function as a cross-entropy loss between the logit of weakly-
augmented sample and the logit of the strongly-augmented sample. The
similarity function can be substituted in other ways. Specifically, it can be
formulated by L2 distance like
$\|p_{\mathrm{model}}(y|\mathcal{A}(r);\theta)-p_{\mathrm{model}}(y|\alpha(r);\theta_{\mathrm{freeze}})\|$
or feature cosine-similarity like
$\frac{p_{\mathrm{model}}(y|\mathcal{A}(r);\theta)\cdot
p_{\mathrm{model}}(y|\alpha(r);\theta_{\mathrm{freeze}})}{\|p_{\mathrm{model}}(y|\mathcal{A}(r);\theta)\|\|p_{\mathrm{model}}(y|\alpha(r);\theta_{\mathrm{freeze}})\|}$
where $p_{\mathrm{model}}$ is considered as logit and feature, respectively.
Table 7 shows the results of replacing the similarity function with the L2
distance of logits and feature cosine-similarity. Through the result, we can
find that the similarity function based on cross-entropy is the most effective
confidence measure, compared to others.

Table 6:  Ablation study on network architecture variants for confidence
estimator.The best results are in bold.

| Network Arch. for Confidence estimator | CIFAR-10
---|---|---
Basic | Reduced | Top-K | BatchNorm | 40
(I) | ✓ | ✗ | ✗ | ✗ | 92.35
(II) | ✗ | ✓ | ✗ | ✗ | 93.26
(III) | ✗ | ✓ | ✓ | ✗ | 93.94
(IV) | ✗ | ✓ | ✓ | ✓ | 94.87

## Appendix D. Visualization

####  5.0.8 Visualization of AUC-ROC curve of Confidence.

The area under the ROC curve (AUC) is used for measuring the accuracy of a
classifier and the better confidence prediction is closer to 1. On confidence
estimation, it is also used to evaluate the capability of the confidence
measure to distinguish correct class assignments from erroneous ones. As shown
in Fig. 2, for overall training iterations, ConMatch w/ [47] and ConMatch w/
[58] show the higher AUC-ROC value compared to each baseline, FixMatch [47],
and FlexMatch [58]. This explains that the confidence estimator measures the
confidences of augmented instances more properly than FixMatch and FlexMatch,
which use max class probability as the confidence measure. This will be also
proven in the following section.

Table 7:  Ablation study on similarity functions in non-parametric
approach.The best results are in bold.

Similarity Function | Target | CIFAR-10
---|---|---
40
Cross Entropy | logit | 4.89
L2 Norm | logit | 4.93
Cosine Similarity | feature | 5.02

Figure 2: AUC-ROC Curve of ConMatch with various baselines The orange line
means FlexMatch w/ ConMatch, the blue line means FixMatch w/ ConMatch, the red
line means FlexMatch, and the green line means original FixMatch,
respectively.

####  5.0.9 Visualization on Transition of Confidence.

To explain the effectiveness of ConMatch, we show the feature distribution of
ConMatch w/FixMatch [47], ConMatch w/FlexMatch [58], compared to [47] and [58]
at different training iterations in Table 8, Table 9, Table 10 and Table 11,
respectively. For visualizing the high dimensional features, the complex
feature matrices are transformed into two-dimensional points based on t-SNE
[35]. As training progresses, we can confirm that t-SNE visualization of
labeled and weakly-augmented samples are clustered into distinct groups,
relatively faster than baseline [47] by comparing Table 8 and Table 9. And we
can also find that confident strongly-augmented samples are also well-
organized near confident weakly-augmented samples.

The color of dots means the confidence of each sample. The darkest color dot
means the high-confidence instances, and the empty dot means the low- or zero-
confidence instances, whose confidence values are relatively lower. For
ConMatch, we consider the confidence estimator output as the confidence of
each instance, and for the FixMatch [47] and FlexMatch [58], we consider the
maximum probability of the output distribution as the confidence of each
instance. Both methods, ConMatch and FixMatch / FlexMatch show that the high-
confident strongly-augmented instances are closer to high-confident weakly-
augmented instances than the low-confident strongly-augmented instances. About
FixMatch and FlexMatch, they measure the confidence of the insufficiently
clustered strongly augmented samples as relatively high around $0.5$,
expressed as the light-blue color of figures. Otherwise, different to FixMatch
and FlexMatch, ConMatch shows sharp changes in confidence values,
distinguishing the high- and low-confidence instances definitely so the
insufficiently clustered samples have low confidence values around zero,
expressed as the empty circle of figures, even from the early stage of
training. By this visualization, we can confirm that our confidence estimator
effectively measures the confidence of samples and makes possible the training
between the strongly augmented images.

Con+Fix | Labeled samples | Weakly augmented samples | Strongly augmented samples
---|---|---|---
10000 iter |  |  |
30000 iter |  |  |
50000 iter |  |  |
70000 iter |  |  |
90000 iter |  |  |

Table 8: Visualization on the transitions feature distribution with confidence
in ConMatch w/FixMatch (denoted as blue dots in colors) of training samples
from CIFAR-10 in the feature space at different training stages. Data
projection in 2-D space is attained by t-SNE based on the feature
representation. The dark blue dots represent the most confident samples, and
the empty dots represent the least confident samples.

FixMatch | Labeled samples | Weakly augmented samples | Strongly augmented samples
---|---|---|---
10000 iter |  |  |
30000 iter |  |  |
50000 iter |  |  |
70000 iter |  |  |
90000 iter |  |  |

Table 9: Visualization on the transitions feature distribution with confidence
in FixMatch. The details are same as above.

Con+Flex | Labeled samples | Weakly augmented samples | Strongly augmented samples
---|---|---|---
10000 iter |  |  |
30000 iter |  |  |
50000 iter |  |  |
70000 iter |  |  |
90000 iter |  |  |

Table 10: Visualization on the transitions feature distribution with
confidence in ConMatch w/FlexMatch. The details are same as above.

FlexMatch | Labeled samples | Weakly augmented samples | Strongly augmented samples
---|---|---|---
10000 iter |  |  |
30000 iter |  |  |
50000 iter |  |  |
70000 iter |  |  |
90000 iter |  |  |

Table 11: Visualization on the transitions feature distribution with
confidence in FlexMatch. The details are same as above.

## References

  * [1] Arazo, E., Ortego, D., Albert, P., O’Connor, N.E., McGuinness, K.: Pseudo-labeling and confirmation bias in deep semi-supervised learning. In: IJCNN (2020)
  * [2] Bachman, P., Alsharif, O., Precup, D.: Learning with pseudo-ensembles. In: NeurIPS (2014)
  * [3] Bachman, P., Hjelm, R.D., Buchwalter, W.: Learning representations by maximizing mutual information across views. In: NeurIPS (2019)
  * [4] Berthelot, D., Carlini, N., Cubuk, E.D., Kurakin, A., Sohn, K., Zhang, H., Raffel, C.: Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. arXiv:1911.09785 (2019)
  * [5] Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: Mixmatch: A holistic approach to semi-supervised learning. In: NeurIPS (2019)
  * [6] Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: NeurIPS (2020)
  * [7] Chapelle, O., Zien, A.: Semi-supervised classification by low density separation. In: AISTATS workshops (2005)
  * [8] Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: ICML (2020)
  * [9] Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.E.: Big self-supervised models are strong semi-supervised learners. In: NeurIPS (2020)
  * [10] Chen, X., He, K.: Exploring simple siamese representation learning. In: CVPR (2021)
  * [11] Choi, H., Lee, H., Kim, S., Kim, S., Kim, S., Sohn, K., Min, D.: Adaptive confidence thresholding for monocular depth estimation. In: ICCV (2021)
  * [12] Coates, A., Ng, A., Lee, H.: An analysis of single-layer networks in unsupervised feature learning. In: AISTATS (2011)
  * [13] Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: Practical automated data augmentation with a reduced search space. In: CVPR workshops (2020)
  * [14] Dawid, A.P.: The well-calibrated bayesian. JASA (1982)
  * [15] DeGroot, M.H., Fienberg, S.E.: The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician) (1983)
  * [16] Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: Decaf: A deep convolutional activation feature for generic visual recognition. In: ICML (2014)
  * [17] French, G., Mackiewicz, M., Fisher, M.: Self-ensembling for visual domain adaptation. arXiv:1706.05208 (2017)
  * [18] Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. arXiv:1803.07728 (2018)
  * [19] Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. In: NeurIPS (2004)
  * [20] Grill, J.B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al.: Bootstrap your own latent-a new approach to self-supervised learning. In: NeurIPS (2020)
  * [21] Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: ICML (2017)
  * [22] He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: CVPR (2020)
  * [23] Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: ICML (2015)
  * [24] Ke, Z., Wang, D., Yan, Q., Ren, J., Lau, R.W.: Dual student: Breaking the limits of the teacher in semi-supervised learning. In: ICCV (2019)
  * [25] Kim, B., Choo, J., Kwon, Y.D., Joe, S., Min, S., Gwon, Y.: Selfmatch: Combining contrastive self-supervision and consistency for semi-supervised learning. arXiv:2101.06480 (2021)
  * [26] Kim, S., Min, D., Kim, S., Sohn, K.: Unified confidence estimation networks for robust stereo matching. TIP (2018)
  * [27] Kim, S., Min, D., Kim, S., Sohn, K.: Adversarial confidence estimation networks for robust stereo matching. T-ITS (2020)
  * [28] Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
  * [29] Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. arXiv:1610.02242 (2016)
  * [30] Lee, D.H., et al.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: ICML workshops (2013)
  * [31] Lee, D., Kim, S., Kim, I., Cheon, Y., Cho, M., Han, W.S.: Contrastive regularization for semi-supervised learning. arXiv:2201.06247 (2022)
  * [32] Lerner, B., Shiran, G., Weinshall, D.: Boosting the performance of semi-supervised learning with unsupervised clustering. arXiv:2012.00504 (2020)
  * [33] Li, J., Xiong, C., Hoi, S.C.: Comatch: Semi-supervised learning with contrastive graph regularization. In: ICCV (2021)
  * [34] Lucas, T., Weinzaepfel, P., Rogez, G.: Barely-supervised learning: Semi-supervised learning with very few labeled images. arXiv:2112.12004 (2021)
  * [35] Va der Maaten, L., Hinton, G.: Visualizing data using t-sne. JMLR (2008)
  * [36] Miyato, T., Maeda, S.i., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. TPAMI (2018)
  * [37] Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011)
  * [38] Noroozi, M., Favaro, P.: Unsupervised learning of visual representations by solving jigsaw puzzles. In: ECCV (2016)
  * [39] Park, S., Park, J., Shin, S.J., Moon, I.C.: Adversarial dropout for supervised and semi-supervised learning. In: AAAI (2018)
  * [40] Pham, H., Dai, Z., Xie, Q., Le, Q.V.: Meta pseudo labels. In: CVPR (2021)
  * [41] Poggi, M., Mattoccia, S.: Learning from scratch a confidence measure. In: BMVC (2016)
  * [42] Rizve, M.N., Duarte, K., Rawat, Y.S., Shah, M.: In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. arXiv:2101.06329 (2021)
  * [43] Sajjadi, M., Javanmardi, M., Tasdizen, T.: Mutual exclusivity loss for semi-supervised deep learning. In: ICIP (2016)
  * [44] Sajjadi, M., Javanmardi, M., Tasdizen, T.: Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In: NeurIPS (2016)
  * [45] Seki, A., Pollefeys, M.: Patch based confidence prediction for dense disparity map. In: BMVC (2016)
  * [46] Shi, W., Gong, Y., Ding, C., Tao, Z.M., Zheng, N.: Transductive semi-supervised deep learning using min-max features. In: ECCV (2018)
  * [47] Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C.A., Cubuk, E.D., Kurakin, A., Li, C.L.: Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In: NeurIPS (2020)
  * [48] Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: NeurIPS (2017)
  * [49] Tosi, F., Poggi, M., Benincasa, A., Mattoccia, S.: Beyond local reasoning for stereo confidence estimation with deep learning. In: ECCV (2018)
  * [50] Verma, V., Kawaguchi, K., Lamb, A., Kannala, J., Bengio, Y., Lopez-Paz, D.: Interpolation consistency training for semi-supervised learning. arXiv:1903.03825 (2019)
  * [51] Xie, Q., Dai, Z., Hovy, E., Luong, T., Le, Q.: Unsupervised data augmentation for consistency training. In: NeurIPS (2020)
  * [52] Xie, Q., Dai, Z., Hovy, E., Luong, T., Le, Q.: Unsupervised data augmentation for consistency training. In: NeurIPS (2020)
  * [53] Xie, Q., Luong, M.T., Hovy, E., Le, Q.V.: Self-training with noisy student improves imagenet classification. In: CVPR (2020)
  * [54] Xu, Y., Shang, L., Ye, J., Qian, Q., Li, Y.F., Sun, B., Li, H., Jin, R.: Dash: Semi-supervised learning with dynamic thresholding. In: ICML (2021)
  * [55] Yalniz, I.Z., Jégou, H., Chen, K., Paluri, M., Mahajan, D.: Billion-scale semi-supervised learning for image classification. arXiv:1905.00546 (2019)
  * [56] Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv:1605.07146 (2016)
  * [57] Zhai, X., Oliver, A., Kolesnikov, A., Beyer, L.: S4l: Self-supervised semi-supervised learning. In: ICCV (2019)
  * [58] Zhang, B., Wang, Y., Hou, W., Wu, H., Wang, J., Okumura, M., Shinozaki, T.: Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. In: NeurIPS (2021)
  * [59] Zhang, L., Qi, G.J.: Wcp: Worst-case perturbations for semi-supervised deep learning. In: CVPR (2020)
  * [60] Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: ECCV (2016)
  * [61] Zoph, B., Ghiasi, G., Lin, T.Y., Cui, Y., Liu, H., Cubuk, E.D., Le, Q.: Rethinking pre-training and self-training. In: NeurIPS (2020)