# Transductive Decoupled Variational Inference for Few-Shot Classification

Anuj Singh 1, Hadi Jamali-Rad 1, 2

###### Abstract

The versatility to learn from a handful of samples is the hallmark of human
intelligence. Few-shot learning is an endeavour to transcend this capability
down to machines. Inspired by the promise and power of probabilistic deep
learning, we propose a novel variational inference network for few-shot
classification (coined as TRIDENT) to decouple the representation of an image
into _semantic_ and _label_ latent variables, and simultaneously infer them in
an intertwined fashion. To induce _task-awareness_ , as part of the inference
mechanics of TRIDENT, we exploit information across both query and support
images of a few-shot task using a novel built-in attention-based transductive
feature extraction module (we call AttFEX). Our extensive experimental results
corroborate the efficacy of TRIDENT and demonstrate that, using the simplest
of backbones, it sets a new state-of-the-art in the most commonly adopted
datasets _mini_ ImageNet and _tiered_ ImageNet (offering up to $4\%$ and $5\%$
improvements, respectively), as well as for the recent challenging cross-
domain miniImagenet $\rightarrow$ CUB scenario offering a significant margin
(up to $20\%$ improvement) beyond the best existing cross-domain
baselines.111Code and experimentation can be found in our GitHub repository:
https://github.com/anujinho/trident

## Introduction

Deep learning algorithms are usually data hungry and require massive amounts
of training data to reach a satisfactory level of performance on any task. To
tackle this limitation, few-shot classification aims to learn to classify
images from various unseen tasks in a data-deficient setting. In this exciting
space, metric learning proposes to learn a shared feature extractor to embed
the samples into a metric space of class prototypes (Sung et al. 2018; Vinyals
et al. 2016; Snell, Swersky, and Zemel 2017; Wang et al. 2019; Liu, Song, and
Qin 2020; Bateni et al. 2020). Due to limited data per class, these prototypes
suffer from sample-bias and fail to efficiently represent class
characteristics. Furthermore, sharing a feature extractor across tasks implies
that the discriminative information learnt from the seen classes are equally
effective on any arbitrary unseen classes, which is not true in most cases.
_Task-aware_ few-shot learning approaches (Bateni et al. 2022; Ye et al. 2020)
address these limitations by exploiting information hidden in the unlabeled
data. As a result, the model learns task-specific embeddings by aligning the
features of the labelled and unlabelled task instances for optimal distance
metric based label assignment. Since the alignment of these embeddings is
still subject to the relevance of the characteristics captured by the shared
feature extractors, task-aware methods sometimes fail to extract meaningful
representations particularly relevant to classification. _Probabilistic_
methods address sample-bias by relaxing the need to find point estimates to
approximate data-dependent distributions of either high-dimensional model
weights (Nguyen, Do, and Carneiro 2019; Ravi and Beatson 2019; Gordon et al.
2019; Hu et al. 2020) or lower-dimensional class prototypes (Sun et al. 2021;
Zhang et al. 2019). However, inferring a high-dimensional posterior of model
parameters is inefficient in low-data regimes and estimating distributions of
class prototypes involves using hand-crafted non-parametric aggregation
techniques which may not be well suited for every unseen task.

\begin{overpic}[abs,unit=1cm,width=430.00958pt]{images/Intuition.png}
\end{overpic} Figure 1: High-level process flow of TRIDENT. Inferred label
latent variable $\textbf{z}_{l}$ contains class-characterizing information, as
is reflected by better separation of the distributions when compared to their
semantic latent counterparts $\textbf{z}_{s}$. AttFEX module generates task-
aware feature maps by exploiting information from both support and query
images, which compensates for the lack of label vectors $Y$ in inferring
$\textbf{z}_{l}$.

Although fit for purpose, all these approaches seem to overlook an important
perspective. An image is composed of different attributes such as style,
design, and context, which are not necessarily relevant discriminative
characteristics for classification. Here, we refer to these attributes as
_semantic_ information. On the other hand, other class-characterizing
attributes (such as wings of a bird, trunk of an elephant, hump on a camel’s
back) are critical for classification, irrespective of context. We refer to
such attributes as _label_ information. Typically, contextual information is
majorly governed by semantic attributes, whereas the label characteristics are
subtly embedded throughout an image. In other words, semantic information can
be predominantly present across an image, whereas _attending_ to subtle label
information determines how effective a classification algorithm would be.
Thus, we argue that attention to label-specific information should be
ingrained into the mechanics of the classifier, decoupling it from semantic
information. This becomes even more important in a few-shot setting where the
network has to quickly learn from little data. Building upon this idea, we
propose transductive variational inference of decoupled latent variables
(coined as TRIDENT), to simultaneously infer decoupled label and semantic
information using two intertwined variational networks. To induce task-
awareness while constructing the variational inference mechanics of TRIDENT,
we introduce a novel atention-based transductive feature extraction module (we
call AttFEX) which further enhances the discriminative power of the inferred
label attributes. This way TRIDENT infers distributions instead of point
estimates and injects a handcrafted inductive-bias into the network to guide
the classification process. Our main contributions can be summarized as:

  1. 1.

We propose TRIDENT, a variational inference network to simultaneously infer
two salient _decoupled_ attributes of an image (_label_ and _semantic_), by
inferring these two using two intertwined variational sub-networks (Fig. 1).

  2. 2.

We introduce an attention-based transductive feature extraction module,
AttFEX, to enable TRIDENT see through and compare all images within a task,
inducing task-cognizance in the inference of label information.

  3. 3.

We perform extensive evaluations to demonstrate that TRIDENT sets a new state-
of-the-art by outperforming all existing baselines on the most commonly
adopted datasets _mini_ Imagenet and _tiered_ Imagenet (up to $4\%$ and
$5\%$), as well as for the challenging cross-domain scenario of miniImagenet
$\rightarrow$ CUB (up to $20\%$ improvement).

## Related Work

Metric-based learning. This body of work revolves around mapping input samples
into a lower-dimensional embedding space and then classifying the unlabelled
samples based on a distance or similarity metric. By parameterizing these
mappings with neural networks and using differentiable similarity metrics for
classification, these networks can be trained in an episodic manner (Vinyals
et al. 2016) to perform few-shot classification. Prototypical Nets (Snell,
Swersky, and Zemel 2017), Simple Shot (Wang et al. 2019), Relation Networks
(Sung et al. 2018), Matching Networks (Vinyals et al. 2016) variants of Graph
Neural Nets (Satorras and Estrach 2018; Yang et al. 2020), Simple CNAPS
(Bateni et al. 2020), are a few examples of seminal ideas in this space.

Transductive Feature-Extraction and Inference. Transductive feature extraction
or task-aware learning is a variant of the metric-learning with an adaptation
mechanism that _aligns_ support and query feature vectors in the embedding
space for better representation of task-specific discriminative information.
This not only improves the discriminative ability of classifiers across tasks,
but also alleviates the problem of overfitting on limited support set since
information from the query set is also used for extracting features of images
in a task. CNAPS (Requeima et al. 2019), Transductive-CNAPS (Bateni et al.
2022), FEAT (Ye et al. 2020), Assoc-Align (Afrasiyabi, Lalonde, and Gagné
2020), TPMN (Wu et al. 2021) and CTM (Li et al. 2019) are prime examples of
such methods. Next to transduction for task-aware feature extraction, there
are methods that use _transductive inference_ to classify all the query
samples at once by jointly assigning them labels, as opposed to their
inductive counterparts where prediction is done on the samples one at a time.
This is either done by iteratively propagating labels from the support to the
query samples or by fine-tuning a pre-trained backbone using an additional
entropy loss on all query samples, which encourages confident class
predictions at query samples. TPN (Liu et al. 2019), Ent-Min (Dhillon et al.
2020), TIM (Boudiaf et al. 2020), Transductive-CNAPS (Bateni et al. 2022),
LaplacianShot (Ziko et al. 2020), DPGN (Yang et al. 2020) and ReRank (SHEN et
al. 2021) are a few notable examples in this space that usually report state-
of-the-art results in certain few-shot classification settings (Liu et al.
2019).

Optimization-based meta-learning. These methods search for model parameters
that are sensitive to task objective functions for fast gradient-based
adaptation to new tasks. MAML (Finn, Abbeel, and Levine 2017), its variants
(Rajeswaran et al. 2019; Nichol, Achiam, and Schulman 2018a) and SNAIL (Mishra
et al. 2018) are a few prominent examples while LEO (Rusu et al. 2019)
efficiently meta-updates its parameters in a lower dimensional latent space.

Probabilistic learning. The estimated parameters of typical gradient-based
meta-learning methods discussed earlier (Finn, Abbeel, and Levine 2017; Rusu
et al. 2019; Mishra et al. 2018; Nichol, Achiam, and Schulman 2018a;
Rajeswaran et al. 2019), have high variance due to the small task sample size.
To deal with this, a natural extension is to model the uncertainty by treating
these parameters as latent variables in a Bayesian framework as proposed in
Neural Statistician (Edwards and Storkey 2017), PLATIPUS (Finn, Xu, and Levine
2018), VAMPIRE (Nguyen, Do, and Carneiro 2019), ABML (Ravi and Beatson 2019),
VERSA (Gordon et al. 2019), SIB (Hu et al. 2020), SAMOVAR (Iakovleva, Verbeek,
and Alahari 2020). Methods like ABPML (Sun et al. 2021) and VariationalFSL
(Zhang et al. 2019) infer latent variables of class prototypes to perform
classification and avoid inferring high-dimensional model parameters. ABPML
(Sun et al. 2021) and VariationalFSL (Zhang et al. 2019) are the closest to
our approach. In contrast to these two methods, we avoid hand-crafting class-
level aggregations, as well as we enhance variational inference by
incorporating an inductive bias through decoupling of label and semantic
information.

## Problem Definition

Consider a labelled dataset
$\mathcal{D}=\\{(\textbf{x}_{i},y_{i})\,|\,i\in[1,N^{\prime}]\\}$ of images
$\textbf{x}_{i}$ and class labels $y_{i}$. This dataset $\mathcal{D}$ is
divided into three disjoint subsets:
$\mathcal{D}=\\{\mathcal{D}^{tr}\,\cup\,\mathcal{D}^{val}\,\cup\,\mathcal{D}^{test}\\}$,
respectively, referring to the training, validation, and test subsets. The
validation dataset $\mathcal{D}^{val}$ is used for model selection and the
testing dataset $\mathcal{D}^{test}$ for final evaluation. Following standard
few-shot classification settings (Vinyals et al. 2016; Sung et al. 2018;
Snell, Swersky, and Zemel 2017), we use episodic training on a set of tasks
$\mathcal{T}_{i}\sim p(\mathcal{T})$. The tasks are constructed by drawing $K$
random samples from $N$ different classes, which we denote as an ($N$-way,
$K$-shot) task. Concretely, each task $\mathcal{T}_{i}$ is composed of a
support and a query set. The support set
$\mathcal{S}=\\{(\textbf{x}_{kn}^{S},y_{kn}^{S})\,|\,k\in[1,K],n\in[1,N]\\}$
contains $K$ samples per class and the query set
$\mathcal{Q}=\\{(\textbf{x}_{kn}^{Q},y_{kn}^{Q})\,|\,k\in[1,Q],n\in[1,N]\\}$
contains $Q$ samples per class. For a given task, the $NQ$ query and $NK$
support images are mutually exclusive to assess the generalization
performance.

## The Proposed Method: TRIDENT

Let us start with the high-level idea. The proposed approach is devised to
learn meaningful representations that capture two pivotal characteristics of
an image by modelling them as separate latent variables: (i) ${\bf z}_{s}$
representing semantics, and (ii) ${\bf z}_{l}$ embodying class labels.
Inferring these two latent variables simultaneously allows $\textbf{z}_{l}$ to
learn meaningful distributions of class-discriminating characteristics
decoupled from semantic features represented by $\textbf{z}_{s}$. We argue
that learning $\textbf{z}_{l}$ as the sole latent variable for classification
results in capturing a mixture of true label and other semantic information.
This in turn can lead to sub-optimal classification performance, especially in
a few-shot setting where the information per class is scarce and the network
has to adapt and generalize quickly. By inferring decoupled label and
semantics latent variables, we inject a handcrafted inductive-bias that
incorporates only relevant characteristics, and thus, ameliorates the
network’s classification performance.

Figure 2: Generative Model of TRIDENT. Dotted lines indicate variational
inference and solid lines refer to generative processes. The inference and
generative parameters are color coded to correspond to their respective
architectures indicated in Fig.1 and Fig.4.

### Generative Process

The directed graphical model in Fig. 2 illustrates the common underlying
generative process $p$ such that
$p_{i}=p(\textbf{x}_{i},y_{i}\,|\,\textbf{z}_{li},\textbf{z}_{si})$. For the
sake of brevity, in the following we drop the sample index $i$ as we always
refer to terms associated with a single data sample. We work on the logical
premise that the label latent variable $\textbf{z}_{l}$ is responsible for
generating class label as well as for image reconstruction, whereas the
semantic latent variable $\textbf{z}_{s}$ is only responsible for image
reconstruction (solid lines in the figure). Formally, the data is explained by
the generative processes:
$p_{\theta_{1}}(y\,|\,\textbf{z}_{l})=\textup{Cat}(y\,|\,\textbf{z}_{l})$ and
$p_{\theta_{2}}(\textbf{x}\,|\,\textbf{z}_{l},\textbf{z}_{s})=g_{\theta_{2}}(\textbf{x};\textbf{z}_{l},\textbf{z}_{s})$,
where Cat(.) refers to a multinomial distribution and
$g_{\theta_{2}}(\textbf{x};\textbf{z}_{l},\textbf{z}_{s})$ is a suitable
likelihood function such as a Gaussian or Bernoulli distribution. The
likelihoods of both these generative processes are parameterized using deep
neural networks and the priors of the latent variables are chosen to be
standard multivariate Gaussian distributions (Kingma and Welling 2014; Kingma
et al. 2014):
$p(\textbf{z}_{s})=\mathcal{N}(\textbf{z}_{s}\,|\,\textbf{0},\textbf{I})$ and
$p(\textbf{z}_{l})=\mathcal{N}(\textbf{z}_{l}\,|\,\textbf{0},\textbf{I})$.

### Variational Inference of Decoupled ${\bf Z}_{l}$ and ${\bf Z}_{s}$

Computing exact posterior distributions is intractable due to high
dimensionality and non-linearity of the deep neural network parameter space.
Following (Kingma and Welling 2014; Kingma et al. 2014), we instead construct
an approximate posterior over the latent variables by introducing a fixed-form
distribution $q(\textbf{z}_{l},\textbf{z}_{s}\,|\,\textbf{x},y)$ parameterized
by $\phi$. By using $q_{\phi}(.)$ as an inference network, the inference is
rendered tractable, scalable and amortized since $\phi$ now acts as the global
variational parameter. We assume $q_{\phi}$ has a factorized form
$q_{\phi}\left(\textbf{z}_{s},\textbf{z}_{l}\mid\textbf{x},y\right)=q_{\phi_{1}}\left(\textbf{z}_{l}\mid\textbf{x},\textbf{z}_{s}\right)q_{\phi_{2}}\left(\textbf{z}_{s}\mid\textbf{x}\right)$,
where $q_{\phi_{1}}(.),q_{\phi_{2}}(.)$ are assumed to be multivariate
Gaussian distributions. As is also depicted in Fig. 2, we use ${\bf z}_{s}$ as
input to $q_{\phi_{1}}(.)$ to infer ${\bf z}_{l}$ because of their conditional
dependence given ${\bf x}$. This way we forge a path to allow _necessary_
semantic latent information flow through the label inference network. On the
other hand, the opposite direction (using ${\bf z}_{l}$ to infer ${\bf
z}_{s}$) is unnecessary, because label information does not directly
contribute to the extraction of semantic features. We will further reflect on
this design choice in the next subsection. Neural networks are then used to
parameterize both inference networks as:

$\begin{split}&q_{\phi_{2}}\left(\textbf{z}_{s}\,|\,\textbf{x}\right)=\mathcal{N}\left(\textbf{z}_{s}\,|\,\boldsymbol{\mu}_{\phi_{2}}(\textbf{x}),diag(\boldsymbol{\sigma}^{2}_{\phi_{2}}(\textbf{x}))\right),\\\
&q_{\phi_{1}}\left(\textbf{z}_{l}\,|\,\textbf{x},\textbf{z}_{s}\right)=\mathcal{N}\left(\textbf{z}_{l}\,|\,\boldsymbol{\mu}_{\phi_{1}}(\textbf{x},\textbf{z}_{s}),diag(\boldsymbol{\sigma}^{2}_{\phi_{1}}(\textbf{x},\textbf{z}_{s}))\right).\end{split}$
(1)

To find the optimal _approximate_ posterior, we derive the evidence lower
bound (ELBO) on the marginal likelihood of the data to form our objective
function:

$\begin{split}&p(\textbf{x},y)=\iint
p(\textbf{x},y\,|\,\textbf{z}_{s},\textbf{z}_{l})\,p(\textbf{z}_{s,}\textbf{z}_{l})\,d\textbf{z}_{s}\,d\textbf{z}_{l},\\\
&=\mathbb{E}_{q(\textbf{z}_{s},\textbf{z}_{l}\,|\,x)}\left[\frac{p(\textbf{x}\,|\,\textbf{z}_{l},\textbf{z}_{s})p(y\,|\,\textbf{z}_{l})p(\textbf{z}_{l})p(\textbf{z}_{s})}{q(\textbf{z}_{l},\textbf{z}_{s}\,|\,\textbf{x})}\right].\\\
&\ln
p(\textbf{x},y)\geqslant\mathbb{E}_{q(\textbf{z}_{s},\textbf{z}_{l}|\textbf{x})}\left[\ln\left(\frac{p(\textbf{x}\,|\,\textbf{z}_{l},\textbf{z}_{s})p(y\,|\,\textbf{z}_{l})p(\textbf{z}_{l})p(\textbf{z}_{s})}{q(\textbf{z}_{s},\textbf{z}_{l}\,|\,\textbf{x})}\right)\right],\\\
&=\mathbb{E}_{q_{\phi_{2}}}\left[\mathbb{E}_{q_{\phi_{1}}}\left[\ln\left(\frac{p(\textbf{x}\,|\,\textbf{z}_{s},\textbf{z}_{l})p(y\,|\,\textbf{z}_{l})p(\textbf{z}_{s})p(\textbf{z}_{l})}{q(\textbf{z}_{s}\,|\,\textbf{x})q(\textbf{z}_{l}\,|\,\textbf{x},\textbf{z}_{s})}\right)\right]\right].\end{split}$

Denoting $\Psi=\left(\theta_{1},\theta_{2},\phi_{1},\phi_{2}\right)$, the ELBO
can be given by

$\begin{split}&\mathcal{L}(\Psi)=-\mathbb{E}_{q_{\phi_{2}}}\mathbb{E}_{q_{\phi_{1}}}\left[\ln
p_{\theta_{2}}(\textbf{x}\,|\,\textbf{z}_{s},\textbf{z}_{l})+\ln
p_{\theta_{1}}(y\,|\,\textbf{z}_{l})\right]+\\\
&D_{KL}\big{(}q_{\phi_{1}}(\textbf{z}_{l}\,|\,\textbf{x},\textbf{z}_{s})\|\,p(\textbf{z}_{l})\big{)}+D_{KL}\big{(}q_{\phi_{2}}(\textbf{z}_{s}\,|\,\textbf{x})\|\,p(\textbf{z}_{s})\big{)},\end{split}$
(2)

where the second line follows the graphical model in Fig 2, and
$\mathbb{E}(.)$ and $\ln(.)$ denote the expectation operator and the natural
logarithm, respectively. We avoid computing biased gradients by following the
re-parameterization trick from (Kingma and Welling 2014). Assuming Gaussian
distributions for the priors as well as the variational distributions allows
us to compute the KL Divergences of $\textbf{z}_{l}$ and $\textbf{z}_{s}$
(last two terms in (LABEL:eq:loss)) analytically (Kingma and Welling 2014). By
considering a multivariate Gaussian distribution and a multinomial
distribution as the likelihood functions for
$p_{\theta_{2}}\left(\textbf{x}\,|\,\textbf{z}_{s},\textbf{z}_{l}\right)$ and
$p_{\theta_{1}}\left(y\,|\,\textbf{z}_{l}\right)$, respectively, the negative
log-likelihood of x becomes the mean squared error (MSE) between the
reconstructed images $\tilde{\textbf{x}}$ and the ground-truth images x while
the negative log-likelihood of $y$ becomes the cross-entropy between the
actual labels $y$ and the predicted labels $\tilde{y}$. After working
(LABEL:eq:loss) out, we arrive at our overall objective function
$\mathcal{L}=\mathcal{L}_{R}+\mathcal{L}_{C}$, where:

$\begin{split}&\mathcal{L}_{R}=\alpha_{1}\lVert\textbf{x}-\tilde{\textbf{x}}\rVert^{2}-KL(\mu_{s},\sigma_{s}),\\\
&\mathcal{L}_{C}=-\alpha_{2}\sum_{n=1}^{N}y_{n}\ln
p_{\theta_{1}}(\tilde{y}=n\,|\,\textbf{z}_{l})-KL(\mu_{l},\sigma_{l}),\end{split}$
(3)

where
$KL(\mu,\sigma)=\frac{1}{2}\sum_{d=1}^{D}\big{(}1+2\ln(\sigma^{d})-(\mu^{d})^{2}-(\sigma^{d})^{2}\big{)}$,
$D$ denotes the dimension of the latent space, $N$ is the total number of
classes in an ($N$-way, $K$-shot) task, $\alpha_{1},\alpha_{2}$ are constant
scaling factors, $\mu_{s}$ and $\sigma^{2}_{s}$ denote the mean and variance
vectors of semantic latent distribution, and $\mu_{l}$ and $\sigma^{2}_{l}$
denote the mean and variance vectors of label latent distribution. The hyper-
parameters $\alpha_{1},\alpha_{2}$ only scale the evidence lower-bound
appropriately, since the reconstruction loss is in practice three orders of
magnitude greater than the cross-entropy loss. As such, this scaling helps
convergence but impacts the tightness of the ELBO slightly; nonetheless,
(LABEL:eq:loss) and (3) are still considered variational inference by
consensus among the literature (Higgins et al. 2017; Joy et al. 2021; Mathieu
et al. 2019; Dupont 2018). The loss is calculated for each given task on query
and support sets separately; i.e.,
$\mathcal{L}^{g}=\mathcal{L}_{R}^{g}+\mathcal{L}_{C}^{g}$ with
$g\in\\{\mathcal{S}_{i},\mathcal{Q}_{i}\\}$. Note that in (1) we deliberately
choose to exclude the label information $y$ as input to $q_{\phi_{1}}(.)$ to
be able to exploit the associated generative network
$p_{\theta_{1}}(y\,|\,\textbf{z}_{l})$ as a classifier. The consequence and
the proposed solution to accommodate this design choice are discussed in the
next subsection.

##  AttFEX for Transductive Feature Extraction

We first extract the feature maps of all images in the task using a
convolutional block $\textbf{F}=\tt{ConvEnc}(\textbf{X})$ where
$\textbf{X}\in\mathbb{R}^{N(K+Q)\times C\times W\times H}$,
$\textbf{F}\in\mathbb{R}^{N(K+Q)\times C^{\prime}\times W^{\prime}\times
H^{\prime}}$. The feature map tensor F is then transposed into
$\textbf{F}^{\prime}\in\mathbb{R}^{C^{\prime}\times N(K+Q)\times
W^{\prime}\times H^{\prime}}$ and fed into two consecutive $1\times 1$
convolution blocks. This helps the network utilize information across
corresponding pixels of all images in a task $\mathcal{T}_{i}$ which acts as a
parametric comparison of classes. We leverage the fact that ConvEnc already
extracts local pixel information by using larger kernels, and thus, use
parameter-light $1\times 1$ convolutions subsequently to focus only on
individual pixels. Let $\textbf{F}^{\prime}_{i}$ denote the $i^{th}$ channel
(or feature map layer) out of total of $C^{\prime}$ available and $\tt{ReLU}$
denote the rectified linear unit activation. The $1\times 1$ convolution block
($\texttt{Conv}_{1\times 1}$) is formulated as follows:

$\begin{split}&\textbf{M}_{i}=\texttt{ReLU}\big{(}\texttt{Conv}_{1\times
1}(\textbf{F}^{\prime}_{i},\textbf{W}_{M})\big{)},\forall
i\in[1,C^{\prime}];\,\\\
&\textbf{N}_{j}=\texttt{ReLU}\big{(}\texttt{Conv}_{1\times
1}(\textbf{M}_{j},\textbf{W}_{N})\big{)},\forall
j\in[1,C^{\prime}];\,\end{split}$ (4)

where $\textbf{N}\in\mathbb{R}^{C^{\prime}\times 32\times W^{\prime}\times
H^{\prime}}$ and $\textbf{W}_{M}\in\mathbb{R}^{64\times N(K+Q)\times 1\times
1}$, $\textbf{W}_{N}\in\mathbb{R}^{32\times 64\times 1\times 1}$ denote the
learnable weights. Next, we want to blend information across feature maps for
which we use a self-attention mechanism (Vaswani et al. 2017) across
$\textbf{N}_{j},\forall j\in[1,32]$. To do so, we feed N to query, key and
value extraction networks $f_{q}(,;\textbf{W}_{Q})$,
$f_{k}(.;\textbf{W}_{K})$, $f_{v}(.;\textbf{W}_{V})$ which are also designed
to be $1\times 1$ convolutions as:

$\begin{split}\textbf{Q}_{i}=\texttt{ReLU}\left(\texttt{Conv}_{1\times
1}(\textbf{N}_{i},\textbf{W}_{Q})\right),\quad\forall i\in[1,C^{\prime}];\,\\\
\textbf{K}_{i}=\texttt{ReLU}\left(\texttt{Conv}_{1\times
1}(\textbf{N}_{i},\textbf{W}_{K})\right),\quad\forall i\in[1,C^{\prime}];\,\\\
\textbf{V}_{i}=\texttt{ReLU}\left(\texttt{Conv}_{1\times
1}(\textbf{N}_{i},\textbf{W}_{V})\right),\quad\forall
i\in[1,C^{\prime}];\,\end{split}$ (5)

Figure 3: AttFEX module depicting colors as images and shades as feature maps.
We illustrate only 3 image feature maps and 3 channels instead of 32 for N,
for the sake of simplicity.

where $\textbf{W}_{Q},\textbf{W}_{K},\textbf{W}_{V}\in\mathbb{R}^{1\times
32\times 1\times 1}$ are the learnable weights and Q, K, V
$\in\mathbb{R}^{C^{\prime}\times 1\times W^{\prime}\times H^{\prime}}$ are the
query, key and value tensors. Next, each feature map $\textbf{N}_{j}$ is
mapped to its output tensor $\textbf{G}_{j}$ by computing a weighted sum of
the values, where each weight (within parentheses in (6)) measures the
compatibility (or similarity) between the query and its corresponding key
tensor using an inner-product:

$\displaystyle\textbf{G}_{i}=\sum_{j=1}^{C^{\prime}}\left({\frac{\exp{\left(\textbf{Q}_{i}\cdot\textbf{K}_{j}\right)}}{\sqrt{d_{k}}.\sum_{k=1}^{C^{\prime}}{\exp{\left(\textbf{Q}_{i}\cdot\textbf{K}_{k}\right)}}}}\right)\textbf{V}_{i},$
(6)

where $d_{k}=W^{\prime}\times H^{\prime}$, and
$\textbf{G}_{i}\in\mathbb{R}^{1\times C^{\prime}\times W^{\prime}\times
H^{\prime}}$, $\forall i$. Finally, we transform the original feature maps
$\mathbf{F}$ by applying a Hadamard product between the feature mask G and F,
thus, rendering the required feature maps transductive:

$\displaystyle\tilde{\textbf{F}}^{S}=\textbf{G}\circ\textbf{F}^{S}\quad
or\quad\tilde{\textbf{F}}^{Q}=\textbf{G}\circ\textbf{F}^{Q}.$

Here, $\textbf{F}^{S}$ and $\textbf{F}^{Q}$ represent the feature maps
corresponding to the support and query images, respectively. As a result of
operating on this channel-pixel distribution across images in a task,
$\textbf{F}^{S}$ and $\textbf{F}^{Q}$ are rendered task-aware. Note that the
query tensor Q must not be confused with the query set $\mathcal{Q}$ of a
task.

### Algorithmic Overview and Training Strategy

Figure 4: TRIDENT is comprised of two intertwined variational networks.
$\textbf{Z}_{s}^{g}$ is concatenated with the output of AttFEX, and used for
inferring $\textbf{Z}_{l}^{g}$, where $g\in\\{\mathcal{S},\mathcal{Q}\\}$.
Next, both $\textbf{Z}_{l}^{g}$ and $\textbf{Z}_{s}^{g}$ are used to
reconstruct images $\tilde{\textbf{X}}^{g}$ while $\textbf{Z}_{l}^{g}$ is used
to extract $\tilde{Y}^{g}$.

Overview of TRIDENT. The complete architecture of TRIDENT is illustrated in
Fig. 4. The ConvEnc feature extractor and the linear layers
$\mu_{\phi_{2}}(.)$, $\sigma^{2}_{\phi_{2}}(.)$ constitute the inference
network $q_{\phi_{2}}$ of the semantic latent variable (bottom row of Fig. 4).
The AttFEX module, another ConvEnc, and linear layers $\mu_{\phi_{1}}(.)$ and
$\sigma^{2}_{\phi_{1}}(.)$ make up the inference network $q_{\phi_{1}}$ of the
label latent variable (top row of Fig. 4). The proposed approach, TRIDENT, is
described in Algorithm 1. Note that TRIDENT is trained in a MAML (Finn,
Abbeel, and Levine 2017) fashion, where depending on the inner or outer loop,
the support or query set ($g\in\\{\mathcal{S},\mathcal{Q}\\}$) will be the
reference, respectively. First, the lower ConvEnc block extracts feature maps
$\textbf{X}^{g}_{\texttt{CE}}=\texttt{ConvEnc}(\textbf{X}^{g})$.
$\textbf{X}^{g}_{\texttt{CE}}$’s are then flattened and passed onto
$\mu_{\phi_{2}}(.)$, $\sigma^{2}_{\phi_{2}}(.)$, which respectively output the
mean and variance vectors of the _semantic_ latent distribution, as discussed
in (1). This is done either for the entire support or the query images
$\textbf{X}^{g}$, where $g\in\\{S,Q\\}$ for a given task $\mathcal{T}_{i}$. We
then sample a set of vectors $\textbf{Z}_{s}^{g}$ (subscript $s$ for
_semantic_) from their corresponding Gaussian distributions using the re-
parameterization trick (line $1$, Algorithm 1). Upon passing
$\textbf{X}=\textbf{X}^{\mathcal{S}}\cup\textbf{X}^{\mathcal{Q}}$ through the
upper ConvEnc, the AttFEX module of $q_{\phi_{1}}$ comes into play to create
_task-cognizant_ feature maps $\tilde{\textbf{F}}^{g}$ for either
$\mathcal{S}$ or $\mathcal{Q}$ (line $2$). $\textbf{Z}_{s}^{g}$ together with
$\tilde{\textbf{F}}^{g}$ are passed onto the linear layers
$\mu_{\phi_{1}}(.)$, $\sigma^{2}_{\phi_{1}}(.)$ to generate the mean and
variance vectors of the _label_ latent Gaussian distributions (line $3$).
After sampling the set of vectors $\textbf{Z}_{l}^{g}$ (subscript $l$ for
_label_) from their corresponding distributions, we use $\textbf{Z}_{l}^{g}$
and $\textbf{Z}_{s}^{g}$ to reconstruct the input images
$\tilde{\textbf{X}}^{g}$ using the generative network $p_{\theta_{2}}$ (line
4). Next, $\textbf{Z}_{l}^{g}$’s are input to the classifier network
$p_{\theta_{1}}$ to generate the class logits, which are normalized using a
softmax(.), resulting in class-conditional probabilities
$p(\tilde{Y}^{g}\,|\,\textbf{Z}_{l}^{g})$ (line $5$). Finally (in line $6$),
using the outputs of all the components discussed earlier, we calculate the
loss $\mathcal{L}^{g}$ as formulated in (LABEL:eq:loss) and (3).

Require: $\textbf{X}^{\mathcal{S}},\textbf{X}^{\mathcal{Q}},Y^{g},$
$\textbf{X}^{g}_{\texttt{CE}},$ where $g\in\\{\mathcal{S},\mathcal{Q}\\}$

1

2Sample:

$\textbf{Z}_{s}^{g}\sim
q_{\phi_{2}}\big{(}\textbf{Z}_{s}\,|\,\boldsymbol{\mu}_{\phi_{2}}(\textbf{X}^{g}_{\texttt{CE}}),\,diag\big{(}\boldsymbol{\sigma}^{2}_{\phi_{2}}(\textbf{X}^{g}_{\texttt{CE}})\big{)}\big{)}$

3 Compute task-cognizant embeddings:

$[\tilde{\textbf{F}}^{\mathcal{S}},\tilde{\textbf{F}}^{\mathcal{Q}}]=\texttt{AttFEX}(\texttt{ConvEnc}(\textbf{X}));\textbf{X}=\textbf{X}^{\mathcal{S}}\cup\textbf{X}^{\mathcal{Q}}$

4 Concatenate $\textbf{Z}_{s}^{g}$ and $\tilde{\textbf{F}^{g}}$ into
$[\tilde{\textbf{F}^{g}},\textbf{Z}_{s}^{g}]$ and sample:

$\textbf{Z}_{l}^{g}\sim
q_{\phi_{1}}\big{(}\textbf{Z}_{l}\,|\,\boldsymbol{\mu}_{\phi_{1}}([\tilde{\textbf{F}}^{g},\textbf{Z}_{s}^{g}]),\,diag(\boldsymbol{\sigma}^{2}_{\phi_{1}}([\tilde{\textbf{F}}^{g},\textbf{Z}_{s}^{g}]))\big{)}$

5 Reconstruct

$\textbf{X}^{g}$ using
$\tilde{\textbf{X}}^{g}=p_{\theta_{2}}(\textbf{X}\,|\,\textbf{Z}_{l}^{g},\textbf{Z}_{s}^{g})$

6 Extract class-conditional probabilities using:

$p\big{(}\tilde{Y}^{g}\,|\,\textbf{Z}_{l}^{g}\big{)}=\texttt{softmax}\big{(}p_{\theta_{1}}(Y^{g}\,|\,\textbf{Z}_{l}^{g})\big{)}$

7 Compute

$\mathcal{L}^{g}=\mathcal{L}^{g}_{R}+\mathcal{L}^{g}_{C}$

using (3)

Return: $\mathcal{L}^{g}$

Algorithm 1

TRIDENT

Require: $\mathcal{D}^{tr}$, $\alpha$, $\beta$, $B$

1 Randomly initialise $\Psi=(\phi_{1},\phi_{2},\theta_{1},\theta_{2})$

2

3while  _not converged_ do

4 Sample $B$ tasks $\mathcal{T}_{i}=\mathcal{S}_{i}\cup\mathcal{Q}_{i}$ from
$\mathcal{D}^{tr}$

5 for  _each task $\mathcal{T}_{i}$_ do

6 for  _number of adaptation steps_ do

7 Compute

$\mathcal{L}^{\mathcal{S}_{i}}(\Psi)=\texttt{TRIDENT}(\mathcal{T}_{i}-\\{Y^{\mathcal{Q}_{i}}\\})$

8 Evaluate

$\nabla_{(\Psi)}\mathcal{L}^{\mathcal{S}_{i}}(\Psi)$

9

$\Psi\leftarrow\Psi-\alpha\nabla_{\Psi}\mathcal{L}^{\mathcal{S}_{i}}(\Psi)$

10

11 end for

12

$(\Psi^{\prime})_{i}=\Psi$

13 end for

14 Compute

$\mathcal{L}^{\mathcal{Q}_{i}}(\Psi^{\prime}_{i})=\texttt{TRIDENT}(\mathcal{T}_{i}-\\{Y^{\mathcal{S}_{i}}\\});\forall
i\in[1,B]$

15 Meta-update on $\mathcal{Q}_{i}$:

$\Psi\leftarrow\Psi-\beta\nabla_{\Psi}\sum_{i=1}^{B}\mathcal{L}^{\mathcal{Q}_{i}}(\Psi^{\prime}_{i})$

16

17 end while

Algorithm 3

End to End Meta-Training of TRIDENT

Table 1: Accuracies in (% $\pm$ std). The predominant methodology of the
baselines: Ind.: inductive inference, TF: transductive feature extraction
methods, TI: transductive inference methods. Conv: convolutional blocks, RN:
ResNet backbone, $\dagger$: extra data. Style: best and second best. TRIDENT
employs a transductive feature extraction module (TF), and the simplest of
backbones (Conv4).

|  |  | miniImagenet | tieredImagenet | mini$\rightarrow$CUB
---|---|---|---|---|---
Methods | Backbone | Approach | 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot
MAML (Finn, Abbeel, and Levine 2017) | Conv4 | Ind. | 48.70 $\pm$ 1.84 | 63.11 $\pm$ 0.92 | 51.67 $\pm$ 1.81 | 70.30 $\pm$ 0.08 | 34.01 $\pm$ 1.25 | 48.83 $\pm$ 0.62
ABML (Ravi and Beatson 2019) | Conv4 | Ind. | 40.88 $\pm$ 0.25 | 58.19 $\pm$ 0.17 | - | - | 31.51 $\pm$ 0.32 | 47.80 $\pm$ 0.51
OVE(PL) (Patacchiola et al. 2020) | Conv4 | Ind. | 48.00 $\pm$ 0.24 | 67.14 $\pm$ 0.23 | - | - | 37.49 $\pm$ 0.11 | 57.23 $\pm$ 0.31
DKT+Cos (Patacchiola et al. 2020) | Conv4 | Ind. | 48.64 $\pm$ 0.45 | 62.85 $\pm$ 0.37 | - | - | 40.22 $\pm$ 0.54 | 55.65 $\pm$ 0.05
BOIL (Oh et al. 2021) | Conv4 | Ind. | 49.61 $\pm$ 0.16 | 48.58 $\pm$ 0.27 | 66.45 $\pm$ 0.37 | 69.37 $\pm$ 0.12 | - | -
LFWT(Tseng et al. 2020) | RN10 | TF+TI | 66.32 $\pm$ 0.80 | 81.98 $\pm$ 0.55 | - | - | 47.47 $\pm$ 0.75 | 66.98 $\pm$ 0.68
FRN(Wertheimer, Tang, and Hariharan 2021) | RN12 | Ind. | 66.45 $\pm$ 0.19 | 82.83 $\pm$ 0.13 | 71.16 $\pm$ 0.22 | 86.01 $\pm$ 0.15 | 54.11 $\pm$ 0.19 | 77.09 $\pm$ 0.15
DPGN(Yang et al. 2020) | RN12 | TF+TI | 67.77 | 84.6 | 72.45 | 87.24 | - | -
PAL(Ma et al. 2021) | RN12 | TF+TI | 69.37 $\pm$ 0.64 | 84.40 $\pm$ 0.44 | 72.25 $\pm$ 0.72 | 86.95 $\pm$ 0.47 | - | -
Proto-Completion(Zhang et al. 2021a) | RN12 | TF+TI | 73.13 $\pm$ 0.85 | 82.06 $\pm$ 0.54 | 81.04 $\pm$ 0.89 | 87.42 $\pm$ 0.57 | - | -
TPMN(Wu et al. 2021) | RN12 | TF+TI | 67.64 $\pm$ 0.63 | 83.44 $\pm$ 0.43 | 72.24 $\pm$ 0.70 | 86.55 $\pm$ 0.63 | - | -
LIF-EMD(Li, Wang, and Hu 2021) | RN12 | TF+TI | 68.94 $\pm$ 0.28 | 85.07 $\pm$ 0.50 | 73.76 $\pm$ 0.32 | 87.83 $\pm$ 0.59 | - | -
Transd-CNAPS(Bateni et al. 2022) | RN18 | TF+TI | 55.6 $\pm$ 0.9 | 73.1 $\pm$ 0.7 | 65.9 $\pm$ 1.0 | 81.8 $\pm$ 0.7 | - | -
Baseline++(Chen et al. 2019) | RN18 | TF | 51.87 $\pm$ 0.77 | 75.68 $\pm$ 0.63 | - | - | 42.85 $\pm$ 0.69 | 62.04 $\pm$ 0.76
FEAT(Ye et al. 2020) | RN18 | TF | 66.78 | 82.05 | 70.80 | 84.79 | 50.67 $\pm$ 0.78 | 71.08 $\pm$ 0.73
SimpleShot(Wang et al. 2019) | WRN | Ind. | 63.32 | 80.28 | 69.98 | 85.45 | 48.56 | 65.63
Assoc-Align(Afrasiyabi, Lalonde, and Gagné 2020) | WRN | TF | 65.92 $\pm$ 0.60 | 82.85 $\pm$ 0.55 | 74.40 $\pm$ 0.68 | 86.61 $\pm$ 0.59 | 47.25 $\pm$ 0.76 | 72.37 $\pm$ 0.89
ReRank(SHEN et al. 2021) | WRN | TF+TI | 72.4$\pm$0.6 | 80.2$\pm$0.4 | 79.5$\pm$0.6 | 84.8$\pm$0.4 | - | -
TIM-GD(Boudiaf et al. 2020) | WRN | TI | 77.8 | 87.4 | 82.1 | 89.8 | - | 71
LaplacianShot(Ziko et al. 2020) | WRN | TI | 74.9 | 84.07 | 80.22 | 87.49 | 55.46 | 66.33
S2M2(Mangla et al. 2020) | WRN | TF | 64.93 $\pm$ 0.18 | 83.18 $\pm$ 0.11 | 73.71 $\pm$ 0.22 | 88.59 $\pm$ 0.14 | 48.24 $\pm$ 0.84 | 70.44 $\pm$ 0.75
MetaQDA(Zhang et al. 2021b) | WRN | TF | 67.83 $\pm$ 0.64 | 84.28 $\pm$ 0.69 | 74.33 $\pm$ 0.65 | 89.56 $\pm$ 0.79 | 53.75 $\pm$ 0.72 | 71.84 $\pm$ 0.66
PT+MAP(Hu, Gripon, and Pateux 2021) | WRN | TF+TI | 82.92 $\pm$ 0.26 | 88.82 $\pm$ 0.13 | 85.67 $\pm$ 0.26 | 90.45 $\pm$ 0.14 | 62.49 $\pm$ 0.32 | 76.51 $\pm$ 0.18
PE$M_{n}$E-BMS(Hu, Pateux, and Gripon 2022) | WRN | TF+TI | 83.35 $\pm$ 0.25 | 89.53 $\pm$ 0.13 | 86.07 $\pm$ 0.25 | 91.09 $\pm$ 0.14 | 63.90 $\pm$ 0.31 | 79.15 $\pm$ 0.18
Transd-CNAPS+FETI(Bateni et al. 2022) | $\texttt{RN18}^{\dagger}$ | TF+TI | 79.9 $\pm$ 0.8 | 91.50 $\pm$ 0.4 | 73.8 $\pm$ 0.1 | 87.7 $\pm$ 0.6 | - | -
TRIDENT(Ours) | Conv4 | TF | 86.11 $\pm$ 0.59 | 95.95 $\pm$ 0.28 | 86.97 $\pm$ 0.50 | 96.57 $\pm$ 0.17 | 84.61 $\pm$ 0.33 | 80.74 $\pm$ 0.35

Training strategy. An important aspect of the training procedure of TRIDENT is
that its set of parameters $\Psi=(\theta_{1},\theta_{2},\phi_{1},\phi_{2})$
are meta-learnt by back-propagating through the adaptation procedure on the
support set, as proposed in MAML (Finn, Abbeel, and Levine 2017) and
illustrated here in Algorithm 3. This increases the sensitivity of the
parameters $\Psi$ towards the loss function for fast adaptation to unseen
tasks and reduces generalization errors on the query set $\mathcal{Q}$. First,
we randomly initialize the parameters $\Psi$ (line 1, Algorithm 3) to compute
the objective function over the support set
$\mathcal{L}^{\mathcal{S}_{i}}(\Psi)$ using equation (3) in the main
manuscript, and perform a number of gradient descent steps on the parameters
$\Psi$ to adapt them to the support set (lines $5$ to $9$). This is called the
_inner-update_ and is done separately for all the support sets corresponding
to their $B$ different tasks (line $3$). Once the inner-update is computed for
each of the $B$ parameter sets, the loss is evaluated on the query set
$\mathcal{L}^{\mathcal{Q}_{i}}(\Psi^{\prime}_{i})$ (line $12$), following
which a _meta-update_ is conducted over all the corresponding query sets,
which involves computing a gradient through a gradient procedure as described
in (Finn, Abbeel, and Levine 2017) (line $13$).

## Experimental Evaluation

The goal of this section is to address the following four questions: (i) How
well does TRIDENT perform when compared against the state-of-the-art methods
for few-shot classification? (ii) How reliable is TRIDENT in terms of the
confidence and uncertainty metrics? (iii) How well does TRIDENT perform in a
cross-domain setting where there is a domain shift between the training and
testing datasets? (iv) Does TRIDENT actually decouple latent variables?

Benchmark Datasets. We evaluate TRIDENT on the three most commonly adopted
datasets: miniImagenet (Ravi and Larochelle 2017), tieredImagenet (Ren et al.
2018) and CUB (Welinder et al. 2010). miniImagenet and tieredImagenet are
subsets of ImageNet (Deng et al. 2009) utilized for few-shot classification.
Further details on these datasets can be found in the Appendix.

Implementational Details. We use PyTorch (Paszke et al. 2019) and learn2learn
(Arnold et al. 2020) for all our implementations. We use a commonly adopted
Conv4 architecture (Ravi and Larochelle 2017; Finn, Abbeel, and Levine 2017;
Patacchiola et al. 2020; Afrasiyabi, Lalonde, and Gagné 2020; Wang et al.
2019; Boudiaf et al. 2020) as ConvEnc to obtain the generic feature maps.
Following the standard setting in the literature (Finn, Abbeel, and Levine
2017; Ravi and Larochelle 2017), the Conv4 has four convolutional blocks where
each block has a $3\times 3$ convolution layer with 32 feature maps, followed
by a batch normalization (BN) (Ioffe and Szegedy 2015) layer, a $2\times 2$
max-pooling layer and a LeakyReLU(0.2) activation. The generative network
$p_{\theta_{1}}$ for $\textbf{z}_{l}$ is a classifier with two linear layers
and a LeakyReLU(0.2) activation in between, while $p_{\theta_{2}}$ for
$\textbf{z}_{s}$ consists of four blocks of a $2$-D upsampling layer, followed
by a $3\times 3$ convolution and LeakyReLU(0.2) activation. Both latent
variables $\textbf{z}_{l}$ and $\textbf{z}_{s}$ have a dimensionality of $64$.
Following (Nichol, Achiam, and Schulman 2018b; Liu et al. 2019; Vaswani et al.
2017), images are resized to $84\times 84$ for all configurations and we train
and report test accuracy of ($5$-way, $1$ and $5$-shot) settings with $10$
query images per class for all datasets. Hyper-parameter settings can be found
in the Appendix.

### Evaluation Results

We report test accuracies indicating $95$% confidence intervals over $600$
tasks for miniImagenet, and $2000$ tasks for both tieredImagenet and CUB, as
is customary across the literature (Chen et al. 2019; Dhillon et al. 2020;
Bateni et al. 2022). We compare our performance against a wide variety of
state-of-the-art few-shot classification methods such as: (i) metric-learning
(Wang et al. 2019; Bateni et al. 2020; Afrasiyabi, Lalonde, and Gagné 2020;
Yang et al. 2020), (ii) transductive feature-extraction based (Oreshkin,
Rodríguez López, and Lacoste 2018; Ye et al. 2020; Li et al. 2019; Xu et al.
2021), (iii) optimization-based (Finn, Abbeel, and Levine 2017; Mishra et al.
2018; Oh et al. 2021; Lee et al. 2019; Rusu et al. 2019), (iv) transductive
inference-based (Bateni et al. 2022; Boudiaf et al. 2020; Ziko et al. 2020;
Liu et al. 2019), and (v) Bayesian (Iakovleva, Verbeek, and Alahari 2020;
Zhang et al. 2019; Hu et al. 2020; Patacchiola et al. 2020; Ravi and Beatson
2019) approaches. Previous works (Liu et al. 2019), (Hou et al. 2019) have
demonstrated the superiority of transductive inference methods over their
inductive counterparts. In this light, we compare against a larger number of
transductive ($18$ baselines) rather than inductive ($7$ baselines) methods
for a fair comparison.

Table 2: Parameter count of TRIDENT against competitors.

| Conv4 | $\mu_{\phi}$ | $\sigma_{\phi}$ | AttFEX | TRIDENT | Conv4 | RN18 | WRN
---|---|---|---|---|---|---|---|---
$q_{\phi_{1}}$ | 28896 | 51264 | 51264 | 6994 |  |  |  |
$q_{\phi_{2}}$ | 28896 | 51264 | 51264 | - |  |  |  |
$p_{\theta_{1}}$\+ $p_{\theta_{2}}$ | 2245 + 132009 | 412,238 | $190,410$ | $12.4$M | $36.482$M

It is important to note that TRIDENT is only a _transductive feature-
extraction_ based method as we utilize the query set images to extract task-
aware feature embeddings; it is not a transductive inference based method
since we perform inference of class-labels over the entire domain of
definition and not just for the selected query samples (Vapnik 2006;
Gammerman, Vovk, and Vapnik 1998). The results on miniImagenet and
tieredImagenet for both ($5$-way, $1$ and $5$-shot) settings are summarized in
Table 1. We accentuate on the fact that we also compare against Transd-
CNAPS+FETI (Bateni et al. 2022), where the authors pre-train the ResNet-18
backbone on the entire train split of Imagenet. We, however, avoid training on
additional datasets, in favor of fair comparison with the rest of literature.
Regardless of the choice of backbone (simplest in our case), TRIDENT sets a
new state-of-the-art on miniImagenet and tieredImagenet for both ($5$-way, $1$
and $5$-shot) settings, offering up to $5\%$ gain over the prior art.
Recently, a more challenging _cross-domain_ setting has been proposed for few-
shot classification to assess its generalization capabilities to unseen
datasets. The commonly adopted setting is where one trains on miniImagenet and
tests on CUB (Chen et al. 2019). The results of this experiment are also
presented in Table 1. We compare against _any existing baselines_ for which
this cross-domain experiment has been conducted. As can be seen, and to the
best of our knowledge, TRIDENT again sets a new state-of-the-art by a
significant margin of $20\%$ for ($5$-way, $1$-shot) setting, and $1.5\%$ for
($5$-way, $5$-shot) setting.

Computational Complexity. Most of the reported baselines in Table 1 use
stronger backbones such as ResNet12, ResNet18 and WRN which contain $11.5$,
$12.4$ and $36.4$ millions of parameters respectively. On the other hand, we
use three Conv4s along with two fully connected layers and an AttFEX module
which accounts for 410,958 and 412,238 parameters in the ($5$-way, $1$-shot)
and ($5$-way, $5$-shot) scenarios, respectively. This is summarized in details
in Table 2. Even though we are more parameter heavy than approaches that use a
single Conv4 as feature extractor, TRIDENT’s total parameters still lies in
the same order of magnitude as these approaches. In summary, when it comes to
complexity in parameter space, we are considerably more efficient than the
vast majority of the cited competitors.

Reliability Metrics. A complementary set of metrics are typically used in
probabilistic settings to measure the uncertainty and reliability of
predictions. More specifically, expected calibration error (ECE) and maximum
calibration error (MCE) respectively measure the expected and maximum binned
difference between confidence and accuracy (Guo et al. 2017). This is
illustrated in Table 3 where TRIDENT offers superior calibration on
miniImagenet ($5$-way, $1$ and $5$-shot) as compared to other probabilistic
approaches, and MAML (Finn, Abbeel, and Levine 2017).

### Ablation Study

We analyze the classification performance of TRIDENT across various paramaters
and hyper-parameters, as is summarized in Table 4. We use miniImagenet
($5$-way, $1$-shot) setting to carry out ablation study experiments. To cover
different design perspectives, we carry out ablation on: (i) MAML-style
training parameters: meta-batch size $B$ and number of inner adaption steps
$n$, (ii) latent space dimensionality: $\textbf{z}_{l}$ and $\textbf{z}_{s}$
to assess the impact of their size, (iii) AttFEX features: number of features
extracted by $\textbf{W}_{M}$, $\textbf{W}_{N}$. Looking at the results,
TRIDENT’s performance is directly proportional to the number of tasks and
inner-adaptation steps, as is previously demonstrated in (Antreas Antoniou,
Harrison Edwards, and Amos J. Storkey 2019; Finn, Abbeel, and Levine 2017) for
MAML based training. Regarding latent space dimensions, a correlation between
a higher dimension of $\textbf{z}_{l}$ and $\textbf{z}_{s}$ and a better
performance can be observed. Even though, the results show that increasing
both dimensions beyond $64$ leads to performance degradation. As such,
$(64,64)$ seems to be the sweet spot. Finally, on feature space dimensions of
AttFEX, the performance improves when $\textbf{W}_{M}$ $>$ $\textbf{W}_{N}$,
and the best performance is achieved when the parameters are set to ($64$,
$32$). Notably, the exact set of parameters return the best performance for
($5$-way, $5$-shot) setting.

Table 3: Calibration errors of TRIDENT. Style: best and second best.

| Metrics | MAML | PLATIPUS | ABPML | ABML | BMAML | VAMPIRE | TRIDENT
---|---|---|---|---|---|---|---|---
| ECE | 0.046 | 0.032 | 0.013 | 0.026 | 0.025 | 0.008 | 0.0036
5-way, 1-shot | MCE | 0.073 | 0.108 | 0.037 | 0.058 | 0.092 | 0.038 | 0.029
| ECE | 0.032 | - | 0.006 | - | 0.027 | - | 0.0015
5-way, 5-shot | MCE | 0.044 | - | 0.030 | - | 0.049 | - | 0.018

Table 4: Ablation study for miniImagenet ($5$-way, $1$-shot) tasks. Accuracies
in (% $\pm$ std.).

(B, n) | (5, 3) | (5, 5) | (10, 3) | (10, 5) | (20, 3) | (20, 5)
---|---|---|---|---|---|---
- | 67.43 $\pm$ 0.75 | 69.21 $\pm$ 0.66 | 74.6 $\pm$ 0.84 | 80.82 $\pm$ 0.68 | 86.11 $\pm$ 0.59
($dim(\textbf{z}_{l})$, $dim(\textbf{z}_{s}))$ | (32, 32) | (32, 64) | (32, 128) | (64, 32) | (64, 64) | (64, 128) | (128, 32) | (128, 64) | (128, 128)
76.29 $\pm$ 0.72 | 75.44 $\pm$ 0.81 | 79.1 $\pm$ 0.57 | 82.93 $\pm$ 0.8 | 86.11 $\pm$ 0.59 | 85.62 $\pm$ 0.52 | 81.49 $\pm$ 0.65 | 82.89 $\pm$ 0.48 | 84.42 $\pm$ 0.59
($dim(\textbf{W}_{M})$, $dim(\textbf{W}_{N}))$ | (32, 32) | (32, 64) | (32, 128) | (64, 32) | (64, 64) | (64, 128) | (128, 32) | (128, 64) | (128, 128)
78.4 $\pm$ 0.23 | 77.89 $\pm$ 0.39 | 79.55 $\pm$ 0.87 | 86.11 $\pm$ 0.59 | 84.87 $\pm$ 0.45 | 82.11 $\pm$ 0.35 | 84.67 $\pm$ 0.7 | 85.8 $\pm$ 0.58 | 83.92 $\pm$ 0.63

### Decoupling Analysis

As a qualitative demonstration, we visualize the _label_ and _semantic_ latent
means ($\mu_{l}$ and $\mu_{s}$) of query images for a randomly selected
($5$-way, $5$-shot) task from miniImagenet, before and after the MAML meta-
update procedure. The UMAP (McInnes, Healy, and Melville 2018) plots in Fig. 5
illustrate significant improvement in class-conditional separation of query
samples for _label_ latent space upon meta-adaption, whereas negligible
improvement is visible on the semantic latent space. This is a qualitative
evidence that $\textbf{Z}_{L}$ captures more class-discriminating information
as compared to $\textbf{Z}_{S}$. To substantiate this quantitatively, the
clustering capacity of these latent spaces is also measured by the Davies-
Bouldin score (DBI) (Davies and Bouldin 1979), where, the lower the DBI score,
the better both the inter-cluster separation and intra-cluster “tightness”.
Fig. 5 shows that the DBI score drops significantly more after meta-adaptation
in the case of $\textbf{Z}_{L}$ as compared to $\textbf{Z}_{S}$, indicating
better clustering of features in the former than the latter. This aligns with
the proposed decoupling strategy of TRIDENT and corroborates the validity of
our proposition to put an emphasis on label latent information for the
downstream few-shot task.

Figure 5: Better class separation upon meta-update is confirmed by lower DBI
scores. Different colors/markers indicate classes.

## Concluding Remarks

We introduce a novel variational inference network (coined as TRIDENT) that
simultaneously infers decoupled latent variables representing semantic and
label information of an image. The proposed network is comprised of two
intertwined variational sub-networks responsible for inferring the semantic
and label information separately, the latter being enhanced using an
attention-based transductive feature extraction module (AttFEX). Our extensive
experimental results corroborate the efficacy of this transductive decoupling
strategy on a variety of few-shot classification settings demonstrating
superior performance and setting a new state-of-the-art for the most commonly
adopted dataset mini and tieredImagenet as well as for the recent challenging
cross-domain scenario of miniImagenet $\rightarrow$ CUB. As future work, we
plan to demonstrate the applicability of TRIDENT in semi-supervised and
unsupervised settings by including the likelihood of unlabelled samples
derived from the graphical model. This would render TRIDENT as an all-
inclusive holistic approach towards solving few-shot classification.

> ## References
>
>   * Afrasiyabi, Lalonde, and Gagné (2020) Afrasiyabi, A.; Lalonde, J.-F.;
> and Gagné, C. 2020.  Associative alignment for few-shot image
> classification.  In _European Conference on Computer Vision_ , 18–35.
> Springer.
>   * Antreas Antoniou, Harrison Edwards, and Amos J. Storkey (2019) Antreas
> Antoniou; Harrison Edwards; and Amos J. Storkey. 2019.  How to train your
> MAML.  In _ICLR (Poster)_. OpenReview.net.
>   * Arnold et al. (2020) Arnold, S. M. R.; Mahajan, P.; Datta, D.; Bunner,
> I.; and Zarkias, K. S. 2020.  learn2learn: A Library for Meta-Learning
> Research.
>   * Bateni et al. (2022) Bateni, P.; Barber, J.; van de Meent, J.-W.; and
> Wood, F. 2022.  Enhancing Few-Shot Image Classification With Unlabelled
> Examples.  In _Proceedings of the IEEE/CVF Winter Conference on Applications
> of Computer Vision (WACV)_ , 2796–2805.
>   * Bateni et al. (2020) Bateni, P.; Goyal, R.; Masrani, V.; Wood, F.; and
> Sigal, L. 2020.  Improved Few-Shot Visual Classification.  In _Proceedings
> of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
> (CVPR)_.
>   * Boudiaf et al. (2020) Boudiaf, M.; Ziko, I.; Rony, J.; Dolz, J.;
> Piantanida, P.; and Ben Ayed, I. 2020\.  Information Maximization for Few-
> Shot Learning.  In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.;
> and Lin, H., eds., _Advances in Neural Information Processing Systems_ ,
> volume 33, 2445–2457. Curran Associates, Inc.
>   * BRIER (1950) BRIER, G. W. 1950.  VERIFICATION OF FORECASTS EXPRESSED IN
> TERMS OF PROBABILITY.  _Monthly Weather Review_ , 78(1): 1 – 3.
>   * Chen et al. (2019) Chen, W.-Y.; Liu, Y.-C.; Kira, Z.; Wang, Y.-C.; and
> Huang, J.-B. 2019.  A Closer Look at Few-shot Classification.  In
> _International Conference on Learning Representations_.
>   * Davies and Bouldin (1979) Davies, D. L.; and Bouldin, D. W. 1979.  A
> Cluster Separation Measure.  _IEEE Transactions on Pattern Analysis and
> Machine Intelligence_ , PAMI-1(2): 224–227.
>   * Deng et al. (2009) Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.;
> and Fei-Fei, L. 2009.  Imagenet: A large-scale hierarchical image database.
> In _2009 IEEE conference on computer vision and pattern recognition_ ,
> 248–255. Ieee.
>   * Dhillon et al. (2020) Dhillon, G. S.; Chaudhari, P.; Ravichandran, A.;
> and Soatto, S. 2020.  A Baseline for Few-Shot Image Classification.  In
> _International Conference on Learning Representations_.
>   * Dupont (2018) Dupont, E. 2018.  Learning Disentangled Joint Continuous
> and Discrete Representations.  In Bengio, S.; Wallach, H.; Larochelle, H.;
> Grauman, K.; Cesa-Bianchi, N.; and Garnett, R., eds., _Advances in Neural
> Information Processing Systems_ , volume 31. Curran Associates, Inc.
>   * Edwards and Storkey (2017) Edwards, H.; and Storkey, A. J. 2017.
> Towards a Neural Statistician.  _ArXiv_ , abs/1606.02185.
>   * Finn, Abbeel, and Levine (2017) Finn, C.; Abbeel, P.; and Levine, S.
> 2017.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks.
> In _Proceedings of the 34th International Conference on Machine Learning_ ,
> volume 70 of _Proceedings of Machine Learning Research_ , 1126–1135.
>   * Finn, Xu, and Levine (2018) Finn, C.; Xu, K.; and Levine, S. 2018.
> Probabilistic Model-Agnostic Meta-Learning.  In Bengio, S.; Wallach, H.;
> Larochelle, H.; Grauman, K.; Cesa-Bianchi, N.; and Garnett, R., eds.,
> _Advances in Neural Information Processing Systems_ , volume 31. Curran
> Associates, Inc.
>   * Galy-Fajou et al. (2020) Galy-Fajou, T.; Wenzel, F.; Donner, C.; and
> Opper, M. 2020.  Multi-Class Gaussian Process Classification Made Conjugate:
> Efficient Inference via Data Augmentation.  In Adams, R. P.; and Gogate, V.,
> eds., _Proceedings of The 35th Uncertainty in Artificial Intelligence
> Conference_ , volume 115 of _Proceedings of Machine Learning Research_ ,
> 755–765. PMLR.
>   * Gammerman, Vovk, and Vapnik (1998) Gammerman, A.; Vovk, V.; and Vapnik,
> V. 1998.  Learning by transduction, vol UAI’98.
>   * Gordon et al. (2019) Gordon, J.; Bronskill, J.; Bauer, M.; Nowozin, S.;
> and Turner, R. 2019.  Meta-Learning Probabilistic Inference for Prediction.
> In _International Conference on Learning Representations_.
>   * Guo et al. (2017) Guo, C.; Pleiss, G.; Sun, Y.; and Weinberger, K. Q.
> 2017.  On Calibration of Modern Neural Networks.  In _Proceedings of the
> 34th International Conference on Machine Learning_ , volume 70 of
> _Proceedings of Machine Learning Research_ , 1321–1330. PMLR.
>   * Higgins et al. (2017) Higgins, I.; Matthey, L.; Pal, A.; Burgess, C.;
> Glorot, X.; Botvinick, M.; Mohamed, S.; and Lerchner, A. 2017.  beta-VAE:
> Learning Basic Visual Concepts with a Constrained Variational Framework.  In
> _International Conference on Learning Representations_.
>   * Hou et al. (2019) Hou, R.; Chang, H.; MA, B.; Shan, S.; and Chen, X.
> 2019.  Cross Attention Network for Few-shot Classification.  In Wallach, H.;
> Larochelle, H.; Beygelzimer, A.; d'Alché-Buc, F.; Fox, E.; and Garnett, R.,
> eds., _Advances in Neural Information Processing Systems_ , volume 32.
> Curran Associates, Inc.
>   * Hu et al. (2020) Hu, S. X.; Moreno, P.; Xiao, Y.; Shen, X.; Obozinski,
> G.; Lawrence, N.; and Damianou, A. 2020.  Empirical Bayes Transductive Meta-
> Learning with Synthetic Gradients.  In _International Conference on Learning
> Representations (ICLR)_.
>   * Hu, Gripon, and Pateux (2021) Hu, Y.; Gripon, V.; and Pateux, S. 2021.
> Leveraging the feature distribution in transfer-based few-shot learning.  In
> _International Conference on Artificial Neural Networks_ , 487–499.
> Springer.
>   * Hu, Pateux, and Gripon (2022) Hu, Y.; Pateux, S.; and Gripon, V. 2022.
> Squeezing backbone feature distributions to the max for efficient few-shot
> learning.  _Algorithms_ , 15(5): 147.
>   * Iakovleva, Verbeek, and Alahari (2020) Iakovleva, E.; Verbeek, J.; and
> Alahari, K. 2020.  Meta-Learning with Shared Amortized Variational
> Inference.  In _Proceedings of the 37th International Conference on Machine
> Learning_ , Proceedings of Machine Learning Research. PMLR.
>   * Ioffe and Szegedy (2015) Ioffe, S.; and Szegedy, C. 2015.  Batch
> Normalization: Accelerating Deep Network Training by Reducing Internal
> Covariate Shift.  In _Proceedings of the 32nd International Conference on
> Machine Learning_ , volume 37 of _Proceedings of Machine Learning Research_
> , 448–456. PMLR.
>   * Joy et al. (2021) Joy, T.; Schmon, S.; Torr, P.; N, S.; and Rainforth,
> T. 2021.  Capturing Label Characteristics in {VAE}s.  In _International
> Conference on Learning Representations_.
>   * Kingma and Ba (2015) Kingma, D. P.; and Ba, J. 2015.  Adam: A Method for
> Stochastic Optimization.  In _ICLR (Poster)_.
>   * Kingma et al. (2014) Kingma, D. P.; Mohamed, S.; Jimenez Rezende, D.;
> and Welling, M. 2014.  Semi-supervised Learning with Deep Generative Models.
> In _Advances in Neural Information Processing Systems_ , volume 27.
>   * Kingma and Welling (2014) Kingma, D. P.; and Welling, M. 2014.  Auto-
> Encoding Variational Bayes.  arXiv:1312.6114.
>   * Lee et al. (2019) Lee, K.; Maji, S.; Ravichandran, A.; and Soatto, S.
> 2019.  Meta-Learning with Differentiable Convex Optimization.  In _CVPR_.
>   * Li et al. (2019) Li, H.; Eigen, D.; Dodge, S.; Zeiler, M.; and Wang, X.
> 2019.  Finding Task-Relevant Features for Few-Shot Learning by Category
> Traversal.  In _CVPR_.
>   * Li, Wang, and Hu (2021) Li, J.; Wang, Z.; and Hu, X. 2021.  Learning
> Intact Features by Erasing-Inpainting for Few-shot Classification.
> _Proceedings of the AAAI Conference on Artificial Intelligence_ , 35(9):
> 8401–8409.
>   * Liu, Song, and Qin (2020) Liu, J.; Song, L.; and Qin, Y. 2020.
> Prototype Rectification for Few-Shot Learning.  In _Computer Vision - ECCV
> 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020,
> Proceedings, Part I_.
>   * Liu et al. (2019) Liu, Y.; Lee, J.; Park, M.; Kim, S.; Yang, E.; Hwang,
> S.; and Yang, Y. 2019.  Learning to Propagate Labels: Transductive
> Propagation Network for Few-shot Learning.  In _International Conference on
> Learning Representations_.
>   * Long et al. (2018) Long, M.; CAO, Z.; Wang, J.; and Jordan, M. I. 2018.
> Conditional Adversarial Domain Adaptation.  In _Advances in Neural
> Information Processing Systems_ , volume 31. Curran Associates, Inc.
>   * Ma et al. (2021) Ma, J.; Xie, H.; Han, G.; Chang, S.-F.; Galstyan, A.;
> and Abd-Almageed, W. 2021\.  Partner-assisted learning for few-shot image
> classification.  In _Proceedings of the IEEE/CVF International Conference on
> Computer Vision_ , 10573–10582.
>   * Mangla et al. (2020) Mangla, P.; Kumari, N.; Sinha, A.; Singh, M.;
> Krishnamurthy, B.; and Balasubramanian, V. N. 2020.  Charting the right
> manifold: Manifold mixup for few-shot learning.  In _Proceedings of the
> IEEE/CVF winter conference on applications of computer vision_ , 2218–2227.
>   * Mathieu et al. (2019) Mathieu, E.; Rainforth, T.; Siddharth, N.; and
> Teh, Y. W. 2019.  Disentangling Disentanglement in Variational Autoencoders.
> In Chaudhuri, K.; and Salakhutdinov, R., eds., _Proceedings of the 36th
> International Conference on Machine Learning_ , volume 97 of _Proceedings of
> Machine Learning Research_ , 4402–4412. PMLR.
>   * McInnes, Healy, and Melville (2018) McInnes, L.; Healy, J.; and
> Melville, J. 2018.  Umap: Uniform manifold approximation and projection for
> dimension reduction.  _arXiv preprint arXiv:1802.03426_.
>   * Mishra et al. (2018) Mishra, N.; Rohaninejad, M.; Chen, X.; and Abbeel,
> P. 2018.  A Simple Neural Attentive Meta-Learner.  In _International
> Conference on Learning Representations_.
>   * Nguyen, Do, and Carneiro (2019) Nguyen, C. C.; Do, T.; and Carneiro, G.
> 2019.  Uncertainty in Model-Agnostic Meta-Learning using Variational
> Inference.  _CoRR_.
>   * Nichol, Achiam, and Schulman (2018a) Nichol, A.; Achiam, J.; and
> Schulman, J. 2018a.  On First-Order Meta-Learning Algorithms.
> arXiv:1803.02999.
>   * Nichol, Achiam, and Schulman (2018b) Nichol, A.; Achiam, J.; and
> Schulman, J. 2018b.  On First-Order Meta-Learning Algorithms.  _CoRR_ ,
> abs/1803.02999.
>   * Oh et al. (2021) Oh, J.; Yoo, H.; Kim, C.; and Yun, S.-Y. 2021.  BOIL:
> Towards Representation Change for Few-shot Learning.  In _International
> Conference on Learning Representations_.
>   * Oreshkin, Rodríguez López, and Lacoste (2018) Oreshkin, B.; Rodríguez
> López, P.; and Lacoste, A. 2018.  TADAM: Task dependent adaptive metric for
> improved few-shot learning.  In _Advances in Neural Information Processing
> Systems_ , volume 31.
>   * Paszke et al. (2019) Paszke, A.; Gross, S.; Massa, F.; Lerer, A.;
> Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.;
> Desmaison, A.; Kopf, A.; Yang, E.; DeVito, Z.; Raison, M.; Tejani, A.;
> Chilamkurthy, S.; Steiner, B.; Fang, L.; Bai, J.; and Chintala, S. 2019.
> PyTorch: An Imperative Style, High-Performance Deep Learning Library.  In
> _Advances in Neural Information Processing Systems 32_.
>   * Patacchiola et al. (2020) Patacchiola, M.; Turner, J.; Crowley, E. J.;
> O’Boyle, M. F. P.; and Storkey, A. J. 2020.  Bayesian Meta-Learning for the
> Few-Shot Setting via Deep Kernels.  In _NeurIPS_.
>   * Rajeswaran et al. (2019) Rajeswaran, A.; Finn, C.; Kakade, S. M.; and
> Levine, S. 2019.  Meta-Learning with Implicit Gradients.  In Wallach, H.;
> Larochelle, H.; Beygelzimer, A.; d'Alché-Buc, F.; Fox, E.; and Garnett, R.,
> eds., _Advances in Neural Information Processing Systems_ , volume 32.
> Curran Associates, Inc.
>   * Ravi and Beatson (2019) Ravi, S.; and Beatson, A. 2019.  Amortized
> Bayesian Meta-Learning.  In _International Conference on Learning
> Representations_.
>   * Ravi and Larochelle (2017) Ravi, S.; and Larochelle, H. 2017.
> Optimization as a Model for Few-Shot Learning.  In _ICLR_.
>   * Ren et al. (2018) Ren, M.; Ravi, S.; Triantafillou, E.; Snell, J.;
> Swersky, K.; Tenenbaum, J. B.; Larochelle, H.; and Zemel, R. S. 2018.  Meta-
> Learning for Semi-Supervised Few-Shot Classification.  In _International
> Conference on Learning Representations_.
>   * Requeima et al. (2019) Requeima, J.; Gordon, J.; Bronskill, J.; Nowozin,
> S.; and Turner, R. E. 2019.  Fast and Flexible Multi-Task Classification
> using Conditional Neural Adaptive Processes.  In _Advances in Neural
> Information Processing Systems_ , volume 32.
>   * Rusu et al. (2019) Rusu, A. A.; Rao, D.; Sygnowski, J.; Vinyals, O.;
> Pascanu, R.; Osindero, S.; and Hadsell, R. 2019.  Meta-Learning with Latent
> Embedding Optimization.  In _International Conference on Learning
> Representations_.
>   * Satorras and Estrach (2018) Satorras, V. G.; and Estrach, J. B. 2018.
> Few-Shot Learning with Graph Neural Networks.  In _International Conference
> on Learning Representations_.
>   * SHEN et al. (2021) SHEN, X.; Xiao, Y.; Hu, S. X.; Sbai, O.; and Aubry,
> M. 2021.  Re-ranking for image retrieval and transductive few-shot
> classification.  In Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J.
> W., eds., _Advances in Neural Information Processing Systems_.
>   * Snell, Swersky, and Zemel (2017) Snell, J.; Swersky, K.; and Zemel, R.
> 2017.  Prototypical Networks for Few-shot Learning.  In _Advances in Neural
> Information Processing Systems_ , volume 30.
>   * Snell and Zemel (2021) Snell, J.; and Zemel, R. 2021.  Bayesian Few-Shot
> Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes.
> In _International Conference on Learning Representations_.
>   * Sun et al. (2021) Sun, Z.; Wu, J.; Li, X.; Yang, W.; and Xue, J.-H.
> 2021.  Amortized Bayesian Prototype Meta-learning: A New Probabilistic Meta-
> learning Approach to Few-shot Image Classification.  In _Proceedings of The
> 24th International Conference on Artificial Intelligence and Statistics_ ,
> Proceedings of Machine Learning Research.
>   * Sung et al. (2018) Sung, F.; Yang, Y.; Zhang, L.; Xiang, T.; Torr, P.
> H.; and Hospedales, T. M. 2018\.  Learning to Compare: Relation Network for
> Few-Shot Learning.  In _Proceedings of the IEEE Conference on Computer
> Vision and Pattern Recognition (CVPR)_.
>   * Tseng et al. (2020) Tseng, H.-Y.; Lee, H.-Y.; Huang, J.-B.; and Yang,
> M.-H. 2020.  Cross-domain few-shot classification via learned feature-wise
> transformation.  _arXiv preprint arXiv:2001.08735_.
>   * Vapnik (2006) Vapnik, V. N. 2006.  Estimation of Dependences Based on
> Empirical Data.  _Estimation of Dependences Based on Empirical Data_.
>   * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit,
> J.; Jones, L.; Gomez, A. N.; Kaiser, L. u.; and Polosukhin, I. 2017.
> Attention is All you Need.  In _Advances in Neural Information Processing
> Systems_ , volume 30.
>   * Vinyals et al. (2016) Vinyals, O.; Blundell, C.; Lillicrap, T.;
> kavukcuoglu, k.; and Wierstra, D. 2016\.  Matching Networks for One Shot
> Learning.  In _Advances in Neural Information Processing Systems_ , volume
> 29.
>   * Wang et al. (2019) Wang, Y.; Chao, W.-L.; Weinberger, K. Q.; and van der
> Maaten, L. 2019.  SimpleShot: Revisiting Nearest-Neighbor Classification for
> Few-Shot Learning.  arXiv:1911.04623.
>   * Welinder et al. (2010) Welinder, P.; Branson, S.; Mita, T.; Wah, C.;
> Schroff, F.; Belongie, S.; and Perona, P. 2010.  Caltech-UCSD Birds 200.
> Technical Report CNS-TR-2010-001, California Institute of Technology.
>   * Wertheimer, Tang, and Hariharan (2021) Wertheimer, D.; Tang, L.; and
> Hariharan, B. 2021.  Few-Shot Classification With Feature Map Reconstruction
> Networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and
> Pattern Recognition (CVPR)_ , 8012–8021.
>   * Wu et al. (2021) Wu, J.; Zhang, T.; Zhang, Y.; and Wu, F. 2021.  Task-
> Aware Part Mining Network for Few-Shot Learning.  In _Proceedings of the
> IEEE/CVF International Conference on Computer Vision_ , 8433–8442.
>   * Xu et al. (2021) Xu, W.; yifan xu; Wang, H.; and Tu, Z. 2021.
> Attentional Constellation Nets for Few-Shot Learning.  In _International
> Conference on Learning Representations_.
>   * Yang et al. (2020) Yang, L.; Li, L.; Zhang, Z.; Zhou, X.; Zhou, E.; and
> Liu, Y. 2020.  DPGN: Distribution Propagation Graph Network for Few-Shot
> Learning.  _2020 IEEE/CVF Conference on Computer Vision and Pattern
> Recognition (CVPR)_ , 13387–13396.
>   * Ye et al. (2020) Ye, H.-J.; Hu, H.; Zhan, D.-C.; and Sha, F. 2020.  Few-
> Shot Learning via Embedding Adaptation with Set-to-Set Functions.  In
> _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ ,
> 8808–8817.
>   * Yoon et al. (2018) Yoon, J.; Kim, T.; Dia, O.; Kim, S.; Bengio, Y.; and
> Ahn, S. 2018.  Bayesian Model-Agnostic Meta-Learning.  In _Advances in
> Neural Information Processing Systems_ , volume 31.
>   * Zhang et al. (2021a) Zhang, B.; Li, X.; Ye, Y.; Huang, Z.; and Zhang, L.
> 2021a.  Prototype Completion With Primitive Knowledge for Few-Shot Learning.
> In _CVPR_ , 3754–3762.
>   * Zhang et al. (2019) Zhang, J.; Zhao, C.; Ni, B.; Xu, M.; and Yang, X.
> 2019.  Variational Few-Shot Learning.  In _2019 IEEE/CVF International
> Conference on Computer Vision (ICCV)_ , 1685–1694.
>   * Zhang et al. (2021b) Zhang, X.; Meng, D.; Gouk, H.; and Hospedales, T.
> M. 2021b.  Shallow bayesian meta learning for real-world few-shot
> recognition.  In _Proceedings of the IEEE/CVF International Conference on
> Computer Vision_ , 651–660.
>   * Ziko et al. (2020) Ziko, I. M.; Dolz, J.; Granger, E.; and Ayed, I. B.
> 2020.  Laplacian Regularized Few-Shot Learning.  In _ICML_ , 11660–11670.
>

##  Appendix A Appendix

### Impact of AttFEX

In order to study the impact of the transductive feature extractor AttFEX, we
exclude it during training and train the remaining architecture. Training
proceeds exactly as mentioned before in the manuscript.

Table 5: Impact of AttFEX on classification accuracies.

| miniImagenet | tieredImagenet
---|---|---
| (5-way, 1-shot) | (5-way, 5-shot) | (5-way, 1-shot) | (5-way, 5-shot)
AttFEX OFF | 67.68 $\pm$ 0.55 | 78.53 $\pm$ 0.21 | 69.32 $\pm$ 0.76 | 79.32 $\pm$ 0.76
AttFEX ON | 86.11 $\pm$ 0.59 | 95.95 $\pm$ 0.28 | 86.97 $\pm$ 0.50 | 96.57 $\pm$ 0.17

As can be seen in Table 5, the exclusion of AttFEX from TRIDENT results in a
substantial drop in classification performance across both datasets and task
settings. Empirically, this further substantiates the importance of AttFEX’s
ability to render the feature maps transductive/task-aware. As explained
earlier in the main manuscript, it is imperative to include $y$ in the input
to $q_{\phi_{1}}(.)$ for mathematical correctness of the variational inference
formulation. However, in order to utilize TRIDENT as a classification and not
a label reconstruction network, we choose not to input $y$ to
$q_{\phi_{1}}(.)$, but rather do so indirectly by inducing a semblance of
label characteristics in the features extracted from the images in a task.
Thus, it is important to realize that this ability of AttFEX to render feature
maps transductive is not just an adhoc performance enhancer, but rather an
essential part of TRIDENT since it allows us to not violate our generative and
inference mechanics.

### Additional Details of Datasets

miniImagenet (Vinyals et al. 2016) is a subset of ImageNet (Deng et al. 2009)
for few-shot classification. It contains $100$ classes with $600$ samples
each. We follow the predominantly adopted settings of (Ravi and Larochelle
2017; Chen et al. 2019) where we split the entire dataset into $64$ classes
for training, $16$ for validation and $20$ for testing. tieredImagenet is a
larger subset of ImageNet (Deng et al. 2009) with $608$ classes and $779,165$
total images, which are grouped into $34$ higher-level nodes in the ImageNet
human-curated hierarchy. This set of nodes is partitioned into $20$, $6$, and
$8$ disjoint sets of training, validation, and testing nodes, and the
corresponding classes form the respective meta-sets. CUB (Welinder et al.
2010) dataset has a total of $200$ classes, split into training, validation
and test sets following (Chen et al. 2019). We use this dataset to simulate
the effect of a domain shift where the model is first trained on a ($5$-way,
$1$ or $5$-shot) configuration of miniImagenet and then tested on the test
classes of CUB, as used in (Chen et al. 2019; Boudiaf et al. 2020; Ziko et al.
2020; Long et al. 2018).

### Implementational Details

Let $\alpha_{1}$ and $\alpha_{2}$ respectively denote the scaling factors of
the MSE and cross-entropy terms in our objective functions $\mathcal{L}_{R}$
and $\mathcal{L}_{C}$, as already defined in Subsection 4.2. The terms
$\alpha$ and $\beta$ respectively denote the learning rates of the _inner_ and
_meta_ updates whereas $B$ and $n$ respectively denote the number of sampled
tasks and adaptation steps of the _inner_ -update of our end-to-end training
process, as described in Algorithm 2. The hyperparameter values (H.P.) used
for training TRIDENT on miniImagenet and tieredImagenet are shown in Table 6.

Table 6: Hyperparameter values when training TRIDENT.

| miniImagenet | tieredImagenet
---|---|---
H.P. |  $5$-way, $1$-shot |  $5$-way, $5$-shot |  $5$-way, $1$-shot |  $5$-way, $5$-shot
$\alpha_{1}$ | 1e-2 | 1e-2 | 1e-2 | 1e-2
$\alpha_{2}$ | 100 | 100 | 150 | 150
$\alpha$ | 1e-3 | 1e-3 | 1.5e-3 | 1.7e-3
$\beta$ | 1e-4 | 1e-4 | 1.5e-4 | 1.7e-4
$B$ | 20 | 20 | 20 | 20
$n$ | 5 | 5 | 5 | 5

We apply the same hyperparameters for the cross-domain testing scenario of
miniImagenet $\rightarrow$ CUB used for training TRIDENT on miniImagenet, for
the given ($N$-way, $K$-shot) configuration. Hyperparameters are kept fixed
throughout training, validation and testing for a given configuration. Adam
(Kingma and Ba 2015) optimizer is used for inner and meta-updates. Finally,
the query, key and value extraction networks $f_{q}(,;\textbf{W}_{Q})$,
$f_{k}(.;\textbf{W}_{K})$, $f_{v}(.;\textbf{W}_{V})$ of the AttFEX module only
use $\texttt{Conv}_{1\times 1}(.)$ and not the LeakyReLU(0.2) activation
function for ($5$-way, $1$-shot) tasks, irrespective of the dataset. We
observed that utilizing BatchNorm (Ioffe and Szegedy 2015) in the decoder of
$z_{s}$ ($p_{\theta_{2}}$) to train TRIDENT on ($5$-way, $5$-shot) tasks of
miniImagenet and on ($5$-way, $1$-shot) tasks of tieredImagenet leads to
better scores and improved stability during training. We used the ReLU
activation function instead of LeakyReLU(0.2) to carry out training on
($5$-way, $1$-shot) tasks of tieredImagenet. Meta-learning objectives can lead
to unstable optimization processes in practice, especially when coupled with
stochastic sampling in latent spaces, as also previously observed in (Antreas
Antoniou, Harrison Edwards, and Amos J. Storkey 2019; Rusu et al. 2019). For
ease of experimentation we clip the meta-gradient norm at an absolute value of
1. TRIDENT converges in $82,000$ and $22,500$ epochs for ($5$-way, $1$-shot)
and ($5$-way, $5$-shot) tasks of miniImagenet, respectively and takes $67,500$
and $48,000$ epochs for convergence on ($5$-way, $1$-shot) and ($5$-way,
$5$-shot) tasks of tieredImagenet, respectively. This translates to an average
training time of $110$ hours on an $11$GB NVIDIA 1080Ti GPU. Note that we did
not employ any data augmentation, feature averaging or any other data apart
from the corresponding training subset $\mathcal{D}^{tr}$, during training and
evaluation.

### Additional Calibration Results

To further examine the reliability and calibration of our method, we assess
the ECE, MCE (Guo et al. 2017) and Brier scores (BRIER 1950) of TRIDENT on the
challenging _cross-domain_ scenario of miniImagenet $\rightarrow$ CUB for
($5$-way, $5$-shot) tasks. This table can be treated as an extension to Table
3 since here, we compare the calibration metrics for an additional scenario.
When compared against other baselines that report these metrics on the
aforementioned scenario, TRIDENT proves to be the most calibrated with the
best reliability scores. This is shown in Table 7.

Table 7: Style: best and second best.

Methods | ECE | MCE | Brier
---|---|---|---
Feature Transfer(Chen et al. 2019) | 0.275 | 0.646 | 0.772
Baseline(Chen et al. 2019) | 0.315 | 0.537 | 0.716
Matching Nets(Vinyals et al. 2016) | 0.030 | 0.079 | 0.630
Proto Nets(Snell, Swersky, and Zemel 2017) | 0.009 | 0.025 | 0.604
Relation Net(Sung et al. 2018) | 0.234 | 0.554 | 0.730
DKT+Cos(Patacchiola et al. 2020) | 0.236 | 0.426 | 0.670
BMAML(Yoon et al. 2018) | 0.048 | 0.077 | 0.619
BMAML+Chaser(Yoon et al. 2018) | 0.066 | 0.260 | 0.639
LogSoftGP(ML)(Galy-Fajou et al. 2020) | 0.220 | 0.513 | 0.709
LogSoftGP(PL)(Galy-Fajou et al. 2020) | 0.022 | 0.042 | 0.564
OVE(ML)(Snell and Zemel 2021) | 0.049 | 0.066 | 0.576
OVE(PL)(Snell and Zemel 2021) | 0.020 | 0.032 | 0.556
TRIDENT(Ours) | 0.009 | 0.02 | 0.276