Few-Shot Learning (learning with limited labeled data) aims to overcome the limitations of traditional machine learning approaches which require thousands of labeled examples to train an effective model. Considered as a hallmark of human intelligence, the community has recently witnessed several contributions on this topic, in particular through meta-learning, where a model learns how to learn an effective model for few-shot learning. The main idea is to acquire prior knowledge from a set of training tasks, which is then used to perform (few-shot) test tasks. Most existing work assumes that both training and test tasks are drawn from the same distribution, and a large amount of labeled data is available in the training tasks. This is a very strong assumption which restricts the usage of meta-learning strategies in the real world where ample training tasks following the same distribution as test tasks may not be available. In this paper, we propose a novel meta-learning paradigm wherein a few-shot learning model is learnt, which simultaneously overcomes domain shift between the train and test tasks via adversarial domain adaptation. We demonstrate the efficacy the proposed method through extensive experiments. Few-Shot Learning aims to learn a prediction model from very limited amount of labelled data BID13 . Specifically, given a K−shot, N −class data for a classification task, the aim is to learn a multi-class classification model for N − classes, with K−labeled training examples for each class. Here K is usually a small number (e.g. 1, or 5). Considered as one of the hallmarks of human intelligence BID12 , this topic has received considerable interest in recent years BID13 BID11 BID33 BID2 . Modern techniques solve this problem through meta-learning, using an episodic learning paradigm. The main idea is to use a labeled training dataset to effectively acquire prior knowledge, such that this knowledge can be transferred to novel tasks where few-shot learning is to be performed. Different from traditional transfer learning BID21 BID35 , here few-shot tasks are simulated using the labeled training data through episodes, in order to acquire prior knowledge that is specifically tailored for performing few-shot tasks. For example, given a set of labeled training data with a finite label space Y train , the epsiodic paradigm is used to acquire prior knowledge which is stored in a model. Each episode is generated i.i.d from an unknown task distribution τ train . This model is then used to do a novel few shot classification task which is drawn from an unknown task distribution τ test . The test task comprises small amount of labeled data with a finite label space Y test , and the sets Y train and Y test are (possibly) mutually exclusive. Using this labeled data, and acquired prior knowledge, the goal is to predict the labels of all unlabeled instances in the test task.A very restrictive assumption of existing meta-learning approaches for few-shot learning is that train and test tasks are drawn from the same distribution, i.e., τ train = τ test . In this scenario, the metalearner's objective is to minimize its expected loss over the tasks drawn from the task distribution τ train . This assumption prohibits the use of meta-learning strategies for real-world applications, where training tasks with ample labeled data, and drawn from the same distribution as the test tasks are very unlikely to be available. Consider the case of a researcher or practitioner who wishes to train a prediction model for their own dataset where labeled data is very limited. It is unreasonable to assume that they would have a large corpus of labeled data for a set of related tasks in the same domain. Without this, they are not able to train effective few-shot models for their task. A more desirable option is to use the training tasks where ample training data is available, and adapt the model to be effective on test tasks in a different domain. A possible way to tackle this problem could be through the use of domain adaptation techniques BID4 BID7 that address the domain shift between the training and test data. However, all of these approaches address the single-task scenario, i.e., Y train = Y test , where the training data and test data are sampled from the same task but there is a domain shift at a data-level. This is in contrast to the meta-learning setting where the training data contains multiple tasks and the goal is to learn new tasks from test data, i.e., domain shift exists at a task-level and Y train ∩ Y test = ∅. As a result, these domain adaptation approaches cannot be directly applied. We show an overview of different problem settings in TAB0 . DISPLAYFORM0 In order to solve the few-shot learning problem under a domain shift we propose a novel meta-learning paradigm: Meta-Learning with Domain Adaptation (MLDA). Existing meta-learning approaches for few-shot learning use only the given training data to learn a model, and as a result they do not account for any domain shift between the training tasks and the few-shot test tasks. In contrast, we assume that the model has access to the unlabeled instances in the domain of the few-shot test tasks prior to the training procedure, and utilize these instances for incorporating the domain-shift information. We train the model under the episodic-learning paradigm, but in each episode we aim to train a model which achieves two goals: first the model should be good at few-shot learning, and second the model should be unaffected by a possible domain shift. The first goal is achieved by updating the model based on the few-shot learning loss suffered by the model for a given episode. The second goal is achieved by an adversarial domain adaptation approach, where a mapping is used which styles the training task to resemble the test task. In this way, the trained model can perform few-shot predictions on the test tasks, and achieve what we term task-level domain adaptation.The episodic update is done via Prototypical Networks BID28 (as a specific instantiation, though other approaches can be applied), where on a simulated few-shot task (a small support set behaves as training, and a query set behaves as test data), an embedding is produced for both support and query instances. The mean of support embedding of each class is the prototype, and query instances are labeled based on their distance to these prototypes. Based on the loss on these query instances, the embedding function is updated. For achieving invariance to domain shift, we follow the principle of adversarial domain adaptation, but we differ from the traditional approaches in that we are performing task-level domain adaptation, whereas they performed data-level domain adaptation. The early approaches to adversarial domain adaptation aimed at obtaining a feature embedding that was invariant to both the training domain and the test domain, as well as learning a prediction model in the training domain BID4 . However, these approaches possibly learnt a highly unconstrained feature embedding (particularly when the embedding was very high dimensional), and were outperformed by GAN-based approaches (often used for image translation) BID29 BID38 BID7 . As a result we use a mapping function to style the training tasks to resemble test tasks, and optimize it using a GAN loss. The overall framework delivers a model that uses training tasks from one distribution to meta-learn a few-shot model for a task from another distribution. We perform extensive experiments to show the efficacy of the proposed method.2 RELATED WORK 2.1 META-LEARNING FOR FEW-SHOT LEARNING Few-Shot Learning refers to learning a prediction model from small amount of labeled data BID1 BID12 . Early approaches used a Bayesian model BID1 , or hand-designed priors BID13 . More recently, meta-learning approaches have become extremely successful for addressing few-shot learning BID33 BID2 . Instead of training a model directly on the few-shot data, meta-learning approaches use a corpus of labeled data, and simulate few-shot tasks on them to learn how to do few-shot learning. Some approaches follow the non-parametric principle, and develop a differentiable K−nearest neighbour solution BID33 BID27 BID28 . The main concept is to learn an embedding space that is tailored for performing effective K-nearest neighbour. BID20 extend these approaches with metric scaling to condition the embedding based on the given task. Another category of meta-learning aims to learn how to quickly adapt a model in few gradient steps for a few-shot learning task BID2 BID23 . These optimization based approaches aim to learn an initialization from a set of training tasks, which can be quickly adapted (e.g. one-step gradient update) when presented with a novel few-shot task. Some other approaches consider using a "memory"-based approach BID26 BID19 . There have also been approaches that try to enhance meta-learning performance through use of additional information. For example, BID24 use unlabeled data to develop semi supervised few-shot learning. BID37 use external data to generate concepts, and performs meta-learning in the concept space. However, all of these approaches assume that the training tasks and testing tasks are drawn from the same distribution (τ train = τ test ). If there is a task-level domain shift, the above approaches will fail to perform few-shot learning on novel test tasks. Our approach of meta-learning with domain adaptation overcomes this domain shift, to perform few-shot learning on tasks in a different domain. In this paper we investigated a novel problem setting: Meta-Learning for few-shot learning under task-level domain shift. Existing meta learning paradigm for few-shot learning was designed under the assumption that both training tasks and test tasks were drawn from the same distribution. This may not be the case for real world applications, where researchers may not find ample labeled data to simulate training tasks to be drawn from the same distribution as their test tasks. To alleviate this, we propose a meta learning with domain adaptation paradigm, which performs meta-learning by incorporating few-shot learning and task-level domain adaptation unified into a single meta-learner. We instantiate our few-shot model with Prototypical Networks and adopt an adversarial approach for task level domain adaptation. We conduct several experiments to validate the proposed ideas.6 APPENDIX: DATASET CONSTRUCTION DISPLAYFORM0 Here we show the details of the original Omniglot dataset and the statistical details, and some examples of how the characters look in a different domain. The meta-train, meta-test, and domain adaptation split of classes we used are based on the same split used in prior work. There is no overlap of classes or instances among the three sets, i.e., they are all mutually exclusive both at instance and class-level. 7 APPENDIX: MODEL CONFIGURATION AND TRAINING For MLDA, we followed training procedures adopted similar to BID38 and BID28 . Specifically, for CycleGAN, we changed the parameters related to image dimensions (scaling and cropping pre-processing) to keep the generated image size fixed to the original size i.e. 28 × 28 for Omniglot/Omniglot-M and 84 × 84 for OfficeHome Clipart/Product. The generative networks are the same as the original work BID38 , each including two stride-2 convolutions with residual blocks, and two fractionally-strided convolutions with stride 1 2 . For all experiments, we used 6 blocks to generate images. We initialized the learning rate to 0.0002 and kept this learning rate for training till 100 epochs. The model after each epoch was used to translate source task to target task. Weights were initialized with Gaussian distribution with mean 0 and standard deviation 0.02. We use the Adam solver with a batch size of 1. We also modified the loss function for diffent settings of MLDA. Specifically, for MLDA, we removed the losses related to target → source (B → A) mapping and set λ idt = 0. For MLDA+idt, we set λ idt = 0.1. For MLDA+idt+revMap, we kept the loss function the same as the original CycleGAN.For Prototypical Networks, we followed the best hyperparameter settings in BID28 . We used the same embedding architecture in the original work, including four convoluational blocks, each of which comprises a 64-filter 3 × 3 convolution, batch normalization layer, ReLU activation, and 2 × 2 max-pooling layer. This results in 64-dimensional output space for Omniglot/Omniglotm and 1600-dimensional output space for HomeOffice Clipart/Product. For Omniglot/Omniglotm experiments, the learning rate was set to 0.001 and reduced by half every 2K iterations, starting from iteration 2K. The network is trained for a total of 20K iterations. For OfficeHome Product/Clipart experiments, we initialized the learning rate to 0.001 and decayed the learning rate by half every 25K iterations, starting from iteration 25K. The model is trained up to 100K iterations. We also use Adam solver to optimize the networks. Following BID28 , we chose squared Euclidean distance to perform kNN classification as this metric showed superior performance in prior work.For all the baselines, we reused the official code and ran them with default hyperparameters. We only modified parameters to make the models compatible with the image resolution and number of classes in Omniglot/Omniglot-M and Product/Clipart datasets. In all experiments, we set N c classes and N S support points per class identical at training and test-time. We also fixed 15 query points per class per episode in all experiments. We computed the classification accuracy by averaging over 600 randomly generated episodes from the Meta-test set.
Meta Learning for Few Shot learning assumes that training tasks and test tasks are drawn from the same distribution. What do you do if they are not? Meta Learning with task-level Domain Adaptation! This paper proposes a model combining unsupervised adversarial domain adaptation with prototypical networks that performs better than few-shot learning baselines on few-shot learning tasks with domain shift. The authors proposed meta domain adaptation to address domain shift scenario in meta learning setup, demonstrating performance improvements in several experiments.