Previous work on adversarially robust neural networks requires large training sets and computationally expensive training procedures.   On the other hand, few-shot learning methods are highly vulnerable to adversarial examples.   The goal of our work is to produce networks which both perform well at few-shot tasks and are simultaneously robust to adversarial examples.   We adapt adversarial training for meta-learning, we adapt robust architectural features to small networks for meta-learning, we test pre-processing defenses as an alternative to adversarial training for meta-learning, and we investigate the advantages of robust meta-learning over robust transfer-learning for few-shot tasks.   This work provides a thorough analysis of adversarially robust methods in the context of meta-learning, and we lay the foundation for future work on defenses for few-shot tasks. For safety-critical applications like facial recognition, traffic sign detection, and copyright control, adversarial attacks pose an actionable threat (Zhao et al., 2018; Eykholt et al., 2017; . Conventional adversarial training and pre-processing defenses aim to produce networks that resist attack (Madry et al., 2017; Zhang et al., 2019; Samangouei et al., 2018) , but such defenses rely heavily on the availability of large training data sets. In applications that require few-shot learning, such as face recognition from few images, recognition of a video source from a single clip, or recognition of a new object from few example photos, the conventional robust training pipeline breaks down. When data is scarce or new classes arise frequently, neural networks must adapt quickly (Duan et al., 2017; Kaiser et al., 2017; Pfister et al., 2014; Vartak et al., 2017) . In these situations, metalearning methods conduct few-shot learning by creating networks that learn quickly from little data and with computationally cheap fine-tuning. While state-of-the-art meta-learning methods perform well on benchmark few-shot classification tasks, these naturally trained neural networks are highly vulnerable to adversarial examples. In fact, even adversarially trained feature extractors fail to resist attacks in the few-shot setting (see Section 4.1). We propose a new approach, called adversarial querying, in which the network is exposed to adversarial attacks during the query step of meta-learning. This algorithm-agnostic method produces a feature extractor that is robust, even without adversarial training during fine-tuning. In the few-shot setting, we show that adversarial querying outperforms other robustness techniques by a wide margin in terms of both clean accuracy and adversarial robustness (see Table 1 ). We solve the following minimax problem: where S and (x, y) are data sampled from the training distribution, A is a fine-tuning algorithm for the model parameters, θ, and is a p-norm bound for the attacker. In Section 4, we further motivate adversarial querying and exhibit a wide range of experiments. To motivate the necessity for adversarial querying, we test methods, such as adversarial fine-tuning and pre-processing defenses, which if successful, would eliminate the need for expensive adversarial training routines. We find that these methods are far less effective than adversarial querying. Naturally trained networks for few-shot image classification are vulnerable to adversarial attacks, and existing robust transfer learning methods do not perform well on few-shot tasks. Even, when adversarially fine-tuned, naturally trained networks suffer from adversarial vulnerability. We thus identify the need for few-shot methods for adversarial robustness. In particular, we study robustness in the context of meta-learning. We develop an algorithm-agnostic method, called adversarial querying, for hardening meta-learning models. We find that meta-learning models are most robust when the feature extractor is fixed and only the last layer is retrained during fine-tuning. We further identify that choice of classification head significantly impacts robustness. We believe that this paper is a starting point for developing adversarially robust methods for few-shot applications. We train ProtoNet, R2-D2, and MetaOptNet models for 60 epochs with SGD. We use a learning rate of 0.1, momentum (Nesterov) of 0.9, and a weight decay term of 5(10 −4 ) for the parameters of both the head and the embedding. We decrease the learning rate to 0.06 after epoch 20, 0.012 after epoch 40, and 0.0024 after epoch 50. MAML is trained for 60000 epochs with meta learning rate of 0.001 and fine-tuning learning rate of 0.01. Fine-tuning is performed for 10 steps per task.
We develop meta-learning methods for adversarially robust few-shot learning. This paper presents a method that enhances the robustness of few-shot learning by introducing adversarial query data attack in the inner-task fine-tuning phase of a meta-learning algorithm. The authors of this paper propose a novel approach for training a robust few-shot model. 