Meta-learning will be crucial to creating lifelong, generalizable AI. In practice, however, it is hard to define the meta-training task distribution that is used to train meta-learners. If made too small, tasks are too similar for a model to meaningfully generalize. If made too large, generalization becomes incredibly difficult. We argue that both problems can be alleviated by introducing a teacher model that controls the sequence of tasks that a meta-learner is trained on. This teacher model is incentivized to start the student meta-learner on simple tasks then adaptively increase task difficulty in response to student progress. While this approach has been previously studied in curriculum generation, our main contribution is in extending it to meta-learning. Humans are incredibly good at generalizing to unseen tasks. But, humans are only able to do so because they lean on a vast history of experience. Within a single lifespan, we begin by learning simple tasks: crawling, walking, talking. As we age, we learn progressively more and more difficult tasks, borrowing from the simpler to inform the more complex.In order for machines to exhibit this same behavior, they have to learn how to generalize and borrow from previous experiences. Once they can do so reliably, we move a step closer to the holy grail of Artificial General Intelligence. However, artificial intelligence systems are incredibly brittle. Because we have an incomplete understanding of how to best learn from past experiences, it is unclear how we can create robust, generalizable AI agents.We propose to tackle this problem by combining curriculum learning and meta learning into an approach called metateaching. We aim to teach meta-AI agents to start from easy Preliminary work. Under review by the International Conference on Machine Learning (ICML). Do not distribute. Sample meta-batch of tasks from perturbed task- DISPLAYFORM0 Evaluate L S T i with respect to K samples from task 5:Compute adapted student parameters using either a gradient based or gradient free update 6: end for DISPLAYFORM1 Approximate difficulty gradient of current task-space DISPLAYFORM2 Update adapted task-space parameters with gradient descent over i student losses: DISPLAYFORM3 . 9: end while tasks, progressively learn harder tasks, and use information about easy tasks to inform the harder ones. In order to do so, we introduce a teacher that updates the difficulty of the current task in response to student progress.Before describing meta-teaching, we will discuss related works and provide preliminary information.
Teacher that trains meta-learners like humans