Transferring knowledge across tasks to improve data-efficiency is one of
 the open key challenges in the area of global optimization algorithms. Readily
 available algorithms are typically designed to be universal optimizers and, thus,
 often suboptimal for specific tasks. We propose a novel transfer learning method to
 obtain customized optimizers within the well-established framework of Bayesian
 optimization, allowing our algorithm to utilize the proven generalization
 capabilities of Gaussian processes. Using reinforcement learning to meta-train an
 acquisition function (AF) on a set of related tasks, the proposed method learns to
 extract implicit structural information and to exploit it for improved data-efficiency.
 We present experiments on a sim-to-real transfer task as well as on several simulated
 functions and two hyperparameter search problems. The results show that our
 algorithm (1) automatically identifies structural properties of objective functions
 from available source tasks or simulations, (2) performs favourably in settings with
 both scarse and abundant source data, and (3) falls back to the performance level
 of general AFs if no structure is present. Global optimization of black-box functions is highly relevant for a wide range of real-world tasks. Examples include the tuning of hyperparameters in machine learning, the identification of control parameters or the optimization of system designs. Such applications oftentimes require the optimization of relatively low-dimensional ( 10D) functions where each function evaluation is expensive in either time or cost. Furthermore, there is typically no gradient information available. In this context of data-efficient global black-box optimization, Bayesian optimization (BO) has emerged as a powerful solution (Mockus, 1975; Brochu et al., 2010; Snoek et al., 2012; Shahriari et al., 2016 ). BO's data efficiency originates from a probabilistic surrogate model which is used to generalize over information from individual data points. This model is typically given by a Gaussian process (GP), whose well-calibrated uncertainty prediction allows for an informed explorationexploitation trade-off during optimization. The exact manner of performing this trade-off, however, is left to be encoded in an acquisition function (AF). There is wide range of AFs available in the literature which are designed to yield universal optimization strategies and thus come with minimal assumptions about the class of target objective functions. To achieve optimal data-efficiency on new instances of previously seen tasks, however, it is crucial to incorporate the information obtained from these tasks into the optimization. Therefore, transfer learning (or warm-starting) is an important and active field of research. Indeed, in many practical applications, optimizations are repeated numerous times in similar settings, underlining the need for specialized optimizers. Examples include hyperparameter optimization which is repeatedly done for the same machine learning model on varying datasets or the optimization of control parameters for a given system with varying physical configurations. Following recent approaches (Swersky et al., 2013; Feurer et al., 2018; Wistuba et al., 2018) , we argue that it is beneficial to perform transfer learning for global black-box optimization in the framework of BO to retain the proven generalization capabilities of its underlying GP surrogate model. To not restrict the expressivity of this model, we propose to implicitly encode the task structure in a specialized AF, i.e., in the optimization strategy. We realize this encoding via a novel method which meta-learns a neural AF, i.e., a neural network representing the AF, on a set of training tasks. The meta-training is performed using reinforcement learning, making the proposed approach applicable to the standard BO setting, where we do not assume access to objective function gradients. Our contributions are (1) a novel transfer learning method allowing the incorporation of implicit structural knowledge about a class of objective functions into the framework of BO through learned neural AFs to increase data-efficiency on new task instances, (2) an automatic and practical metalearning procedure for training such neural AFs which is fully compatible with the black-box optimization setting, i.e, not requiring gradients, and (3) the demonstration of the efficiency and practical applicability of our approach on a challenging hardware control task, hyperparameter optimization problems, as well as a set of synthetic functions. We introduced MetaBO, a novel approach for transfer learning in the framework of BO. Via a flexible meta-learning approach we inject prior knowledge directly into the optimization strategy of BO using neural AFs. Our experiments on several real-world optimization tasks show that our method consistently outperforms the popular general-purpose AF EI as well as the state-of-the-art solution TAF for warmstarting BO, for instance in simulation-to-real settings or on hyperparameter search tasks. Our approach is broadly applicable to a wide range of practical problems, covering both the cases of scarse and abundant source data. The resulting neural AFs generalize well beyond the training distribution, allowing our algorithm to perform robustly unseen problems. In future work, we aim to tackle the multi-task multi-fidelity setting (Valkov et al., 2018) , where we expect MetaBO's sample efficiency to be of high impact.
We perform efficient and flexible transfer learning in the framework of Bayesian optimization through meta-learned neural acquisition functions. The authors present MetaBO which uses reinforcement learning to meta-learn the acquisition function for Bayesian Optimization, showing increasing sample efficiency on new tasks. The authors propose a meta-learning based alternative to standard acquisition functions (AFs), whereby a pretrained neural network outputs acquisition values as a function of hand-chosen features.