The neural linear model is a simple adaptive Bayesian linear regression method that has recently been used in a number of problems ranging from Bayesian optimization to reinforcement learning. Despite its apparent successes in these settings, to the best of our knowledge there has been no systematic exploration of its capabilities on simple regression tasks. In this work we characterize these on the UCI datasets, a popular benchmark for Bayesian regression models, as well as on the recently introduced ''gap'' datasets, which are better tests of out-of-distribution uncertainty. We demonstrate that the neural linear model is a simple method that shows competitive performance on these tasks.
We benchmark the neural linear model on the UCI and UCI "gap" datasets.