# GP Regression with Uncertain Inputs

## Introduction

In this notebook, we're going to demonstrate one way of dealing with uncertainty in our training data. Let's say that we're collecting training data that models the following function.

\begin{align}
y &= \sin(2\pi x) + \epsilon \\
  \epsilon &\sim \mathcal{N}(0, 0.2) 
\end{align}

However, now assume that we're a bit uncertain about our features. In particular, we're going to assume that every `x_i` value is not a point but a distribution instead. E.g.

$$ x_i \sim \mathcal{N}(\mu_i, \sigma_i). $$

### Using a distributional kernel to deal with uncertain inputs

Rather than using a variational method (see the GP Regression with Uncertian Inputs tutorial in the variational examples), if we explicitly know the type of uncertainty in our inputs we can pass that into our kernel.

More specifically, assuming Gaussian inputs, we will compute the symmetrized KL divergence between the Gaussian inputs.




```python
import math
import torch
import tqdm
import gpytorch
from matplotlib import pyplot as plt

%matplotlib inline
%load_ext autoreload
%autoreload 2
```


```python
# Training data is 100 points in [0,1] inclusive regularly spaced
train_x_mean = torch.linspace(0, 1, 20)
# We'll assume the variance shrinks the closer we get to 1
train_x_stdv = torch.linspace(0.03, 0.01, 20)

# True function is sin(2*pi*x) with Gaussian noise
train_y = torch.sin(train_x_mean * (2 * math.pi)) + torch.randn(train_x_mean.size()) * 0.2
```

To effectively pass in the training distributional data, we will need to stack the mean and log variances.


```python
train_x_distributional = torch.stack((train_x_mean, (train_x_stdv**2).log()), dim=1)
```


```python
f, ax = plt.subplots(1, 1, figsize=(8, 3))
ax.errorbar(train_x_mean, train_y, xerr=(train_x_stdv * 2), fmt="k*", label="Train Data")
ax.legend()
```

We train the hyperparameters of the resulting distributional GP via type-II gradient descent, as is standard in many settings. We could also do fully Bayesian inference.


```python
from gpytorch.models import ExactGP
from gpytorch.kernels import GaussianSymmetrizedKLKernel, ScaleKernel
from gpytorch.means import ConstantMean

class ExactGPModel(ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = ConstantMean()
        self.covar_module = ScaleKernel(GaussianSymmetrizedKLKernel())
        
    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

# initialize likelihood and model
likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = ExactGPModel(train_x_distributional, train_y, likelihood)
```


```python
# this is for running the notebook in our testing framework
import os
smoke_test = ('CI' in os.environ)
training_iter = 2 if smoke_test else 500


# Find optimal model hyperparameters
model.train()
likelihood.train()

# Use the adam optimizer
optimizer = torch.optim.Adam(model.parameters(), lr=0.25)  # Includes GaussianLikelihood parameters

# "Loss" for GPs - the marginal log likelihood
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)

for i in range(training_iter):
    # Zero gradients from previous iteration
    optimizer.zero_grad()
    # Output from model
    output = model(train_x_distributional)
    # Calc loss and backprop gradients
    loss = -mll(output, train_y)
    loss.backward()
    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (
        i + 1, training_iter, loss.item(),
        model.covar_module.base_kernel.lengthscale.item(),
        model.likelihood.noise.item()
    ))
    optimizer.step()
```

    Iter 1/500 - Loss: 1.274   lengthscale: 0.693   noise: 0.693
    Iter 2/500 - Loss: 1.242   lengthscale: 0.826   noise: 0.576
    Iter 3/500 - Loss: 1.173   lengthscale: 0.955   noise: 0.475
    Iter 4/500 - Loss: 1.141   lengthscale: 1.102   noise: 0.389
    Iter 5/500 - Loss: 1.121   lengthscale: 1.266   noise: 0.317
    Iter 6/500 - Loss: 1.097   lengthscale: 1.442   noise: 0.261
    Iter 7/500 - Loss: 1.093   lengthscale: 1.621   noise: 0.220
    Iter 8/500 - Loss: 1.112   lengthscale: 1.801   noise: 0.192
    Iter 9/500 - Loss: 1.112   lengthscale: 1.988   noise: 0.177
    Iter 10/500 - Loss: 1.095   lengthscale: 2.188   noise: 0.172
    Iter 11/500 - Loss: 1.081   lengthscale: 2.401   noise: 0.172
    Iter 12/500 - Loss: 1.068   lengthscale: 2.624   noise: 0.177
    Iter 13/500 - Loss: 1.049   lengthscale: 2.856   noise: 0.183
    Iter 14/500 - Loss: 1.031   lengthscale: 3.095   noise: 0.188
    Iter 15/500 - Loss: 1.020   lengthscale: 3.340   noise: 0.191
    Iter 16/500 - Loss: 1.014   lengthscale: 3.589   noise: 0.189
    Iter 17/500 - Loss: 1.007   lengthscale: 3.842   noise: 0.183
    Iter 18/500 - Loss: 0.997   lengthscale: 4.098   noise: 0.173
    Iter 19/500 - Loss: 0.983   lengthscale: 4.356   noise: 0.159
    Iter 20/500 - Loss: 0.968   lengthscale: 4.616   noise: 0.144
    Iter 21/500 - Loss: 0.951   lengthscale: 4.877   noise: 0.128
    Iter 22/500 - Loss: 0.933   lengthscale: 5.139   noise: 0.112
    Iter 23/500 - Loss: 0.913   lengthscale: 5.402   noise: 0.097
    Iter 24/500 - Loss: 0.891   lengthscale: 5.667   noise: 0.083
    Iter 25/500 - Loss: 0.870   lengthscale: 5.933   noise: 0.070
    Iter 26/500 - Loss: 0.850   lengthscale: 6.202   noise: 0.059
    Iter 27/500 - Loss: 0.832   lengthscale: 6.474   noise: 0.050
    Iter 28/500 - Loss: 0.814   lengthscale: 6.748   noise: 0.042
    Iter 29/500 - Loss: 0.794   lengthscale: 7.025   noise: 0.035
    Iter 30/500 - Loss: 0.775   lengthscale: 7.304   noise: 0.029
    Iter 31/500 - Loss: 0.756   lengthscale: 7.586   noise: 0.024
    Iter 32/500 - Loss: 0.737   lengthscale: 7.868   noise: 0.020
    Iter 33/500 - Loss: 0.719   lengthscale: 8.152   noise: 0.017
    Iter 34/500 - Loss: 0.701   lengthscale: 8.437   noise: 0.014
    Iter 35/500 - Loss: 0.684   lengthscale: 8.722   noise: 0.012
    Iter 36/500 - Loss: 0.668   lengthscale: 9.007   noise: 0.010
    Iter 37/500 - Loss: 0.653   lengthscale: 9.291   noise: 0.008
    Iter 38/500 - Loss: 0.637   lengthscale: 9.575   noise: 0.007
    Iter 39/500 - Loss: 0.621   lengthscale: 9.859   noise: 0.006
    Iter 40/500 - Loss: 0.606   lengthscale: 10.142   noise: 0.005
    Iter 41/500 - Loss: 0.591   lengthscale: 10.424   noise: 0.005
    Iter 42/500 - Loss: 0.577   lengthscale: 10.705   noise: 0.004
    Iter 43/500 - Loss: 0.564   lengthscale: 10.985   noise: 0.003
    Iter 44/500 - Loss: 0.551   lengthscale: 11.263   noise: 0.003
    Iter 45/500 - Loss: 0.539   lengthscale: 11.540   noise: 0.003
    Iter 46/500 - Loss: 0.526   lengthscale: 11.815   noise: 0.002
    Iter 47/500 - Loss: 0.514   lengthscale: 12.088   noise: 0.002
    Iter 48/500 - Loss: 0.503   lengthscale: 12.359   noise: 0.002
    Iter 49/500 - Loss: 0.492   lengthscale: 12.627   noise: 0.002
    Iter 50/500 - Loss: 0.482   lengthscale: 12.892   noise: 0.001
    Iter 51/500 - Loss: 0.472   lengthscale: 13.154   noise: 0.001
    Iter 52/500 - Loss: 0.462   lengthscale: 13.412   noise: 0.001
    Iter 53/500 - Loss: 0.453   lengthscale: 13.668   noise: 0.001
    Iter 54/500 - Loss: 0.445   lengthscale: 13.919   noise: 0.001
    Iter 55/500 - Loss: 0.437   lengthscale: 14.167   noise: 0.001
    Iter 56/500 - Loss: 0.429   lengthscale: 14.410   noise: 0.001
    Iter 57/500 - Loss: 0.422   lengthscale: 14.649   noise: 0.001
    Iter 58/500 - Loss: 0.415   lengthscale: 14.883   noise: 0.001
    Iter 59/500 - Loss: 0.409   lengthscale: 15.111   noise: 0.001
    Iter 60/500 - Loss: 0.403   lengthscale: 15.335   noise: 0.001
    Iter 61/500 - Loss: 0.398   lengthscale: 15.552   noise: 0.001
    Iter 62/500 - Loss: 0.393   lengthscale: 15.764   noise: 0.001
    Iter 63/500 - Loss: 0.389   lengthscale: 15.970   noise: 0.001
    Iter 64/500 - Loss: 0.385   lengthscale: 16.170   noise: 0.000
    Iter 65/500 - Loss: 0.381   lengthscale: 16.363   noise: 0.000
    Iter 66/500 - Loss: 0.378   lengthscale: 16.551   noise: 0.000
    Iter 67/500 - Loss: 0.375   lengthscale: 16.731   noise: 0.000
    Iter 68/500 - Loss: 0.373   lengthscale: 16.905   noise: 0.000
    Iter 69/500 - Loss: 0.371   lengthscale: 17.072   noise: 0.000
    Iter 70/500 - Loss: 0.369   lengthscale: 17.231   noise: 0.000
    Iter 71/500 - Loss: 0.367   lengthscale: 17.383   noise: 0.000
    Iter 72/500 - Loss: 0.366   lengthscale: 17.527   noise: 0.000
    Iter 73/500 - Loss: 0.365   lengthscale: 17.662   noise: 0.000
    Iter 74/500 - Loss: 0.364   lengthscale: 17.791   noise: 0.000
    Iter 75/500 - Loss: 0.363   lengthscale: 17.911   noise: 0.000
    Iter 76/500 - Loss: 0.363   lengthscale: 18.024   noise: 0.000
    Iter 77/500 - Loss: 0.362   lengthscale: 18.129   noise: 0.000
    Iter 78/500 - Loss: 0.362   lengthscale: 18.227   noise: 0.000
    Iter 79/500 - Loss: 0.362   lengthscale: 18.317   noise: 0.000
    Iter 80/500 - Loss: 0.362   lengthscale: 18.400   noise: 0.000
    Iter 81/500 - Loss: 0.362   lengthscale: 18.477   noise: 0.000
    Iter 82/500 - Loss: 0.362   lengthscale: 18.546   noise: 0.000
    Iter 83/500 - Loss: 0.362   lengthscale: 18.608   noise: 0.000
    Iter 84/500 - Loss: 0.362   lengthscale: 18.664   noise: 0.000
    Iter 85/500 - Loss: 0.362   lengthscale: 18.712   noise: 0.000
    Iter 86/500 - Loss: 0.362   lengthscale: 18.755   noise: 0.000
    Iter 87/500 - Loss: 0.362   lengthscale: 18.791   noise: 0.000
    Iter 88/500 - Loss: 0.362   lengthscale: 18.822   noise: 0.000
    Iter 89/500 - Loss: 0.362   lengthscale: 18.848   noise: 0.000
    Iter 90/500 - Loss: 0.363   lengthscale: 18.868   noise: 0.000
    Iter 91/500 - Loss: 0.363   lengthscale: 18.884   noise: 0.000
    Iter 92/500 - Loss: 0.363   lengthscale: 18.896   noise: 0.000
    Iter 93/500 - Loss: 0.363   lengthscale: 18.904   noise: 0.000
    Iter 94/500 - Loss: 0.363   lengthscale: 18.909   noise: 0.000
    Iter 95/500 - Loss: 0.363   lengthscale: 18.910   noise: 0.000
    Iter 96/500 - Loss: 0.363   lengthscale: 18.907   noise: 0.000
    Iter 97/500 - Loss: 0.363   lengthscale: 18.902   noise: 0.000
    Iter 98/500 - Loss: 0.363   lengthscale: 18.894   noise: 0.000
    Iter 99/500 - Loss: 0.362   lengthscale: 18.884   noise: 0.000
    Iter 100/500 - Loss: 0.362   lengthscale: 18.873   noise: 0.000
    Iter 101/500 - Loss: 0.362   lengthscale: 18.859   noise: 0.000
    Iter 102/500 - Loss: 0.362   lengthscale: 18.844   noise: 0.000
    Iter 103/500 - Loss: 0.362   lengthscale: 18.828   noise: 0.000
    Iter 104/500 - Loss: 0.362   lengthscale: 18.811   noise: 0.000
    Iter 105/500 - Loss: 0.362   lengthscale: 18.794   noise: 0.000
    Iter 106/500 - Loss: 0.362   lengthscale: 18.776   noise: 0.000
    Iter 107/500 - Loss: 0.362   lengthscale: 18.758   noise: 0.000
    Iter 108/500 - Loss: 0.362   lengthscale: 18.740   noise: 0.000
    Iter 109/500 - Loss: 0.362   lengthscale: 18.722   noise: 0.000
    Iter 110/500 - Loss: 0.362   lengthscale: 18.704   noise: 0.000
    Iter 111/500 - Loss: 0.362   lengthscale: 18.687   noise: 0.000
    Iter 112/500 - Loss: 0.362   lengthscale: 18.670   noise: 0.000
    Iter 113/500 - Loss: 0.362   lengthscale: 18.653   noise: 0.000
    Iter 114/500 - Loss: 0.362   lengthscale: 18.638   noise: 0.000
    Iter 115/500 - Loss: 0.362   lengthscale: 18.622   noise: 0.000
    Iter 116/500 - Loss: 0.362   lengthscale: 18.608   noise: 0.000
    Iter 117/500 - Loss: 0.362   lengthscale: 18.595   noise: 0.000
    Iter 118/500 - Loss: 0.362   lengthscale: 18.583   noise: 0.000
    Iter 119/500 - Loss: 0.362   lengthscale: 18.572   noise: 0.000
    Iter 120/500 - Loss: 0.362   lengthscale: 18.561   noise: 0.000
    Iter 121/500 - Loss: 0.362   lengthscale: 18.552   noise: 0.000
    Iter 122/500 - Loss: 0.362   lengthscale: 18.543   noise: 0.000
    Iter 123/500 - Loss: 0.362   lengthscale: 18.536   noise: 0.000
    Iter 124/500 - Loss: 0.362   lengthscale: 18.529   noise: 0.000
    Iter 125/500 - Loss: 0.362   lengthscale: 18.524   noise: 0.000
    Iter 126/500 - Loss: 0.362   lengthscale: 18.519   noise: 0.000
    Iter 127/500 - Loss: 0.362   lengthscale: 18.516   noise: 0.000
    Iter 128/500 - Loss: 0.362   lengthscale: 18.513   noise: 0.000
    Iter 129/500 - Loss: 0.362   lengthscale: 18.511   noise: 0.000
    Iter 130/500 - Loss: 0.362   lengthscale: 18.510   noise: 0.000
    Iter 131/500 - Loss: 0.362   lengthscale: 18.509   noise: 0.000
    Iter 132/500 - Loss: 0.362   lengthscale: 18.510   noise: 0.000
    Iter 133/500 - Loss: 0.362   lengthscale: 18.511   noise: 0.000
    Iter 134/500 - Loss: 0.362   lengthscale: 18.512   noise: 0.000
    Iter 135/500 - Loss: 0.362   lengthscale: 18.514   noise: 0.000
    Iter 136/500 - Loss: 0.362   lengthscale: 18.517   noise: 0.000
    Iter 137/500 - Loss: 0.362   lengthscale: 18.520   noise: 0.000
    Iter 138/500 - Loss: 0.362   lengthscale: 18.524   noise: 0.000
    Iter 139/500 - Loss: 0.362   lengthscale: 18.528   noise: 0.000
    Iter 140/500 - Loss: 0.362   lengthscale: 18.532   noise: 0.000
    Iter 141/500 - Loss: 0.362   lengthscale: 18.537   noise: 0.000
    Iter 142/500 - Loss: 0.362   lengthscale: 18.542   noise: 0.000
    Iter 143/500 - Loss: 0.362   lengthscale: 18.547   noise: 0.000
    Iter 144/500 - Loss: 0.362   lengthscale: 18.552   noise: 0.000
    Iter 145/500 - Loss: 0.362   lengthscale: 18.558   noise: 0.000
    Iter 146/500 - Loss: 0.362   lengthscale: 18.563   noise: 0.000
    Iter 147/500 - Loss: 0.362   lengthscale: 18.569   noise: 0.000
    Iter 148/500 - Loss: 0.362   lengthscale: 18.574   noise: 0.000
    Iter 149/500 - Loss: 0.362   lengthscale: 18.580   noise: 0.000
    Iter 150/500 - Loss: 0.362   lengthscale: 18.586   noise: 0.000
    Iter 151/500 - Loss: 0.362   lengthscale: 18.592   noise: 0.000
    Iter 152/500 - Loss: 0.362   lengthscale: 18.597   noise: 0.000
    Iter 153/500 - Loss: 0.362   lengthscale: 18.603   noise: 0.000
    Iter 154/500 - Loss: 0.362   lengthscale: 18.608   noise: 0.000
    Iter 155/500 - Loss: 0.362   lengthscale: 18.614   noise: 0.000
    Iter 156/500 - Loss: 0.362   lengthscale: 18.619   noise: 0.000
    Iter 157/500 - Loss: 0.362   lengthscale: 18.625   noise: 0.000
    Iter 158/500 - Loss: 0.362   lengthscale: 18.630   noise: 0.000
    Iter 159/500 - Loss: 0.362   lengthscale: 18.635   noise: 0.000
    Iter 160/500 - Loss: 0.362   lengthscale: 18.640   noise: 0.000
    Iter 161/500 - Loss: 0.362   lengthscale: 18.645   noise: 0.000
    Iter 162/500 - Loss: 0.362   lengthscale: 18.650   noise: 0.000
    Iter 163/500 - Loss: 0.362   lengthscale: 18.655   noise: 0.000
    Iter 164/500 - Loss: 0.362   lengthscale: 18.659   noise: 0.000
    Iter 165/500 - Loss: 0.361   lengthscale: 18.664   noise: 0.000
    Iter 166/500 - Loss: 0.361   lengthscale: 18.669   noise: 0.000
    Iter 167/500 - Loss: 0.361   lengthscale: 18.673   noise: 0.000
    Iter 168/500 - Loss: 0.361   lengthscale: 18.678   noise: 0.000
    Iter 169/500 - Loss: 0.361   lengthscale: 18.682   noise: 0.000
    Iter 170/500 - Loss: 0.361   lengthscale: 18.686   noise: 0.000
    Iter 171/500 - Loss: 0.361   lengthscale: 18.691   noise: 0.000
    Iter 172/500 - Loss: 0.361   lengthscale: 18.695   noise: 0.000
    Iter 173/500 - Loss: 0.361   lengthscale: 18.699   noise: 0.000
    Iter 174/500 - Loss: 0.361   lengthscale: 18.704   noise: 0.000
    Iter 175/500 - Loss: 0.361   lengthscale: 18.708   noise: 0.000
    Iter 176/500 - Loss: 0.361   lengthscale: 18.713   noise: 0.000
    Iter 177/500 - Loss: 0.361   lengthscale: 18.717   noise: 0.000
    Iter 178/500 - Loss: 0.361   lengthscale: 18.721   noise: 0.000
    Iter 179/500 - Loss: 0.361   lengthscale: 18.726   noise: 0.000
    Iter 180/500 - Loss: 0.361   lengthscale: 18.731   noise: 0.000
    Iter 181/500 - Loss: 0.361   lengthscale: 18.735   noise: 0.000
    Iter 182/500 - Loss: 0.361   lengthscale: 18.740   noise: 0.000
    Iter 183/500 - Loss: 0.361   lengthscale: 18.745   noise: 0.000
    Iter 184/500 - Loss: 0.361   lengthscale: 18.750   noise: 0.000
    Iter 185/500 - Loss: 0.361   lengthscale: 18.755   noise: 0.000
    Iter 186/500 - Loss: 0.361   lengthscale: 18.760   noise: 0.000
    Iter 187/500 - Loss: 0.361   lengthscale: 18.765   noise: 0.000
    Iter 188/500 - Loss: 0.361   lengthscale: 18.771   noise: 0.000
    Iter 189/500 - Loss: 0.361   lengthscale: 18.776   noise: 0.000
    Iter 190/500 - Loss: 0.361   lengthscale: 18.782   noise: 0.000
    Iter 191/500 - Loss: 0.361   lengthscale: 18.788   noise: 0.000
    Iter 192/500 - Loss: 0.361   lengthscale: 18.794   noise: 0.000
    Iter 193/500 - Loss: 0.361   lengthscale: 18.800   noise: 0.000
    Iter 194/500 - Loss: 0.361   lengthscale: 18.806   noise: 0.000
    Iter 195/500 - Loss: 0.361   lengthscale: 18.813   noise: 0.000
    Iter 196/500 - Loss: 0.361   lengthscale: 18.820   noise: 0.000
    Iter 197/500 - Loss: 0.361   lengthscale: 18.827   noise: 0.000
    Iter 198/500 - Loss: 0.361   lengthscale: 18.834   noise: 0.000
    Iter 199/500 - Loss: 0.361   lengthscale: 18.841   noise: 0.000
    Iter 200/500 - Loss: 0.361   lengthscale: 18.848   noise: 0.000
    Iter 201/500 - Loss: 0.361   lengthscale: 18.856   noise: 0.000
    Iter 202/500 - Loss: 0.361   lengthscale: 18.863   noise: 0.000
    Iter 203/500 - Loss: 0.361   lengthscale: 18.871   noise: 0.000
    Iter 204/500 - Loss: 0.361   lengthscale: 18.880   noise: 0.000
    Iter 205/500 - Loss: 0.361   lengthscale: 18.888   noise: 0.000
    Iter 206/500 - Loss: 0.361   lengthscale: 18.897   noise: 0.001
    Iter 207/500 - Loss: 0.361   lengthscale: 18.905   noise: 0.001
    Iter 208/500 - Loss: 0.361   lengthscale: 18.914   noise: 0.001
    Iter 209/500 - Loss: 0.361   lengthscale: 18.924   noise: 0.001
    Iter 210/500 - Loss: 0.361   lengthscale: 18.933   noise: 0.001
    Iter 211/500 - Loss: 0.361   lengthscale: 18.943   noise: 0.001
    Iter 212/500 - Loss: 0.361   lengthscale: 18.953   noise: 0.001
    Iter 213/500 - Loss: 0.361   lengthscale: 18.963   noise: 0.001
    Iter 214/500 - Loss: 0.361   lengthscale: 18.974   noise: 0.001
    Iter 215/500 - Loss: 0.361   lengthscale: 18.985   noise: 0.001
    Iter 216/500 - Loss: 0.361   lengthscale: 18.996   noise: 0.001
    Iter 217/500 - Loss: 0.361   lengthscale: 19.007   noise: 0.001
    Iter 218/500 - Loss: 0.361   lengthscale: 19.019   noise: 0.001
    Iter 219/500 - Loss: 0.361   lengthscale: 19.030   noise: 0.001
    Iter 220/500 - Loss: 0.361   lengthscale: 19.043   noise: 0.001
    Iter 221/500 - Loss: 0.360   lengthscale: 19.055   noise: 0.001
    Iter 222/500 - Loss: 0.360   lengthscale: 19.068   noise: 0.001
    Iter 223/500 - Loss: 0.360   lengthscale: 19.081   noise: 0.001
    Iter 224/500 - Loss: 0.360   lengthscale: 19.095   noise: 0.001
    Iter 225/500 - Loss: 0.360   lengthscale: 19.109   noise: 0.001
    Iter 226/500 - Loss: 0.360   lengthscale: 19.123   noise: 0.001
    Iter 227/500 - Loss: 0.360   lengthscale: 19.138   noise: 0.001
    Iter 228/500 - Loss: 0.360   lengthscale: 19.153   noise: 0.001
    Iter 229/500 - Loss: 0.360   lengthscale: 19.169   noise: 0.001
    Iter 230/500 - Loss: 0.360   lengthscale: 19.185   noise: 0.001
    Iter 231/500 - Loss: 0.360   lengthscale: 19.202   noise: 0.001
    Iter 232/500 - Loss: 0.360   lengthscale: 19.219   noise: 0.001
    Iter 233/500 - Loss: 0.360   lengthscale: 19.236   noise: 0.001
    Iter 234/500 - Loss: 0.360   lengthscale: 19.254   noise: 0.001
    Iter 235/500 - Loss: 0.360   lengthscale: 19.273   noise: 0.001
    Iter 236/500 - Loss: 0.360   lengthscale: 19.292   noise: 0.001
    Iter 237/500 - Loss: 0.360   lengthscale: 19.311   noise: 0.001
    Iter 238/500 - Loss: 0.359   lengthscale: 19.332   noise: 0.001
    Iter 239/500 - Loss: 0.359   lengthscale: 19.353   noise: 0.001
    Iter 240/500 - Loss: 0.359   lengthscale: 19.374   noise: 0.001
    Iter 241/500 - Loss: 0.359   lengthscale: 19.396   noise: 0.001
    Iter 242/500 - Loss: 0.359   lengthscale: 19.419   noise: 0.001
    Iter 243/500 - Loss: 0.359   lengthscale: 19.443   noise: 0.001
    Iter 244/500 - Loss: 0.359   lengthscale: 19.467   noise: 0.001
    Iter 245/500 - Loss: 0.359   lengthscale: 19.492   noise: 0.001
    Iter 246/500 - Loss: 0.359   lengthscale: 19.518   noise: 0.001
    Iter 247/500 - Loss: 0.358   lengthscale: 19.545   noise: 0.001
    Iter 248/500 - Loss: 0.358   lengthscale: 19.572   noise: 0.001
    Iter 249/500 - Loss: 0.358   lengthscale: 19.600   noise: 0.001
    Iter 250/500 - Loss: 0.358   lengthscale: 19.630   noise: 0.001
    Iter 251/500 - Loss: 0.358   lengthscale: 19.660   noise: 0.001
    Iter 252/500 - Loss: 0.358   lengthscale: 19.690   noise: 0.001
    Iter 253/500 - Loss: 0.358   lengthscale: 19.722   noise: 0.001
    Iter 254/500 - Loss: 0.357   lengthscale: 19.755   noise: 0.001
    Iter 255/500 - Loss: 0.357   lengthscale: 19.789   noise: 0.001
    Iter 256/500 - Loss: 0.357   lengthscale: 19.823   noise: 0.001
    Iter 257/500 - Loss: 0.357   lengthscale: 19.859   noise: 0.001
    Iter 258/500 - Loss: 0.357   lengthscale: 19.896   noise: 0.001
    Iter 259/500 - Loss: 0.357   lengthscale: 19.933   noise: 0.001
    Iter 260/500 - Loss: 0.356   lengthscale: 19.972   noise: 0.001
    Iter 261/500 - Loss: 0.356   lengthscale: 20.012   noise: 0.001
    Iter 262/500 - Loss: 0.356   lengthscale: 20.052   noise: 0.001
    Iter 263/500 - Loss: 0.356   lengthscale: 20.094   noise: 0.002
    Iter 264/500 - Loss: 0.356   lengthscale: 20.136   noise: 0.002
    Iter 265/500 - Loss: 0.355   lengthscale: 20.179   noise: 0.002
    Iter 266/500 - Loss: 0.355   lengthscale: 20.224   noise: 0.002
    Iter 267/500 - Loss: 0.355   lengthscale: 20.269   noise: 0.002
    Iter 268/500 - Loss: 0.355   lengthscale: 20.315   noise: 0.002
    Iter 269/500 - Loss: 0.355   lengthscale: 20.362   noise: 0.002
    Iter 270/500 - Loss: 0.354   lengthscale: 20.409   noise: 0.002
    Iter 271/500 - Loss: 0.354   lengthscale: 20.458   noise: 0.002
    Iter 272/500 - Loss: 0.354   lengthscale: 20.507   noise: 0.002
    Iter 273/500 - Loss: 0.354   lengthscale: 20.557   noise: 0.002
    Iter 274/500 - Loss: 0.354   lengthscale: 20.607   noise: 0.002
    Iter 275/500 - Loss: 0.353   lengthscale: 20.658   noise: 0.002
    Iter 276/500 - Loss: 0.353   lengthscale: 20.709   noise: 0.002
    Iter 277/500 - Loss: 0.353   lengthscale: 20.761   noise: 0.002
    Iter 278/500 - Loss: 0.353   lengthscale: 20.813   noise: 0.002
    Iter 279/500 - Loss: 0.353   lengthscale: 20.866   noise: 0.002
    Iter 280/500 - Loss: 0.352   lengthscale: 20.919   noise: 0.002
    Iter 281/500 - Loss: 0.352   lengthscale: 20.972   noise: 0.002
    Iter 282/500 - Loss: 0.352   lengthscale: 21.025   noise: 0.002
    Iter 283/500 - Loss: 0.352   lengthscale: 21.078   noise: 0.002
    Iter 284/500 - Loss: 0.352   lengthscale: 21.132   noise: 0.002
    Iter 285/500 - Loss: 0.352   lengthscale: 21.185   noise: 0.002
    Iter 286/500 - Loss: 0.351   lengthscale: 21.239   noise: 0.003
    Iter 287/500 - Loss: 0.351   lengthscale: 21.292   noise: 0.003
    Iter 288/500 - Loss: 0.351   lengthscale: 21.345   noise: 0.003
    Iter 289/500 - Loss: 0.351   lengthscale: 21.398   noise: 0.003
    Iter 290/500 - Loss: 0.351   lengthscale: 21.451   noise: 0.003
    Iter 291/500 - Loss: 0.350   lengthscale: 21.504   noise: 0.003
    Iter 292/500 - Loss: 0.350   lengthscale: 21.556   noise: 0.003
    Iter 293/500 - Loss: 0.350   lengthscale: 21.608   noise: 0.003
    Iter 294/500 - Loss: 0.350   lengthscale: 21.661   noise: 0.003
    Iter 295/500 - Loss: 0.350   lengthscale: 21.712   noise: 0.003
    Iter 296/500 - Loss: 0.350   lengthscale: 21.764   noise: 0.003
    Iter 297/500 - Loss: 0.349   lengthscale: 21.815   noise: 0.003
    Iter 298/500 - Loss: 0.349   lengthscale: 21.867   noise: 0.003
    Iter 299/500 - Loss: 0.349   lengthscale: 21.918   noise: 0.003
    Iter 300/500 - Loss: 0.349   lengthscale: 21.968   noise: 0.003
    Iter 301/500 - Loss: 0.349   lengthscale: 22.019   noise: 0.003
    Iter 302/500 - Loss: 0.349   lengthscale: 22.070   noise: 0.003
    Iter 303/500 - Loss: 0.348   lengthscale: 22.120   noise: 0.003
    Iter 304/500 - Loss: 0.348   lengthscale: 22.170   noise: 0.003
    Iter 305/500 - Loss: 0.348   lengthscale: 22.220   noise: 0.003
    Iter 306/500 - Loss: 0.348   lengthscale: 22.270   noise: 0.003
    Iter 307/500 - Loss: 0.348   lengthscale: 22.320   noise: 0.003
    Iter 308/500 - Loss: 0.348   lengthscale: 22.370   noise: 0.003
    Iter 309/500 - Loss: 0.347   lengthscale: 22.419   noise: 0.004
    Iter 310/500 - Loss: 0.347   lengthscale: 22.469   noise: 0.004
    Iter 311/500 - Loss: 0.347   lengthscale: 22.519   noise: 0.004
    Iter 312/500 - Loss: 0.347   lengthscale: 22.568   noise: 0.004
    Iter 313/500 - Loss: 0.347   lengthscale: 22.618   noise: 0.004
    Iter 314/500 - Loss: 0.347   lengthscale: 22.668   noise: 0.004
    Iter 315/500 - Loss: 0.347   lengthscale: 22.717   noise: 0.004
    Iter 316/500 - Loss: 0.346   lengthscale: 22.767   noise: 0.004
    Iter 317/500 - Loss: 0.346   lengthscale: 22.817   noise: 0.004
    Iter 318/500 - Loss: 0.346   lengthscale: 22.867   noise: 0.004
    Iter 319/500 - Loss: 0.346   lengthscale: 22.916   noise: 0.004
    Iter 320/500 - Loss: 0.346   lengthscale: 22.966   noise: 0.004
    Iter 321/500 - Loss: 0.346   lengthscale: 23.016   noise: 0.004
    Iter 322/500 - Loss: 0.346   lengthscale: 23.066   noise: 0.004
    Iter 323/500 - Loss: 0.345   lengthscale: 23.116   noise: 0.004
    Iter 324/500 - Loss: 0.345   lengthscale: 23.166   noise: 0.004
    Iter 325/500 - Loss: 0.345   lengthscale: 23.217   noise: 0.004
    Iter 326/500 - Loss: 0.345   lengthscale: 23.267   noise: 0.004
    Iter 327/500 - Loss: 0.345   lengthscale: 23.318   noise: 0.004
    Iter 328/500 - Loss: 0.345   lengthscale: 23.369   noise: 0.004
    Iter 329/500 - Loss: 0.344   lengthscale: 23.420   noise: 0.005
    Iter 330/500 - Loss: 0.344   lengthscale: 23.471   noise: 0.005
    Iter 331/500 - Loss: 0.344   lengthscale: 23.523   noise: 0.005
    Iter 332/500 - Loss: 0.344   lengthscale: 23.574   noise: 0.005
    Iter 333/500 - Loss: 0.344   lengthscale: 23.626   noise: 0.005
    Iter 334/500 - Loss: 0.344   lengthscale: 23.679   noise: 0.005
    Iter 335/500 - Loss: 0.343   lengthscale: 23.731   noise: 0.005
    Iter 336/500 - Loss: 0.343   lengthscale: 23.784   noise: 0.005
    Iter 337/500 - Loss: 0.343   lengthscale: 23.837   noise: 0.005
    Iter 338/500 - Loss: 0.343   lengthscale: 23.891   noise: 0.005
    Iter 339/500 - Loss: 0.343   lengthscale: 23.945   noise: 0.005
    Iter 340/500 - Loss: 0.343   lengthscale: 23.999   noise: 0.005
    Iter 341/500 - Loss: 0.342   lengthscale: 24.053   noise: 0.005
    Iter 342/500 - Loss: 0.342   lengthscale: 24.108   noise: 0.005
    Iter 343/500 - Loss: 0.342   lengthscale: 24.164   noise: 0.005
    Iter 344/500 - Loss: 0.342   lengthscale: 24.219   noise: 0.005
    Iter 345/500 - Loss: 0.342   lengthscale: 24.276   noise: 0.005
    Iter 346/500 - Loss: 0.341   lengthscale: 24.332   noise: 0.005
    Iter 347/500 - Loss: 0.341   lengthscale: 24.389   noise: 0.006
    Iter 348/500 - Loss: 0.341   lengthscale: 24.447   noise: 0.006
    Iter 349/500 - Loss: 0.341   lengthscale: 24.505   noise: 0.006
    Iter 350/500 - Loss: 0.341   lengthscale: 24.563   noise: 0.006
    Iter 351/500 - Loss: 0.340   lengthscale: 24.622   noise: 0.006
    Iter 352/500 - Loss: 0.340   lengthscale: 24.681   noise: 0.006
    Iter 353/500 - Loss: 0.340   lengthscale: 24.741   noise: 0.006
    Iter 354/500 - Loss: 0.340   lengthscale: 24.801   noise: 0.006
    Iter 355/500 - Loss: 0.340   lengthscale: 24.862   noise: 0.006
    Iter 356/500 - Loss: 0.339   lengthscale: 24.924   noise: 0.006
    Iter 357/500 - Loss: 0.339   lengthscale: 24.985   noise: 0.006
    Iter 358/500 - Loss: 0.339   lengthscale: 25.048   noise: 0.006
    Iter 359/500 - Loss: 0.339   lengthscale: 25.111   noise: 0.006
    Iter 360/500 - Loss: 0.338   lengthscale: 25.174   noise: 0.006
    Iter 361/500 - Loss: 0.338   lengthscale: 25.239   noise: 0.006
    Iter 362/500 - Loss: 0.338   lengthscale: 25.303   noise: 0.006
    Iter 363/500 - Loss: 0.338   lengthscale: 25.369   noise: 0.007
    Iter 364/500 - Loss: 0.337   lengthscale: 25.435   noise: 0.007
    Iter 365/500 - Loss: 0.337   lengthscale: 25.501   noise: 0.007
    Iter 366/500 - Loss: 0.337   lengthscale: 25.568   noise: 0.007
    Iter 367/500 - Loss: 0.337   lengthscale: 25.636   noise: 0.007
    Iter 368/500 - Loss: 0.336   lengthscale: 25.704   noise: 0.007
    Iter 369/500 - Loss: 0.336   lengthscale: 25.773   noise: 0.007
    Iter 370/500 - Loss: 0.336   lengthscale: 25.842   noise: 0.007
    Iter 371/500 - Loss: 0.335   lengthscale: 25.913   noise: 0.007
    Iter 372/500 - Loss: 0.335   lengthscale: 25.983   noise: 0.007
    Iter 373/500 - Loss: 0.335   lengthscale: 26.055   noise: 0.007
    Iter 374/500 - Loss: 0.335   lengthscale: 26.127   noise: 0.007
    Iter 375/500 - Loss: 0.334   lengthscale: 26.199   noise: 0.007
    Iter 376/500 - Loss: 0.334   lengthscale: 26.273   noise: 0.007
    Iter 377/500 - Loss: 0.334   lengthscale: 26.347   noise: 0.007
    Iter 378/500 - Loss: 0.333   lengthscale: 26.421   noise: 0.008
    Iter 379/500 - Loss: 0.333   lengthscale: 26.496   noise: 0.008
    Iter 380/500 - Loss: 0.333   lengthscale: 26.572   noise: 0.008
    Iter 381/500 - Loss: 0.332   lengthscale: 26.649   noise: 0.008
    Iter 382/500 - Loss: 0.332   lengthscale: 26.726   noise: 0.008
    Iter 383/500 - Loss: 0.332   lengthscale: 26.804   noise: 0.008
    Iter 384/500 - Loss: 0.331   lengthscale: 26.882   noise: 0.008
    Iter 385/500 - Loss: 0.331   lengthscale: 26.961   noise: 0.008
    Iter 386/500 - Loss: 0.331   lengthscale: 27.041   noise: 0.008
    Iter 387/500 - Loss: 0.330   lengthscale: 27.121   noise: 0.008
    Iter 388/500 - Loss: 0.330   lengthscale: 27.202   noise: 0.008
    Iter 389/500 - Loss: 0.330   lengthscale: 27.284   noise: 0.008
    Iter 390/500 - Loss: 0.329   lengthscale: 27.366   noise: 0.008
    Iter 391/500 - Loss: 0.329   lengthscale: 27.449   noise: 0.008
    Iter 392/500 - Loss: 0.328   lengthscale: 27.532   noise: 0.008
    Iter 393/500 - Loss: 0.328   lengthscale: 27.616   noise: 0.009
    Iter 394/500 - Loss: 0.328   lengthscale: 27.701   noise: 0.009
    Iter 395/500 - Loss: 0.327   lengthscale: 27.786   noise: 0.009
    Iter 396/500 - Loss: 0.327   lengthscale: 27.872   noise: 0.009
    Iter 397/500 - Loss: 0.326   lengthscale: 27.959   noise: 0.009
    Iter 398/500 - Loss: 0.326   lengthscale: 28.046   noise: 0.009
    Iter 399/500 - Loss: 0.326   lengthscale: 28.133   noise: 0.009
    Iter 400/500 - Loss: 0.325   lengthscale: 28.222   noise: 0.009
    Iter 401/500 - Loss: 0.325   lengthscale: 28.311   noise: 0.009
    Iter 402/500 - Loss: 0.324   lengthscale: 28.400   noise: 0.009
    Iter 403/500 - Loss: 0.324   lengthscale: 28.490   noise: 0.009
    Iter 404/500 - Loss: 0.323   lengthscale: 28.580   noise: 0.009
    Iter 405/500 - Loss: 0.323   lengthscale: 28.672   noise: 0.009
    Iter 406/500 - Loss: 0.323   lengthscale: 28.763   noise: 0.009
    Iter 407/500 - Loss: 0.322   lengthscale: 28.855   noise: 0.009
    Iter 408/500 - Loss: 0.322   lengthscale: 28.948   noise: 0.010
    Iter 409/500 - Loss: 0.321   lengthscale: 29.041   noise: 0.010
    Iter 410/500 - Loss: 0.321   lengthscale: 29.135   noise: 0.010
    Iter 411/500 - Loss: 0.320   lengthscale: 29.229   noise: 0.010
    Iter 412/500 - Loss: 0.320   lengthscale: 29.324   noise: 0.010
    Iter 413/500 - Loss: 0.319   lengthscale: 29.419   noise: 0.010
    Iter 414/500 - Loss: 0.319   lengthscale: 29.515   noise: 0.010
    Iter 415/500 - Loss: 0.318   lengthscale: 29.611   noise: 0.010
    Iter 416/500 - Loss: 0.318   lengthscale: 29.708   noise: 0.010
    Iter 417/500 - Loss: 0.317   lengthscale: 29.805   noise: 0.010
    Iter 418/500 - Loss: 0.317   lengthscale: 29.903   noise: 0.010
    Iter 419/500 - Loss: 0.316   lengthscale: 30.001   noise: 0.010
    Iter 420/500 - Loss: 0.316   lengthscale: 30.099   noise: 0.010
    Iter 421/500 - Loss: 0.315   lengthscale: 30.198   noise: 0.010
    Iter 422/500 - Loss: 0.315   lengthscale: 30.297   noise: 0.010
    Iter 423/500 - Loss: 0.314   lengthscale: 30.397   noise: 0.010
    Iter 424/500 - Loss: 0.314   lengthscale: 30.497   noise: 0.010
    Iter 425/500 - Loss: 0.313   lengthscale: 30.598   noise: 0.011
    Iter 426/500 - Loss: 0.313   lengthscale: 30.699   noise: 0.011
    Iter 427/500 - Loss: 0.312   lengthscale: 30.800   noise: 0.011
    Iter 428/500 - Loss: 0.312   lengthscale: 30.901   noise: 0.011
    Iter 429/500 - Loss: 0.311   lengthscale: 31.003   noise: 0.011
    Iter 430/500 - Loss: 0.311   lengthscale: 31.106   noise: 0.011
    Iter 431/500 - Loss: 0.310   lengthscale: 31.209   noise: 0.011
    Iter 432/500 - Loss: 0.310   lengthscale: 31.312   noise: 0.011
    Iter 433/500 - Loss: 0.309   lengthscale: 31.415   noise: 0.011
    Iter 434/500 - Loss: 0.309   lengthscale: 31.519   noise: 0.011
    Iter 435/500 - Loss: 0.308   lengthscale: 31.623   noise: 0.011
    Iter 436/500 - Loss: 0.308   lengthscale: 31.727   noise: 0.011
    Iter 437/500 - Loss: 0.307   lengthscale: 31.832   noise: 0.011
    Iter 438/500 - Loss: 0.306   lengthscale: 31.936   noise: 0.011
    Iter 439/500 - Loss: 0.306   lengthscale: 32.042   noise: 0.011
    Iter 440/500 - Loss: 0.305   lengthscale: 32.147   noise: 0.011
    Iter 441/500 - Loss: 0.305   lengthscale: 32.253   noise: 0.011
    Iter 442/500 - Loss: 0.304   lengthscale: 32.359   noise: 0.011
    Iter 443/500 - Loss: 0.304   lengthscale: 32.465   noise: 0.011
    Iter 444/500 - Loss: 0.303   lengthscale: 32.571   noise: 0.012
    Iter 445/500 - Loss: 0.303   lengthscale: 32.678   noise: 0.012
    Iter 446/500 - Loss: 0.302   lengthscale: 32.785   noise: 0.012
    Iter 447/500 - Loss: 0.301   lengthscale: 32.892   noise: 0.012
    Iter 448/500 - Loss: 0.301   lengthscale: 32.999   noise: 0.012
    Iter 449/500 - Loss: 0.300   lengthscale: 33.106   noise: 0.012
    Iter 450/500 - Loss: 0.300   lengthscale: 33.214   noise: 0.012
    Iter 451/500 - Loss: 0.299   lengthscale: 33.322   noise: 0.012
    Iter 452/500 - Loss: 0.299   lengthscale: 33.430   noise: 0.012
    Iter 453/500 - Loss: 0.298   lengthscale: 33.538   noise: 0.012
    Iter 454/500 - Loss: 0.298   lengthscale: 33.646   noise: 0.012
    Iter 455/500 - Loss: 0.297   lengthscale: 33.755   noise: 0.012
    Iter 456/500 - Loss: 0.296   lengthscale: 33.863   noise: 0.012
    Iter 457/500 - Loss: 0.296   lengthscale: 33.972   noise: 0.012
    Iter 458/500 - Loss: 0.295   lengthscale: 34.081   noise: 0.012
    Iter 459/500 - Loss: 0.295   lengthscale: 34.190   noise: 0.012
    Iter 460/500 - Loss: 0.294   lengthscale: 34.299   noise: 0.012
    Iter 461/500 - Loss: 0.294   lengthscale: 34.408   noise: 0.012
    Iter 462/500 - Loss: 0.293   lengthscale: 34.517   noise: 0.012
    Iter 463/500 - Loss: 0.292   lengthscale: 34.626   noise: 0.012
    Iter 464/500 - Loss: 0.292   lengthscale: 34.736   noise: 0.012
    Iter 465/500 - Loss: 0.291   lengthscale: 34.845   noise: 0.012
    Iter 466/500 - Loss: 0.291   lengthscale: 34.955   noise: 0.012
    Iter 467/500 - Loss: 0.290   lengthscale: 35.064   noise: 0.012
    Iter 468/500 - Loss: 0.290   lengthscale: 35.174   noise: 0.012
    Iter 469/500 - Loss: 0.289   lengthscale: 35.284   noise: 0.013
    Iter 470/500 - Loss: 0.289   lengthscale: 35.394   noise: 0.013
    Iter 471/500 - Loss: 0.288   lengthscale: 35.503   noise: 0.013
    Iter 472/500 - Loss: 0.287   lengthscale: 35.613   noise: 0.013
    Iter 473/500 - Loss: 0.287   lengthscale: 35.723   noise: 0.013
    Iter 474/500 - Loss: 0.286   lengthscale: 35.833   noise: 0.013
    Iter 475/500 - Loss: 0.286   lengthscale: 35.943   noise: 0.013
    Iter 476/500 - Loss: 0.285   lengthscale: 36.052   noise: 0.013
    Iter 477/500 - Loss: 0.285   lengthscale: 36.162   noise: 0.013
    Iter 478/500 - Loss: 0.284   lengthscale: 36.272   noise: 0.013
    Iter 479/500 - Loss: 0.284   lengthscale: 36.382   noise: 0.013
    Iter 480/500 - Loss: 0.283   lengthscale: 36.491   noise: 0.013
    Iter 481/500 - Loss: 0.282   lengthscale: 36.601   noise: 0.013
    Iter 482/500 - Loss: 0.282   lengthscale: 36.711   noise: 0.013
    Iter 483/500 - Loss: 0.281   lengthscale: 36.821   noise: 0.013
    Iter 484/500 - Loss: 0.281   lengthscale: 36.930   noise: 0.013
    Iter 485/500 - Loss: 0.280   lengthscale: 37.040   noise: 0.013
    Iter 486/500 - Loss: 0.280   lengthscale: 37.149   noise: 0.013
    Iter 487/500 - Loss: 0.279   lengthscale: 37.259   noise: 0.013
    Iter 488/500 - Loss: 0.279   lengthscale: 37.368   noise: 0.013
    Iter 489/500 - Loss: 0.278   lengthscale: 37.477   noise: 0.013
    Iter 490/500 - Loss: 0.278   lengthscale: 37.586   noise: 0.013
    Iter 491/500 - Loss: 0.277   lengthscale: 37.695   noise: 0.013
    Iter 492/500 - Loss: 0.276   lengthscale: 37.804   noise: 0.013
    Iter 493/500 - Loss: 0.276   lengthscale: 37.913   noise: 0.013
    Iter 494/500 - Loss: 0.275   lengthscale: 38.022   noise: 0.013
    Iter 495/500 - Loss: 0.275   lengthscale: 38.131   noise: 0.013
    Iter 496/500 - Loss: 0.274   lengthscale: 38.239   noise: 0.013
    Iter 497/500 - Loss: 0.274   lengthscale: 38.348   noise: 0.013
    Iter 498/500 - Loss: 0.273   lengthscale: 38.456   noise: 0.013
    Iter 499/500 - Loss: 0.273   lengthscale: 38.564   noise: 0.013
    Iter 500/500 - Loss: 0.272   lengthscale: 38.672   noise: 0.013


Now, we test predictions. For simplicity, we will assume a fixed variance of $0.01.$


```python
# Get into evaluation (predictive posterior) mode
model.eval()
likelihood.eval()

# Test points are regularly spaced along [0,1]
# Make predictions by feeding model through likelihood
with torch.no_grad(), gpytorch.settings.fast_pred_var():
    test_x = torch.linspace(0, 1, 51)
    test_x_distributional = torch.stack((test_x, (1e-2 * torch.ones_like(test_x)).log()), dim=1)
    observed_pred = likelihood(model(test_x_distributional))

with torch.no_grad():
    # Initialize plot
    f, ax = plt.subplots(1, 1, figsize=(8, 3))

    # Get upper and lower confidence bounds
    lower, upper = observed_pred.confidence_region()
    # Plot training data as black stars
    ax.errorbar(train_x_mean.numpy(), train_y.numpy(), xerr=train_x_stdv, fmt='k*')
    # Plot predictive means as blue line
    ax.plot(test_x.numpy(), observed_pred.mean.numpy(), 'b')
    # Shade between the lower and upper confidence bounds
    ax.fill_between(test_x.numpy(), lower.numpy(), upper.numpy(), alpha=0.5)
    ax.set_ylim([-3, 3])
    ax.legend(['Observed Data', 'Mean', 'Confidence'])
```

As a final note, we've made it very easy to extend the distributional kernel class by exposing a generic `DistributionalInputKernel` class that takes as input any distance function over probability distributions.


```python

```

37158f1af618a5c61dce505a953c1f30b5e7a93d
ipynb
Jupyter Notebook
examples/01_Exact_GPs/GP_Regression_DistributionalKernel.ipynb
llguo95/gpytorch
1fa69935104565c377ce95d2c581c9eedfb55817
MIT
2018-02-19T22:28:58.000Z
2022-03-31T13:22:28.000Z
examples/01_Exact_GPs/GP_Regression_DistributionalKernel.ipynb
llguo95/gpytorch
1fa69935104565c377ce95d2c581c9eedfb55817
MIT
2018-02-19T20:38:20.000Z
2022-03-30T12:53:13.000Z
examples/01_Exact_GPs/GP_Regression_DistributionalKernel.ipynb
llguo95/gpytorch
1fa69935104565c377ce95d2c581c9eedfb55817
MIT
2018-03-07T02:06:05.000Z
2022-03-27T07:05:44.000Z
Qwen/Qwen-72B
1. YES
2. YES