address:
- |
    M. L. Stein\
    Department of Statistics\
    University of Chicago\
    Chicago, Illinois 60637\
    USA\
- |
    J. Chen\
    M. Anitescu\
    Mathematics and Computer\
    Science Division\
    Argonne National Laboratory\
    Argonne, Illinois 60439\
    USA\
    \
author:
- 
- 
- 
title: Stochastic approximation of score functions for Gaussian processes
---

,

Introduction {#sec1}
============

Gaussian process models are widely used in spatial statistics and machine learning. In most applications, the covariance structure of the process is at least partially unknown and must be estimated from the available data. Likelihood-based methods, including Bayesian methods, are natural choices for carrying out the inferences on the unknown covariance structure. For large data sets, however, calculating the likelihood function exactly may be difficult or impossible in many cases.

Assuming we are willing to specify the covariance structure up to some parameter $\theta\in\Theta\subset{{\mathbb R}}^p$, the generic problem we are faced with is computing the loglikelihood for $Z\sim N(0,K(\theta))$ for some random vector $Z\in{{\mathbb R}}^n$ and $K$ an $n\times n$ positive definite matrix indexed by the unknown $\theta$. In many applications, there would be a mean vector that also depends on unknown parameters, but since unknown mean parameters generally cause fewer computational difficulties, for simplicity we will 