The description of the method is somewhat unclear and it is hard to understand all the design choices.
Some natural baselines and important related work seem to be missing.
Major comments:
The lack of flexibility of standard GPs is not a new observation as has been approached in the past, possibly most famously by the deep GP [1]. These models have recently become a mainstream tool with easily usable frameworks [e.g., 2], so that it would seems like a natural baseline to compare against.
Generally, a lot of related work seems to be missed by this paper. For instance, meta-learning kernels for GPs for few-shot tasks has already been done by [3] and then later also by [4,5,6]. These should probably be mentioned and it should be discussed how the proposed method compares against them.
The paper proposes to use CNFs, but these require solving a complex-looking integral (e.g., Eq. 9). It should be discussed how tractable this integral is or how it is approximated in practice. Moreover, it seems like an easier choice would be standard NFs, so it should be discussed why CNFs are assumed to be better here. Possibly, one should also directly compare against a model with a standard NF as an ablation study.
In l. 257ff it is claimed that the proposed GP methods are less prone to memorization. How does this compare to the results in [4], where DKT seems to memorize as well? Could the regularization proposed in [4] be combined with the proposed model?
Minor comments:
In l. 104 it is said that every kernel can be described by a feature space parameterized by a neural network, but this is trivially not true. For instance, for RBF kernels, the RKHS is famously infinite-dimensional, such that one would need an NN with infinite width to represent it. So at most, NNs can represent finite-dimensional RKHSs in practice. This limitation should be made more clear.
l. 151 with GP -> with a GP
l. 152 use invertible mapping -> use an invertible mapping
l. 161 the "marginal log-probability" is more commonly called "log marginal likelihood" or "log evidence"
Eq. (8): should it be z
instead of y ?
In the tables, it would be more helpful to also bolden the fonts of the entries where the error bars overlap with the best entry.
[1] Damianou & Lawrence 2012, https://arxiv.org/abs/1211.0358
[2] Dutordoir et al. 2021, https://arxiv.org/abs/2104.05674
[3] Fortuin et al. 2019, https://arxiv.org/abs/1901.08098
[4] Rothfuss et al. 2020, https://arxiv.org/abs/2002.05551
[5] Venkitaraman et al. 2020, https://arxiv.org/abs/2006.07212
[6] Titsias et al. 2020, https://arxiv.org/abs/2009.03228
The limitations of the method are hard to assess, mostly because the choice of CNFs over NFs or any other flexible distribution family is not well motivated and because (theoretical and empirical) comparisons to many relevant related methods are missing. This should be addressed.
104 it is said that every kernel can be described by a feature space parameterized by a neural network, but this is trivially not true. For instance, for RBF kernels, the RKHS is famously infinite-dimensional, such that one would need an NN with infinite width to represent it. So at most, NNs can represent finite-dimensional RKHSs in practice. This limitation should be made more clear. l.