There are two main (and easily fixable) weaknesses.
a) I think the role of the normalizing flow is underexplained. It is stated multiple times that the normalizing flow provides the evidence updates and its purpose is to estimate epistemic uncertainty. The remaining questions for me are 1. From which space to which does the NF map the latent variable z? 2. Why is the arrow in Figure 2 from a Gaussian space into the latent space, rather than from the latent space to n^(i)? I thought the main purpose was to influence n^(i)? 3. Which experiments show that the normalizing flow contributes meaningfully to the epistemic uncertainty (see b))?
b) Figure 1 does a good job of showing the intuition behind NatPNs but it lacks some components and a discussion in the text. The authors choose to show aleatoric (un-)certainty and predictive certainty respectively but don’t show epistemic (un-)certainty. Technically, you could deduce epistemic uncertainty from aleatoric and predictive uncertainty but it would be easier to compare and follow your argument if it was made explicit. Furthermore, I would like to see an explicit discussion of the results. Why, for example, is the difference between aleatoric and predictive uncertainty so low? Is there no or little epistemic uncertainty in this setting? There are two things that would convince me more regarding this problem: a) an additional toy experiment similar to Figure 1 which includes more epistemic uncertainty, e.g. with fewer data points. This could show that the epistemic uncertainty is well-calibrated. b) An argument for why the epistemic uncertainty is (presumably) so low in your setting. a) and b) are not mutually exclusive, doing both would convince me more.
There are a couple of minor improvements: Figure 1 is not referenced in the main text. I find it hard to spot the difference w.r.t the symbols in Figure 1. Maybe just making it less crowded would already improve visibility. In the last paragraph of 3.1, you mention “warm-up” and “fine-tuning”. It would be helpful to explain these concepts briefly in one additional sentence or provide references.
What would raise my score?
I would raise my score by 1 or 2 points if my main weaknesses are well addressed or if evidence is provided that my criticism is the consequence of a misunderstanding.
I would raise my score even further if I’m convinced of the high significance of this work. This will be mostly dependent on the estimate of more expert reviewers but I’m also open to arguments by the authors.
2. Why is the arrow in Figure 2 from a Gaussian space into the latent space, rather than from the latent space to n^(i)? I thought the main purpose was to influence n^(i)?