Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap iid sampling of \(q(\mathbf{\theta})\).

The review comment to be evaluated is:

This paper presents Bayesian Hypernetworks; variational Bayesian neural networks where the variational posterior over the weights is governed by a hyper network that implements a normalizing flow (NF) such as RealNVP and IAF. As directly outputting the weight matrix with a hyper network is computationally expensive the authors instead propose to utilize weight normalisation on the weights and then use the hyper network to output scalar scaling variables for each hidden unit, similarly to what was done at [1]. The main difference with this prior work is that [1] consider these NF scaling variables as auxiliary random variables to a mean field Gaussian distribution over the weights whereas this paper attempts to posit a distribution directly on the weights via the NF. This avoids the nested variational approximation and auxiliary models of [1], which can potentially yield a tighter bound. The proposed method is evaluated on extensive experiments.

This paper seems like a plausible idea with extensive experiments but the similarity with [1] make it an incremental contribution and, furthermore, it seems that it has a technical issue with what is explained at Section 3.3. More specifically, if you generate the parameters \theta according to Eq. 7 and posit a prior over \theta then you will have a problematic variational bound as there will be a KL divergence, KL(q(\theta) || p(\theta)), with distributions of different support (since q(\theta) is defined only along the directions spanned by u), which is infinite. For the KL to be valid you will need to posit a prior distribution over `g`, p(g), and then consider KL(q(g) || p(g)), with q(g) being given by the NF. From the experiment paragraph at page 5 though I deduct that you instead employ “an isotropic standard normal prior over the weights”, i.e. \theta, thus I believe that you indeed have a problematic bound. How do you actually compute logq(\theta) when you employ the parametrisation discussed at 3.3? Did you use that parametrisation in every experiment?

Other than that, I believe that it would be interesting to experiment with a `full` hyper network, i.e. generating directly the entire parameter vector \theta, e.g. at the toy regression experiment where the dimensionality is small. This would then better illustrate the tradeoffs you make when you reduce the flexibility of the hyper-network to just outputting the row scaling variables and the effect this has at the posterior approximation.

Typos:
(1) Page 3, 3.1.1 log(\theta) -> logp(\theta).
(2) Eq. 6, it needs to be |det \frac{\partial h(\epsilon)}{\partial \epsilon}|^{-1} or |det \frac{\partial h^{-1}(\theta)}{\partial \theta}| for a valid change of variables formula.

[1] Louizos & Welling, Multiplicative Normalizing Flows for Variational Bayesian Neural Networks.

The evidence to be evaluated is:Broadly speaking, existing methods either 1) use Markov chain Monte Carlo (Welling & Teh, 2011; Neal, 1996), or 2) directly learn an approximate posterior distribution using (stochastic) variational inference (Graves, 2011; Gal & Ghahramani, 2016; Salimans et al., 2015; Blundell et al., 2015), expectation propagation (Hernandez-Lobato & Adams, 2015; Soudry et al., 2014), or \(\alpha\)-divergences (Li & Gal, 2017).

Kingma et al. (2015) emulates Gaussian dropout, but yields a unimodal approximate posterior, and does not allow arbitrary dependencies between the parameters. The other important points of reference for our work are Bayes by Backprop (BbB) (Blundell et al., 2015), and multiplicative normalizing flows (Louizos & Welling, 2017). Bayes by Backprop can be viewed as a special instance of a Bayesian hypernet, where the hypernetwork only performs an element-wise scale and shift of the input noise (yielding a factorial Gaussian distribution).

Instead, in their work, they use a hypernet to generate scaling factors, \(\mathbf{z}\) on the means \(\mathbf{\mu}\) of a factorial Gaussian distribution. Because \(\mathbf{z}\) follows a complicated distribution, this forms a highly flexible approximate posterior: \(q(\mathbf{\theta})=\int q(\mathbf{\theta}|\mathbf{z})q(\mathbf{z})\mathrm{d}\mathbf{z}\).

A hypernetwork (Ha et al., 2017; Brabandere et al., 2016; Bertinetto et al., 2016) is a neural net that outputs parameters of another neural net (the "primary network").3

Our proposed Bayesian hypernetworks employ a differentiable directed generator network (DDGN) (Goodfellow et al., 2016) as a generative model of the primary net parameters. DDGNs use a neural net to transform simple noise (most commonly isotropic Gaussian) into samples from a complex distribution, and are a common component of modern deep generative models such as variational autoencoders (VAEs) (Kingma and Welling, 2013; Jimenez Rezende et al., 2014) and generative adversarial networks (GANs) (Goodfellow et al., 2014; Goodfellow, 2017). We take advantage of techniques for _invertible_ DDGNs developed in several recent works on generative modeling (Dinh et al., 2014; Dinh et al., 2016) and variational inference of latent variables (Rezende and Mohamed, 2015; Kingma et al., 2016). Training these models uses the change of variables formula, which involves computing the log-determinant of the inverse Jacobian of the generator network. This computation involves a potentially costly matrix determinant, and these works propose innovative architectures which reduce the cost of this operation but can still express complicated deformations, which are referred to as "normalizing flows".

Bayesian hypernets (BHNs) express a flexible \(q(\mathbf{\theta})\) by using a DDGN (section 2.3), \(h\in\mathbb{R}^{D}\to\mathbb{R}^{D}\), to transform random noise \(\mathbf{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I}_{D})\) into independent samples from \(q(\mathbf{\theta})\). This makes it cheap to compute Monte Carlo estimations of expectations with respect to \(q\); these include the ELBO, and its derivatives, which can be backpropagated to train the hypernet \(h\). Since BHNs are both trained and evaluated via samples of \(q(\mathbf{\theta})\), expressing \(q(\mathbf{\theta})\) as a generative model is a natural strategy.

\[q(\mathbf{\theta})= \int q(\mathbf{\theta};h(\mathbf{\epsilon}))q(\mathbf{\epsilon})d\mathbf{ \epsilon}\,. \tag{5}\] To avoid this issue, we use an invertible \(h\), allowing us to compute \(q(\mathbf{\theta})\) simply by using the change of variables formula: \[q(\mathbf{\theta})=q_{\mathbf{\epsilon}}(\epsilon)\left|\det\frac{\partial h(\epsilon )}{\partial\mathbf{\epsilon}}\right|^{-1}\,, \tag{6}\] where \(q_{\mathbf{\epsilon}}\) is the distribution of \(\mathbf{\epsilon}\) and \(\mathbf{\theta}=h(\epsilon)\).

In this work, we employ both RealNVP (RNVP) (Dinh et al., 2016) and Inverse Autoregressive Flows (IAF) (Kingma et al., 2016). Note that the latter can be efficiently applied, since we only require the ability to evaluate likelihood of generated samples (not arbitrary points in the range of \(h\), as in generative modeling applications, e.g., Dinh et al. (2016)); and this also means that we can use a lower-dimensional \(\mathbf{\epsilon}\) to generate samples along a submanifold of the entire parameter space, as detailed below.

This allows us to overcome the computational limitations of naively-parametrized BHNs noted by Louizos & Welling (2017), since computation now scales linearly, instead of quadratically, in the number of primary net units. Using this parametrization restricts the family of approximate posteriors, but still allows for a high degree of multimodality and dependence between the parameters.

Next, we demonstrate the distinctive ability of Bayesian hypernets to learn multi-modal, dependent distributions. Figure 6 (appendix) shows that BHNs do learn approximate posteriors with dependence between different parameters, as measured by the Pearson correlation coefficient. Meanwhile, Figure 2 shows that BHNs are capable of learning multimodal posteriors. For this experiment, we trained an over-parametrized linear (primary) network: \(\hat{y}=a\cdot b\cdot x\) on a dataset generated as \(y=x+\epsilon\), and the BHN learns capture both the modes of \(a=b=1\) and \(a=b=-1\).

furthermore, increasing the flexibility of the posterior by adding more coupling layers improves performance, especially compared with models with 0 coupling layers, which cannot model dependencies between the parameters.

We introduce Bayesian hypernets (BHNs), a new method for variational Bayesian deep learning which uses an invertible hypernetwork as a generative model of parameters. BHNs feature efficient training and sampling, and can express complicated multimodal distributions, thereby addressing issues of overconfidence present in simpler variational approximations. We present a method of parametrizing BHNs which allows them to scale successfully to real world tasks, and show that BHNs can offer significant benefits over simpler methods for Bayesian deep learning.

In this paper, we employ weight normalization in the primary network (7), treating (only) the scaling factors \(\mathbf{g}\) as random variables. We choose an isotropic Gaussian prior for \(\mathbf{g}\): \(p(\mathbf{g})=\mathcal{N}(\mathbf{g};\mathbf{0},\lambda\mathbf{I})\), which results in an \(L_{2}\) weight-decay penalty on \(\mathbf{g}\), or, equivalently, \(\mathbf{w}=\mathbf{g}\frac{\mathbf{v}}{||\mathbf{v}||_{2}}\). Our objective and lower bound are then: \[\log p(\mathcal{D};\mathbf{v},\mathbf{b}) =\log\int_{\mathbf{g}}p(\mathcal{D}|\mathbf{g};\mathbf{v},\mathbf{ b})p(\mathbf{g})d\mathbf{g} \tag{8}\] \[\geq\mathbb{E}_{q(\mathbf{g})}[\log p(\mathcal{D}|\mathbf{g}; \mathbf{v},\mathbf{b})+\log p(\mathbf{g})-\log q(\mathbf{g})]\] (9) \[\geq\mathbb{E}_{\mathbf{\epsilon}\sim q_{\mathbf{\epsilon}}(\mathbf{ \epsilon}),\mathbf{g}=h_{\phi}(\mathbf{\epsilon})}[\log p(\mathcal{D}|\mathbf{g}; \mathbf{v},\mathbf{b})+\log p(\mathbf{g})-\log q(\mathbf{\epsilon})+\log\left| \det\frac{\partial h_{\phi}(\mathbf{\epsilon})}{\partial\mathbf{\epsilon}}\right|] \tag{10}\] where \(\mathbf{v}\) and \(\mathbf{b}\) are the direction and bias parameters of the primary net, and \(\phi\) is the parameters of the hypernetwork. We optimize this bound with respect to \(\{\mathbf{v},\mathbf{b},\phi\}\) during training.

Input: Figure 2: Learning the identity function with an overparametrized network: ŷ = a · b · x. This parametrization results in symmetries shown by the dashed red lines, and the Bayesian hypernetwork assigns significant probability mass to both modes of the posterior (a = b = 1 and a = b = −1).

Input: Figure 6: Histogram of Pearson correlation coefficient p-values (left) and a scatter matrix (right) of samples from a hypernet approximate posterior. We see that the hypernet posterior includes correlations between different parameters. Many of the p-values of the Pearson correlation test are below .05.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find the claim to be largely well-stated with minor caveats. The evidence demonstrates that:

1. The method can indeed represent multimodal posteriors (shown clearly in Figure 2 for a simple case)
2. Parameter correlations are captured (evidenced by Figure 6)
3. The sampling is efficient through the use of invertible networks and the change of variables formula

However, the reviewer raises valid technical concerns about the KL divergence computation when using the weight normalization parametrization. While this doesn't invalidate the core claim, it suggests some theoretical details need clarification.

The experimental results support the key assertions about multimodality and correlations, and the sampling efficiency is well-justified mathematically.

<score>2</score>

Justification:
1. The claim about representing "complex multimodal approximate posterior" is supported by:
- Figure 2 demonstrating multimodality
- Figure 6 showing parameter correlations
- Mathematical formulation using normalizing flows

2. The claim about "cheap iid sampling" is supported by:
- The use of invertible networks allowing direct sampling
- Equations 5-6 showing how sampling is performed efficiently

3. Minor overstatement concerns:
- The reviewer's point about potential issues with KL divergence computation
- Limited demonstration of complex multimodality (main example is relatively simple)
- Some theoretical details that could benefit from clarification

The evidence largely supports the claim, but there are some technical nuances that weren't fully addressed in the claim's broad statements, hence the score of 2 (slightly overstated but generally accurate).


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap iid sampling of \(q(\mathbf{\theta})\).

The review comment to be evaluated is:

This paper proposes Bayesian hypernetworks to carry out Bayesian learning of deep networks. The idea is to construct a generative model capable of approximating the posterior distribution over the parameters of deep networks. I think that the paper is well written and easy to follow.

I like the idea of constructing general approximation strategies for complex posterior distribution and the proposed approach inherits all the scalability properties of modern deep learning techniques. In this respect, I think that the paper tackles a timely topic and is interesting to read.

It is not entirely clear to me why the Authors name their proposal Bayesian hypernetworks. This seems to suggest that also the hypernetwork is infered using Bayesian inference, but if I understand correctly this is not the case.

I have some comments on novelty and realization of the experiments. In the positioning of the work in the literature, the Authors point out that hypernetworks have been proposed before, so it is not clear what is the actual novelty in the proposal. Is it the use of Real NVPs and IAFs as hypernetworks? These methods have been already proposed and extensively studied in the literature, and even if they have been adapted to be hypernetworks here, I believe that the novelty is fairly limited.

The experimental part is interesting as it explores a number of learning scenarios. However, I think that it would have been useful to add comparisons with standard variational inference (e.g., Graves, 2011) for deep networks to substantiate the claims that this approach underestimates uncertainty. I believe that this would strengthen the comparative evaluation.

I think the paper would have made a stronger case by including other approaches to approximate posteriors using generative models. For example, the variational Gaussian process paper sounds like an ideal method to include here.

[1] D. Tran, R. Ranganath, and D. M. Blei. Variational Gaussian process. arXiv preprint arXiv:1511.06499, 2015.

The evidence to be evaluated is:Broadly speaking, existing methods either 1) use Markov chain Monte Carlo (Welling & Teh, 2011; Neal, 1996), or 2) directly learn an approximate posterior distribution using (stochastic) variational inference (Graves, 2011; Gal & Ghahramani, 2016; Salimans et al., 2015; Blundell et al., 2015), expectation propagation (Hernandez-Lobato & Adams, 2015; Soudry et al., 2014), or \(\alpha\)-divergences (Li & Gal, 2017).

Kingma et al. (2015) emulates Gaussian dropout, but yields a unimodal approximate posterior, and does not allow arbitrary dependencies between the parameters. The other important points of reference for our work are Bayes by Backprop (BbB) (Blundell et al., 2015), and multiplicative normalizing flows (Louizos & Welling, 2017). Bayes by Backprop can be viewed as a special instance of a Bayesian hypernet, where the hypernetwork only performs an element-wise scale and shift of the input noise (yielding a factorial Gaussian distribution).

Instead, in their work, they use a hypernet to generate scaling factors, \(\mathbf{z}\) on the means \(\mathbf{\mu}\) of a factorial Gaussian distribution. Because \(\mathbf{z}\) follows a complicated distribution, this forms a highly flexible approximate posterior: \(q(\mathbf{\theta})=\int q(\mathbf{\theta}|\mathbf{z})q(\mathbf{z})\mathrm{d}\mathbf{z}\).

A hypernetwork (Ha et al., 2017; Brabandere et al., 2016; Bertinetto et al., 2016) is a neural net that outputs parameters of another neural net (the "primary network").3

Our proposed Bayesian hypernetworks employ a differentiable directed generator network (DDGN) (Goodfellow et al., 2016) as a generative model of the primary net parameters. DDGNs use a neural net to transform simple noise (most commonly isotropic Gaussian) into samples from a complex distribution, and are a common component of modern deep generative models such as variational autoencoders (VAEs) (Kingma and Welling, 2013; Jimenez Rezende et al., 2014) and generative adversarial networks (GANs) (Goodfellow et al., 2014; Goodfellow, 2017). We take advantage of techniques for _invertible_ DDGNs developed in several recent works on generative modeling (Dinh et al., 2014; Dinh et al., 2016) and variational inference of latent variables (Rezende and Mohamed, 2015; Kingma et al., 2016). Training these models uses the change of variables formula, which involves computing the log-determinant of the inverse Jacobian of the generator network. This computation involves a potentially costly matrix determinant, and these works propose innovative architectures which reduce the cost of this operation but can still express complicated deformations, which are referred to as "normalizing flows".

Bayesian hypernets (BHNs) express a flexible \(q(\mathbf{\theta})\) by using a DDGN (section 2.3), \(h\in\mathbb{R}^{D}\to\mathbb{R}^{D}\), to transform random noise \(\mathbf{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I}_{D})\) into independent samples from \(q(\mathbf{\theta})\). This makes it cheap to compute Monte Carlo estimations of expectations with respect to \(q\); these include the ELBO, and its derivatives, which can be backpropagated to train the hypernet \(h\). Since BHNs are both trained and evaluated via samples of \(q(\mathbf{\theta})\), expressing \(q(\mathbf{\theta})\) as a generative model is a natural strategy.

\[q(\mathbf{\theta})= \int q(\mathbf{\theta};h(\mathbf{\epsilon}))q(\mathbf{\epsilon})d\mathbf{ \epsilon}\,. \tag{5}\] To avoid this issue, we use an invertible \(h\), allowing us to compute \(q(\mathbf{\theta})\) simply by using the change of variables formula: \[q(\mathbf{\theta})=q_{\mathbf{\epsilon}}(\epsilon)\left|\det\frac{\partial h(\epsilon )}{\partial\mathbf{\epsilon}}\right|^{-1}\,, \tag{6}\] where \(q_{\mathbf{\epsilon}}\) is the distribution of \(\mathbf{\epsilon}\) and \(\mathbf{\theta}=h(\epsilon)\).

In this work, we employ both RealNVP (RNVP) (Dinh et al., 2016) and Inverse Autoregressive Flows (IAF) (Kingma et al., 2016). Note that the latter can be efficiently applied, since we only require the ability to evaluate likelihood of generated samples (not arbitrary points in the range of \(h\), as in generative modeling applications, e.g., Dinh et al. (2016)); and this also means that we can use a lower-dimensional \(\mathbf{\epsilon}\) to generate samples along a submanifold of the entire parameter space, as detailed below.

This allows us to overcome the computational limitations of naively-parametrized BHNs noted by Louizos & Welling (2017), since computation now scales linearly, instead of quadratically, in the number of primary net units. Using this parametrization restricts the family of approximate posteriors, but still allows for a high degree of multimodality and dependence between the parameters.

Next, we demonstrate the distinctive ability of Bayesian hypernets to learn multi-modal, dependent distributions. Figure 6 (appendix) shows that BHNs do learn approximate posteriors with dependence between different parameters, as measured by the Pearson correlation coefficient. Meanwhile, Figure 2 shows that BHNs are capable of learning multimodal posteriors. For this experiment, we trained an over-parametrized linear (primary) network: \(\hat{y}=a\cdot b\cdot x\) on a dataset generated as \(y=x+\epsilon\), and the BHN learns capture both the modes of \(a=b=1\) and \(a=b=-1\).

furthermore, increasing the flexibility of the posterior by adding more coupling layers improves performance, especially compared with models with 0 coupling layers, which cannot model dependencies between the parameters.

We introduce Bayesian hypernets (BHNs), a new method for variational Bayesian deep learning which uses an invertible hypernetwork as a generative model of parameters. BHNs feature efficient training and sampling, and can express complicated multimodal distributions, thereby addressing issues of overconfidence present in simpler variational approximations. We present a method of parametrizing BHNs which allows them to scale successfully to real world tasks, and show that BHNs can offer significant benefits over simpler methods for Bayesian deep learning.

In this paper, we employ weight normalization in the primary network (7), treating (only) the scaling factors \(\mathbf{g}\) as random variables. We choose an isotropic Gaussian prior for \(\mathbf{g}\): \(p(\mathbf{g})=\mathcal{N}(\mathbf{g};\mathbf{0},\lambda\mathbf{I})\), which results in an \(L_{2}\) weight-decay penalty on \(\mathbf{g}\), or, equivalently, \(\mathbf{w}=\mathbf{g}\frac{\mathbf{v}}{||\mathbf{v}||_{2}}\). Our objective and lower bound are then: \[\log p(\mathcal{D};\mathbf{v},\mathbf{b}) =\log\int_{\mathbf{g}}p(\mathcal{D}|\mathbf{g};\mathbf{v},\mathbf{ b})p(\mathbf{g})d\mathbf{g} \tag{8}\] \[\geq\mathbb{E}_{q(\mathbf{g})}[\log p(\mathcal{D}|\mathbf{g}; \mathbf{v},\mathbf{b})+\log p(\mathbf{g})-\log q(\mathbf{g})]\] (9) \[\geq\mathbb{E}_{\mathbf{\epsilon}\sim q_{\mathbf{\epsilon}}(\mathbf{ \epsilon}),\mathbf{g}=h_{\phi}(\mathbf{\epsilon})}[\log p(\mathcal{D}|\mathbf{g}; \mathbf{v},\mathbf{b})+\log p(\mathbf{g})-\log q(\mathbf{\epsilon})+\log\left| \det\frac{\partial h_{\phi}(\mathbf{\epsilon})}{\partial\mathbf{\epsilon}}\right|] \tag{10}\] where \(\mathbf{v}\) and \(\mathbf{b}\) are the direction and bias parameters of the primary net, and \(\phi\) is the parameters of the hypernetwork. We optimize this bound with respect to \(\{\mathbf{v},\mathbf{b},\phi\}\) during training.

Input: Figure 2: Learning the identity function with an overparametrized network: ŷ = a · b · x. This parametrization results in symmetries shown by the dashed red lines, and the Bayesian hypernetwork assigns significant probability mass to both modes of the posterior (a = b = 1 and a = b = −1).

Input: Figure 6: Histogram of Pearson correlation coefficient p-values (left) and a scatter matrix (right) of samples from a hypernet approximate posterior. We see that the hypernet posterior includes correlations between different parameters. Many of the p-values of the Pearson correlation test are below .05.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review, the claim appears well-stated and accurately reflects the evidence presented. The paper demonstrates through mathematical formulations, visualizations (Figure 2 and 6), and experimental results that Bayesian hypernets can indeed represent multimodal posteriors with parameter correlations. The evidence shows both the theoretical foundation (equations 5-10) and empirical validation through experiments showing multimodality (Figure 2) and parameter correlations (Figure 6). The sampling efficiency is supported by the invertible architecture design. The reviewer's comments, while noting some limitations in novelty, don't dispute these core capabilities.

<score>1</score>

Justification:
1. Multimodality is clearly demonstrated in Figure 2, showing the hypernet captures both modes (a=b=1 and a=b=-1)
2. Parameter correlations are evidenced by Figure 6's correlation analysis
3. The sampling efficiency is supported by:
   - Use of invertible architectures (RealNVP and IAF)
   - Linear scaling achieved through weight normalization
   - Direct mathematical formulation for sampling via change of variables (equation 6)
4. Reviewer comments primarily focus on novelty rather than questioning these technical capabilities
5. All key claims are backed by both theoretical development and empirical validation


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap iid sampling of \(q(\mathbf{\theta})\).

The review comment to be evaluated is:

* Edit: I increased my rating to 6. The authors fixed the first error I pointed out below. Regarding the second point: I still think it is possible to take a limit of sigma -> 0 in MNF, which makes the methods very similar.

The authors propose a new method of defining approximate posteriors for use in Bayesian neural networks. The idea of using hypernetworks for Bayesian inference is compelling, and the authors show some promising first results. I see two issues, and would be willing to increase my rating if these were sufficiently addressed.

- The paper says it uses an "isotropic standard normal prior on the weights of the network". However, the stochastic part of the generated weights (i.e. the scales) is of a lower dimension than the weights. It seems to me this means that the KL divergence between prior and posterior is undefined, or infinite, as the posterior is only defined on a sub-manifold. What exactly is the loss term that is added to the training objective? And how is this justified?

- The instantiation of Bayesian hypernetworks that is used in experiments seems to be a special case of the method of multiplicative normalizing flows as proposed by Louizos and Welling and discussed in this paper. If the variances / sigmas are zero in the latter method, their approximation seems functionally equivalent to Bayesian hypernetworks (though with different parameterization). Is my understanding correct? If so, the novelty of the proposed method is limited.

The evidence to be evaluated is:Broadly speaking, existing methods either 1) use Markov chain Monte Carlo (Welling & Teh, 2011; Neal, 1996), or 2) directly learn an approximate posterior distribution using (stochastic) variational inference (Graves, 2011; Gal & Ghahramani, 2016; Salimans et al., 2015; Blundell et al., 2015), expectation propagation (Hernandez-Lobato & Adams, 2015; Soudry et al., 2014), or \(\alpha\)-divergences (Li & Gal, 2017).

Kingma et al. (2015) emulates Gaussian dropout, but yields a unimodal approximate posterior, and does not allow arbitrary dependencies between the parameters. The other important points of reference for our work are Bayes by Backprop (BbB) (Blundell et al., 2015), and multiplicative normalizing flows (Louizos & Welling, 2017). Bayes by Backprop can be viewed as a special instance of a Bayesian hypernet, where the hypernetwork only performs an element-wise scale and shift of the input noise (yielding a factorial Gaussian distribution).

Instead, in their work, they use a hypernet to generate scaling factors, \(\mathbf{z}\) on the means \(\mathbf{\mu}\) of a factorial Gaussian distribution. Because \(\mathbf{z}\) follows a complicated distribution, this forms a highly flexible approximate posterior: \(q(\mathbf{\theta})=\int q(\mathbf{\theta}|\mathbf{z})q(\mathbf{z})\mathrm{d}\mathbf{z}\).

A hypernetwork (Ha et al., 2017; Brabandere et al., 2016; Bertinetto et al., 2016) is a neural net that outputs parameters of another neural net (the "primary network").3

Our proposed Bayesian hypernetworks employ a differentiable directed generator network (DDGN) (Goodfellow et al., 2016) as a generative model of the primary net parameters. DDGNs use a neural net to transform simple noise (most commonly isotropic Gaussian) into samples from a complex distribution, and are a common component of modern deep generative models such as variational autoencoders (VAEs) (Kingma and Welling, 2013; Jimenez Rezende et al., 2014) and generative adversarial networks (GANs) (Goodfellow et al., 2014; Goodfellow, 2017). We take advantage of techniques for _invertible_ DDGNs developed in several recent works on generative modeling (Dinh et al., 2014; Dinh et al., 2016) and variational inference of latent variables (Rezende and Mohamed, 2015; Kingma et al., 2016). Training these models uses the change of variables formula, which involves computing the log-determinant of the inverse Jacobian of the generator network. This computation involves a potentially costly matrix determinant, and these works propose innovative architectures which reduce the cost of this operation but can still express complicated deformations, which are referred to as "normalizing flows".

Bayesian hypernets (BHNs) express a flexible \(q(\mathbf{\theta})\) by using a DDGN (section 2.3), \(h\in\mathbb{R}^{D}\to\mathbb{R}^{D}\), to transform random noise \(\mathbf{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I}_{D})\) into independent samples from \(q(\mathbf{\theta})\). This makes it cheap to compute Monte Carlo estimations of expectations with respect to \(q\); these include the ELBO, and its derivatives, which can be backpropagated to train the hypernet \(h\). Since BHNs are both trained and evaluated via samples of \(q(\mathbf{\theta})\), expressing \(q(\mathbf{\theta})\) as a generative model is a natural strategy.

\[q(\mathbf{\theta})= \int q(\mathbf{\theta};h(\mathbf{\epsilon}))q(\mathbf{\epsilon})d\mathbf{ \epsilon}\,. \tag{5}\] To avoid this issue, we use an invertible \(h\), allowing us to compute \(q(\mathbf{\theta})\) simply by using the change of variables formula: \[q(\mathbf{\theta})=q_{\mathbf{\epsilon}}(\epsilon)\left|\det\frac{\partial h(\epsilon )}{\partial\mathbf{\epsilon}}\right|^{-1}\,, \tag{6}\] where \(q_{\mathbf{\epsilon}}\) is the distribution of \(\mathbf{\epsilon}\) and \(\mathbf{\theta}=h(\epsilon)\).

In this work, we employ both RealNVP (RNVP) (Dinh et al., 2016) and Inverse Autoregressive Flows (IAF) (Kingma et al., 2016). Note that the latter can be efficiently applied, since we only require the ability to evaluate likelihood of generated samples (not arbitrary points in the range of \(h\), as in generative modeling applications, e.g., Dinh et al. (2016)); and this also means that we can use a lower-dimensional \(\mathbf{\epsilon}\) to generate samples along a submanifold of the entire parameter space, as detailed below.

This allows us to overcome the computational limitations of naively-parametrized BHNs noted by Louizos & Welling (2017), since computation now scales linearly, instead of quadratically, in the number of primary net units. Using this parametrization restricts the family of approximate posteriors, but still allows for a high degree of multimodality and dependence between the parameters.

Next, we demonstrate the distinctive ability of Bayesian hypernets to learn multi-modal, dependent distributions. Figure 6 (appendix) shows that BHNs do learn approximate posteriors with dependence between different parameters, as measured by the Pearson correlation coefficient. Meanwhile, Figure 2 shows that BHNs are capable of learning multimodal posteriors. For this experiment, we trained an over-parametrized linear (primary) network: \(\hat{y}=a\cdot b\cdot x\) on a dataset generated as \(y=x+\epsilon\), and the BHN learns capture both the modes of \(a=b=1\) and \(a=b=-1\).

furthermore, increasing the flexibility of the posterior by adding more coupling layers improves performance, especially compared with models with 0 coupling layers, which cannot model dependencies between the parameters.

We introduce Bayesian hypernets (BHNs), a new method for variational Bayesian deep learning which uses an invertible hypernetwork as a generative model of parameters. BHNs feature efficient training and sampling, and can express complicated multimodal distributions, thereby addressing issues of overconfidence present in simpler variational approximations. We present a method of parametrizing BHNs which allows them to scale successfully to real world tasks, and show that BHNs can offer significant benefits over simpler methods for Bayesian deep learning.

In this paper, we employ weight normalization in the primary network (7), treating (only) the scaling factors \(\mathbf{g}\) as random variables. We choose an isotropic Gaussian prior for \(\mathbf{g}\): \(p(\mathbf{g})=\mathcal{N}(\mathbf{g};\mathbf{0},\lambda\mathbf{I})\), which results in an \(L_{2}\) weight-decay penalty on \(\mathbf{g}\), or, equivalently, \(\mathbf{w}=\mathbf{g}\frac{\mathbf{v}}{||\mathbf{v}||_{2}}\). Our objective and lower bound are then: \[\log p(\mathcal{D};\mathbf{v},\mathbf{b}) =\log\int_{\mathbf{g}}p(\mathcal{D}|\mathbf{g};\mathbf{v},\mathbf{ b})p(\mathbf{g})d\mathbf{g} \tag{8}\] \[\geq\mathbb{E}_{q(\mathbf{g})}[\log p(\mathcal{D}|\mathbf{g}; \mathbf{v},\mathbf{b})+\log p(\mathbf{g})-\log q(\mathbf{g})]\] (9) \[\geq\mathbb{E}_{\mathbf{\epsilon}\sim q_{\mathbf{\epsilon}}(\mathbf{ \epsilon}),\mathbf{g}=h_{\phi}(\mathbf{\epsilon})}[\log p(\mathcal{D}|\mathbf{g}; \mathbf{v},\mathbf{b})+\log p(\mathbf{g})-\log q(\mathbf{\epsilon})+\log\left| \det\frac{\partial h_{\phi}(\mathbf{\epsilon})}{\partial\mathbf{\epsilon}}\right|] \tag{10}\] where \(\mathbf{v}\) and \(\mathbf{b}\) are the direction and bias parameters of the primary net, and \(\phi\) is the parameters of the hypernetwork. We optimize this bound with respect to \(\{\mathbf{v},\mathbf{b},\phi\}\) during training.

Input: Figure 2: Learning the identity function with an overparametrized network: ŷ = a · b · x. This parametrization results in symmetries shown by the dashed red lines, and the Bayesian hypernetwork assigns significant probability mass to both modes of the posterior (a = b = 1 and a = b = −1).

Input: Figure 6: Histogram of Pearson correlation coefficient p-values (left) and a scatter matrix (right) of samples from a hypernet approximate posterior. We see that the hypernet posterior includes correlations between different parameters. Many of the p-values of the Pearson correlation test are below .05.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find the claim to be well-stated (1) for several reasons:

1. The evidence directly demonstrates the ability to represent multimodal posteriors through Figure 2, which shows the BHN successfully capturing two distinct modes in the parameter space.

2. The correlation analysis in Figure 6 provides clear evidence of dependencies between parameters.

3. The method's sampling efficiency is supported by the mathematical formulation using invertible networks (equations 5-6), allowing direct sampling without rejection or MCMC.

4. The reviewer's concerns about dimensionality and similarity to MNF don't invalidate the core claims about the method's capabilities.

<score>1</score>

Detailed justification:

1. Multimodality: Figure 2 provides clear empirical evidence of capturing multiple modes in the posterior, comparing favorably with NUTS (HMC).

2. Parameter correlations: Figure 6 quantitatively demonstrates correlations between parameters through Pearson correlation tests.

3. Sampling efficiency: The mathematical formulation using invertible networks (equations 5-10) enables direct sampling through the change of variables formula, avoiding the computational costs noted in prior work.

4. The reviewer's technical concerns about dimensionality and similarity to MNF are valid points for discussion but don't contradict the central claims about the method's representational capabilities and sampling efficiency.

5. The claim's wording is measured, using "can represent" rather than stronger claims about superiority or universality, and is directly supported by the experimental evidence provided.

The review comments raise important technical points but don't fundamentally challenge the specific capabilities claimed in the statement. The evidence thoroughly supports each aspect of the claim without overstatement.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In practice, Bayesian hypernets provide a better defense against adversarial examples than dropout, and also exhibit competitive performance on a suite of tasks which evaluate model uncertainty, including regularization, active learning, and anomaly detection.

The evidence to be evaluated is:There is no single metric for how well a model captures uncertainty; to evaluate our model, we perform experiments on regularization (Section 4.2), active learning (Section 4.3), anomaly detection (Section 4.4), and detection of adversarial examples (Section 4.5). Active learning and anomaly detection problems make natural use of uncertainty estimates: In anomaly detection, higher uncertainty indicates a likely anomaly. In active learning, higher uncertainty indicates a greater opportunity for learning. Parameter uncertainty also has regularization benefits: integrating over the posterior creates an implicit ensemble.

Adversarial examples are an especially difficult kind of anomaly designed to fool a classifier, and finding effective defenses against adversarial attacks remains an open challenge in deep learning.

We now show that BHNs act as a regularizer, outperforming dropout and traditional mean field (BbB).

In our experiments, we find that BHNs perform on par with dropout on full datasets of MNIST and CIFAR10; furthermore, increasing the flexibility of the posterior by adding more coupling layers improves performance, especially compared with models with 0 coupling layers, which cannot model dependencies between the parameters.

Overall, warm-started BHNs suffered at the beginning of training, but outperformed all other methods for moderate to large numbers of acquisitions.

We demonstrate that Bayesian DNNs outperform their non-Bayesian counterparts. Just as in active learning, in anomaly detection, we use MC to estimate the predictive posterior, and use this to score datapoints.

This experiment shows the benefit of using scores that reflect dispersion in the posterior samples (such as mean standard deviation and BALD value) in Bayesian DNNs. Finally, we consider this same anomaly detection procedure as a novel tool for detecting adversarial examples. Our setup is similar to Li & Gal (2017) and Louizos & Welling (2017), where it is shown that when more perturbation is added to the data, model uncertainty increases and then drops. We use the Fast Gradient Sign method (FGS) (Goodfellow et al., 2014) for adversarial attack, and use one sample of our model to estimate the gradient.10 We find that, compared with dropout, BHNs are less confident on data points which are far from the data manifold. In particular, BHNs constructed with IAF consistently outperform RealNVP-BHNs and dropout in detecting adversarial examples and errors.

BHNs feature efficient training and sampling, and can express complicated multimodal distributions, thereby addressing issues of overconfidence present in simpler variational approximations. We present a method of parametrizing BHNs which allows them to scale successfully to real world tasks, and show that BHNs can offer significant benefits over simpler methods for Bayesian deep learning.

Input: Figure 7: Adversary detection with 32-sample estimate of gradient.

Input: Figure 5: Adversary detection: Horizontal axis is the step size of the FGS algorithm. While accuracy drops when more perturbation is added to the data (left), uncertainty measures also increase (first row). In particular, the BALD and Mean STD scores, which measure epistemic uncertainty, are strongly increasing for BHNs, but not for dropout. The second row and third row plots show results for adversary detection and error detection (respectively) in terms of the AUC of ROC (y-axis) with increasing perturbation along the x-axis. Gradient direction is estimated with one Monte Carlo sample of the weights/dropout mask.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Looking at the claim and evidence, I'll assess its validity:

The claim states that Bayesian hypernets provide "better defense against adversarial examples than dropout" and show "competitive performance" on uncertainty tasks. The evidence partially supports this:

- For adversarial examples: Figures 5 and 7 show BHNs (particularly IAF variant) consistently outperforming dropout across different uncertainty measures when detecting adversarial examples
- For other uncertainty tasks: The evidence indicates BHNs perform "on par" with dropout on MNIST/CIFAR10, "outperformed all other methods" in active learning after initial warmup, and demonstrate benefits in anomaly detection

While the evidence supports BHNs performing well, the claim is generally well-stated as it uses measured language ("competitive performance") and is backed by experimental results showing superior performance specifically for adversarial examples.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In practice, Bayesian hypernets provide a better defense against adversarial examples than dropout, and also exhibit competitive performance on a suite of tasks which evaluate model uncertainty, including regularization, active learning, and anomaly detection.

The review comment to be evaluated is:

This paper presents Bayesian Hypernetworks; variational Bayesian neural networks where the variational posterior over the weights is governed by a hyper network that implements a normalizing flow (NF) such as RealNVP and IAF. As directly outputting the weight matrix with a hyper network is computationally expensive the authors instead propose to utilize weight normalisation on the weights and then use the hyper network to output scalar scaling variables for each hidden unit, similarly to what was done at [1]. The main difference with this prior work is that [1] consider these NF scaling variables as auxiliary random variables to a mean field Gaussian distribution over the weights whereas this paper attempts to posit a distribution directly on the weights via the NF. This avoids the nested variational approximation and auxiliary models of [1], which can potentially yield a tighter bound. The proposed method is evaluated on extensive experiments.

This paper seems like a plausible idea with extensive experiments but the similarity with [1] make it an incremental contribution and, furthermore, it seems that it has a technical issue with what is explained at Section 3.3. More specifically, if you generate the parameters \theta according to Eq. 7 and posit a prior over \theta then you will have a problematic variational bound as there will be a KL divergence, KL(q(\theta) || p(\theta)), with distributions of different support (since q(\theta) is defined only along the directions spanned by u), which is infinite. For the KL to be valid you will need to posit a prior distribution over `g`, p(g), and then consider KL(q(g) || p(g)), with q(g) being given by the NF. From the experiment paragraph at page 5 though I deduct that you instead employ “an isotropic standard normal prior over the weights”, i.e. \theta, thus I believe that you indeed have a problematic bound. How do you actually compute logq(\theta) when you employ the parametrisation discussed at 3.3? Did you use that parametrisation in every experiment?

Other than that, I believe that it would be interesting to experiment with a `full` hyper network, i.e. generating directly the entire parameter vector \theta, e.g. at the toy regression experiment where the dimensionality is small. This would then better illustrate the tradeoffs you make when you reduce the flexibility of the hyper-network to just outputting the row scaling variables and the effect this has at the posterior approximation.

Typos:
(1) Page 3, 3.1.1 log(\theta) -> logp(\theta).
(2) Eq. 6, it needs to be |det \frac{\partial h(\epsilon)}{\partial \epsilon}|^{-1} or |det \frac{\partial h^{-1}(\theta)}{\partial \theta}| for a valid change of variables formula.

[1] Louizos & Welling, Multiplicative Normalizing Flows for Variational Bayesian Neural Networks.

The evidence to be evaluated is:There is no single metric for how well a model captures uncertainty; to evaluate our model, we perform experiments on regularization (Section 4.2), active learning (Section 4.3), anomaly detection (Section 4.4), and detection of adversarial examples (Section 4.5). Active learning and anomaly detection problems make natural use of uncertainty estimates: In anomaly detection, higher uncertainty indicates a likely anomaly. In active learning, higher uncertainty indicates a greater opportunity for learning. Parameter uncertainty also has regularization benefits: integrating over the posterior creates an implicit ensemble.

Adversarial examples are an especially difficult kind of anomaly designed to fool a classifier, and finding effective defenses against adversarial attacks remains an open challenge in deep learning.

We now show that BHNs act as a regularizer, outperforming dropout and traditional mean field (BbB).

In our experiments, we find that BHNs perform on par with dropout on full datasets of MNIST and CIFAR10; furthermore, increasing the flexibility of the posterior by adding more coupling layers improves performance, especially compared with models with 0 coupling layers, which cannot model dependencies between the parameters.

Overall, warm-started BHNs suffered at the beginning of training, but outperformed all other methods for moderate to large numbers of acquisitions.

We demonstrate that Bayesian DNNs outperform their non-Bayesian counterparts. Just as in active learning, in anomaly detection, we use MC to estimate the predictive posterior, and use this to score datapoints.

This experiment shows the benefit of using scores that reflect dispersion in the posterior samples (such as mean standard deviation and BALD value) in Bayesian DNNs. Finally, we consider this same anomaly detection procedure as a novel tool for detecting adversarial examples. Our setup is similar to Li & Gal (2017) and Louizos & Welling (2017), where it is shown that when more perturbation is added to the data, model uncertainty increases and then drops. We use the Fast Gradient Sign method (FGS) (Goodfellow et al., 2014) for adversarial attack, and use one sample of our model to estimate the gradient.10 We find that, compared with dropout, BHNs are less confident on data points which are far from the data manifold. In particular, BHNs constructed with IAF consistently outperform RealNVP-BHNs and dropout in detecting adversarial examples and errors.

BHNs feature efficient training and sampling, and can express complicated multimodal distributions, thereby addressing issues of overconfidence present in simpler variational approximations. We present a method of parametrizing BHNs which allows them to scale successfully to real world tasks, and show that BHNs can offer significant benefits over simpler methods for Bayesian deep learning.

Input: Figure 7: Adversary detection with 32-sample estimate of gradient.

Input: Figure 5: Adversary detection: Horizontal axis is the step size of the FGS algorithm. While accuracy drops when more perturbation is added to the data (left), uncertainty measures also increase (first row). In particular, the BALD and Mean STD scores, which measure epistemic uncertainty, are strongly increasing for BHNs, but not for dropout. The second row and third row plots show results for adversary detection and error detection (respectively) in terms of the AUC of ROC (y-axis) with increasing perturbation along the x-axis. Gradient direction is estimated with one Monte Carlo sample of the weights/dropout mask.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim based on the evidence and review comment:

Summary: The claim states BHNs provide "better defense against adversarial examples than dropout" and show "competitive performance" on uncertainty tasks. While the evidence shows BHNs (particularly IAF) outperforming dropout in adversarial detection scenarios (as shown in Figures 5 and 7), and demonstrating good performance in various uncertainty tasks, a few considerations suggest slight overstatement:

1. The technical issue raised in the review regarding the KL divergence bound could affect the theoretical soundness
2. The review suggests the contribution is somewhat incremental given similarity to prior work
3. The evidence shows BHNs performing "on par" (not necessarily better) in some cases
4. Some results are qualified (e.g., "warm-started BHNs suffered at beginning")

While the experimental results generally support the claim, the wording could be more precise to acknowledge these nuances.

<score>2</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In practice, Bayesian hypernets provide a better defense against adversarial examples than dropout, and also exhibit competitive performance on a suite of tasks which evaluate model uncertainty, including regularization, active learning, and anomaly detection.

The review comment to be evaluated is:

This paper proposes Bayesian hypernetworks to carry out Bayesian learning of deep networks. The idea is to construct a generative model capable of approximating the posterior distribution over the parameters of deep networks. I think that the paper is well written and easy to follow.

I like the idea of constructing general approximation strategies for complex posterior distribution and the proposed approach inherits all the scalability properties of modern deep learning techniques. In this respect, I think that the paper tackles a timely topic and is interesting to read.

It is not entirely clear to me why the Authors name their proposal Bayesian hypernetworks. This seems to suggest that also the hypernetwork is infered using Bayesian inference, but if I understand correctly this is not the case.

I have some comments on novelty and realization of the experiments. In the positioning of the work in the literature, the Authors point out that hypernetworks have been proposed before, so it is not clear what is the actual novelty in the proposal. Is it the use of Real NVPs and IAFs as hypernetworks? These methods have been already proposed and extensively studied in the literature, and even if they have been adapted to be hypernetworks here, I believe that the novelty is fairly limited.

The experimental part is interesting as it explores a number of learning scenarios. However, I think that it would have been useful to add comparisons with standard variational inference (e.g., Graves, 2011) for deep networks to substantiate the claims that this approach underestimates uncertainty. I believe that this would strengthen the comparative evaluation.

I think the paper would have made a stronger case by including other approaches to approximate posteriors using generative models. For example, the variational Gaussian process paper sounds like an ideal method to include here.

[1] D. Tran, R. Ranganath, and D. M. Blei. Variational Gaussian process. arXiv preprint arXiv:1511.06499, 2015.

The evidence to be evaluated is:There is no single metric for how well a model captures uncertainty; to evaluate our model, we perform experiments on regularization (Section 4.2), active learning (Section 4.3), anomaly detection (Section 4.4), and detection of adversarial examples (Section 4.5). Active learning and anomaly detection problems make natural use of uncertainty estimates: In anomaly detection, higher uncertainty indicates a likely anomaly. In active learning, higher uncertainty indicates a greater opportunity for learning. Parameter uncertainty also has regularization benefits: integrating over the posterior creates an implicit ensemble.

Adversarial examples are an especially difficult kind of anomaly designed to fool a classifier, and finding effective defenses against adversarial attacks remains an open challenge in deep learning.

We now show that BHNs act as a regularizer, outperforming dropout and traditional mean field (BbB).

In our experiments, we find that BHNs perform on par with dropout on full datasets of MNIST and CIFAR10; furthermore, increasing the flexibility of the posterior by adding more coupling layers improves performance, especially compared with models with 0 coupling layers, which cannot model dependencies between the parameters.

Overall, warm-started BHNs suffered at the beginning of training, but outperformed all other methods for moderate to large numbers of acquisitions.

We demonstrate that Bayesian DNNs outperform their non-Bayesian counterparts. Just as in active learning, in anomaly detection, we use MC to estimate the predictive posterior, and use this to score datapoints.

This experiment shows the benefit of using scores that reflect dispersion in the posterior samples (such as mean standard deviation and BALD value) in Bayesian DNNs. Finally, we consider this same anomaly detection procedure as a novel tool for detecting adversarial examples. Our setup is similar to Li & Gal (2017) and Louizos & Welling (2017), where it is shown that when more perturbation is added to the data, model uncertainty increases and then drops. We use the Fast Gradient Sign method (FGS) (Goodfellow et al., 2014) for adversarial attack, and use one sample of our model to estimate the gradient.10 We find that, compared with dropout, BHNs are less confident on data points which are far from the data manifold. In particular, BHNs constructed with IAF consistently outperform RealNVP-BHNs and dropout in detecting adversarial examples and errors.

BHNs feature efficient training and sampling, and can express complicated multimodal distributions, thereby addressing issues of overconfidence present in simpler variational approximations. We present a method of parametrizing BHNs which allows them to scale successfully to real world tasks, and show that BHNs can offer significant benefits over simpler methods for Bayesian deep learning.

Input: Figure 7: Adversary detection with 32-sample estimate of gradient.

Input: Figure 5: Adversary detection: Horizontal axis is the step size of the FGS algorithm. While accuracy drops when more perturbation is added to the data (left), uncertainty measures also increase (first row). In particular, the BALD and Mean STD scores, which measure epistemic uncertainty, are strongly increasing for BHNs, but not for dropout. The second row and third row plots show results for adversary detection and error detection (respectively) in terms of the AUC of ROC (y-axis) with increasing perturbation along the x-axis. Gradient direction is estimated with one Monte Carlo sample of the weights/dropout mask.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
1. Evaluation Summary:
The claim states Bayesian hypernets provide "better defense against adversarial examples than dropout" and exhibit "competitive performance" on uncertainty tasks. The evidence shows BHNs (particularly IAF versions) consistently outperforming dropout in adversarial detection across multiple metrics (BALD, Entropy, etc.) as demonstrated in Figures 5 and 7. However, for other uncertainty tasks, the evidence only shows BHNs performing "on par" or "outperforming" in specific scenarios rather than demonstrating clear superiority across all cases. The claim is slightly overstated in generalizing the performance advantages beyond adversarial defense.

<score>2</score>

Justification:
- The claim about adversarial defense is well-supported by the figures showing BHNs (especially IAF) maintaining higher uncertainty scores compared to dropout when facing adversarial examples
- For other uncertainty tasks, the evidence shows competitive but not consistently superior performance:
  - "perform on par with dropout on full datasets"
  - "outperformed all other methods for moderate to large numbers of acquisitions" (specific scenario)
- The review comment raises valid concerns about limited comparisons to other methods and novelty, but these don't directly impact the claim's validity regarding performance
- While slightly optimistic in tone, the claim's core assertions are mostly supported by the evidence, with just minor overstatement


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In practice, Bayesian hypernets provide a better defense against adversarial examples than dropout, and also exhibit competitive performance on a suite of tasks which evaluate model uncertainty, including regularization, active learning, and anomaly detection.

The review comment to be evaluated is:

* Edit: I increased my rating to 6. The authors fixed the first error I pointed out below. Regarding the second point: I still think it is possible to take a limit of sigma -> 0 in MNF, which makes the methods very similar.

The authors propose a new method of defining approximate posteriors for use in Bayesian neural networks. The idea of using hypernetworks for Bayesian inference is compelling, and the authors show some promising first results. I see two issues, and would be willing to increase my rating if these were sufficiently addressed.

- The paper says it uses an "isotropic standard normal prior on the weights of the network". However, the stochastic part of the generated weights (i.e. the scales) is of a lower dimension than the weights. It seems to me this means that the KL divergence between prior and posterior is undefined, or infinite, as the posterior is only defined on a sub-manifold. What exactly is the loss term that is added to the training objective? And how is this justified?

- The instantiation of Bayesian hypernetworks that is used in experiments seems to be a special case of the method of multiplicative normalizing flows as proposed by Louizos and Welling and discussed in this paper. If the variances / sigmas are zero in the latter method, their approximation seems functionally equivalent to Bayesian hypernetworks (though with different parameterization). Is my understanding correct? If so, the novelty of the proposed method is limited.

The evidence to be evaluated is:There is no single metric for how well a model captures uncertainty; to evaluate our model, we perform experiments on regularization (Section 4.2), active learning (Section 4.3), anomaly detection (Section 4.4), and detection of adversarial examples (Section 4.5). Active learning and anomaly detection problems make natural use of uncertainty estimates: In anomaly detection, higher uncertainty indicates a likely anomaly. In active learning, higher uncertainty indicates a greater opportunity for learning. Parameter uncertainty also has regularization benefits: integrating over the posterior creates an implicit ensemble.

Adversarial examples are an especially difficult kind of anomaly designed to fool a classifier, and finding effective defenses against adversarial attacks remains an open challenge in deep learning.

We now show that BHNs act as a regularizer, outperforming dropout and traditional mean field (BbB).

In our experiments, we find that BHNs perform on par with dropout on full datasets of MNIST and CIFAR10; furthermore, increasing the flexibility of the posterior by adding more coupling layers improves performance, especially compared with models with 0 coupling layers, which cannot model dependencies between the parameters.

Overall, warm-started BHNs suffered at the beginning of training, but outperformed all other methods for moderate to large numbers of acquisitions.

We demonstrate that Bayesian DNNs outperform their non-Bayesian counterparts. Just as in active learning, in anomaly detection, we use MC to estimate the predictive posterior, and use this to score datapoints.

This experiment shows the benefit of using scores that reflect dispersion in the posterior samples (such as mean standard deviation and BALD value) in Bayesian DNNs. Finally, we consider this same anomaly detection procedure as a novel tool for detecting adversarial examples. Our setup is similar to Li & Gal (2017) and Louizos & Welling (2017), where it is shown that when more perturbation is added to the data, model uncertainty increases and then drops. We use the Fast Gradient Sign method (FGS) (Goodfellow et al., 2014) for adversarial attack, and use one sample of our model to estimate the gradient.10 We find that, compared with dropout, BHNs are less confident on data points which are far from the data manifold. In particular, BHNs constructed with IAF consistently outperform RealNVP-BHNs and dropout in detecting adversarial examples and errors.

BHNs feature efficient training and sampling, and can express complicated multimodal distributions, thereby addressing issues of overconfidence present in simpler variational approximations. We present a method of parametrizing BHNs which allows them to scale successfully to real world tasks, and show that BHNs can offer significant benefits over simpler methods for Bayesian deep learning.

Input: Figure 7: Adversary detection with 32-sample estimate of gradient.

Input: Figure 5: Adversary detection: Horizontal axis is the step size of the FGS algorithm. While accuracy drops when more perturbation is added to the data (left), uncertainty measures also increase (first row). In particular, the BALD and Mean STD scores, which measure epistemic uncertainty, are strongly increasing for BHNs, but not for dropout. The second row and third row plots show results for adversary detection and error detection (respectively) in terms of the AUC of ROC (y-axis) with increasing perturbation along the x-axis. Gradient direction is estimated with one Monte Carlo sample of the weights/dropout mask.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find this claim to be somewhat overstated. While the evidence does show that Bayesian hypernets (BHNs) outperform dropout in adversarial example detection (as shown in Figures 5 and 7 where BHN variants consistently show better uncertainty metrics), the claim about "competitive performance" across all uncertainty tasks is not fully supported. The evidence shows mixed results - BHNs perform "on par" with dropout on MNIST/CIFAR-10, and while they eventually outperform in active learning, they initially struggle ("suffered at the beginning of training"). Additionally, the reviewer raises valid concerns about the method's novelty and technical foundations.

<score>3</score>

Justification:
1. The claim about defense against adversarial examples is well-supported by the figures showing BHNs (especially IAF variant) consistently outperforming dropout across multiple uncertainty metrics.

2. The "competitive performance" claim is overstated because:
- The evidence shows only "on par" performance on basic tasks
- Initial struggles in active learning are downplayed
- The reviewer raises important technical concerns about the method's novelty and relationship to existing approaches
- Some results are presented without statistical significance tests or error bars
- Limited experimental details for some tasks make it difficult to fully validate the breadth of the claim

3. The phrasing implies consistent superiority across all tasks, when the evidence suggests more nuanced, task-dependent performance differences.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We here demonstrate the surprising power of a simple approach: Continual Learning with Experience And Replay (CLEAR).

The evidence to be evaluated is:Here, in CLEAR, we demonstrate that a mixture of replay data and fresh experience protects against catastrophic forgetting while also permitting fast learning, and performs better than either pure on-policy learning or pure off-policy learning from replay. We provide a thorough investigation and robust algorithm in CLEAR, conducting a wide variety of tests on both limited-size and unbounded buffers with complex RL tasks using state-of-the-art methods, and improve the stability of simple replay with the addition of behavioral cloning. CLEAR uses actor-critic training on a mixture of new and replayed experiences. In the case of replay experiences, two additional loss terms are added to induce behavioral cloning between the network and its past self. The motivation for behavioral cloning is to prevent network output on replayed tasks from drifting while learning new tasks. We penalize (1) the KL divergence between the historical policy distribution and the present policy distribution, (2) the L2 norm of the difference between the historical and present value functions.

We apply CLEAR in a distributed training context based on the Importance Weighted Actor-Learner Architecture (Espeholt et al., 2018). A single learning network is fed experiences (both novel and \[v_{s}:=V(h_{s})+\sum_{t=s}^{s+n-1}\gamma^{t-s}\left(\prod_{i=s}^{t-1}c_{i} \right)\delta_{t}V,\] where \(\delta_{t}V:=\rho_{t}\left(r_{t}+\gamma V(h_{t+1})-V(h_{t})\right)\), \(c_{i}:=\min(\bar{c},\frac{\pi_{\theta}(a_{i}|h_{i})}{\mu(a_{i}|h_{i})})\), and \(\rho_{t}=\min(\bar{\rho},\frac{\pi_{\theta}(a_{i}|h_{i})}{\mu(a_{i}|h_{i})})\), for constants \(\bar{c}\) and \(\bar{\rho}\).

The loss functions \(L_{\text{value}}\), \(L_{\text{policy-gradient}}\), and \(L_{\text{entropy}}\) are applied both for new and replay experiences. In addition, we add \(L_{\text{policy-cloning}}\) and \(L_{\text{value-cloning}}\) for replay experiences only. In general, our experiments use a 50-50 mixture of novel and replay experiences, though performance does not appear to be very sensitive to this ratio.

Catastrophic forgetting occurs when a task's performance goes down not as a result of incompatibility with another task but as a result of the second task overwriting it within the model. As we aim to illustrate, the two are independent phenomena, and while interference may happen, forgetting is ubiquitous.

By contrast, there is a large difference between either of the above modes of training and sequential training, where performance on a task decays immediately when training switches to another task - that is, catastrophic forgetting.

We here demonstrate the efficacy of CLEAR for diminishing catastrophic forgetting (Figure 2).

Our method effectively eliminates forgetting on all three tasks, while preserving overall training performance (see "Sequential" training in Figure 1 for reference). When the task switches, there is little, if any, dropoff in performance when using CLEAR, and the network picks up immediately where it left off once a task returns later in training. Without behavioral cloning, the mixture of new experience and replay still reduces catastrophic forgetting, though the effect is reduced.

We believe that 50-50 new-replay represents a good tradeoff, combining significantly reduced catastrophic forgetting

Just as Figure 2 shows the importance of behavioral cloning for maintaining past performance on a task, so this experiment shows that off-policy learning can actually increase performance from replay alone. Both ingredients are necessary for the success of CLEAR.

We find that all buffers perform well and conclude that it is possible to learn and reduce catastrophic forgetting even with a replay buffer that is significantly smaller than the total number of experiences.

In this experiment (Figure 5), we find that this is not a problem for CLEAR, relying as it does on a mixture of off- and on-policy learning. Specifically, we find the performance attained on a task is largely independent of the amount of data stored in the buffer and on the identities of the preceding tasks.

Notably, CLEAR succeeds in greatly reducing catastrophic forgetting for all tasks, and the effect on the probe task does not diminish as the probe task is introduced later on in the training sequence.

Finally, we compare our method to Progress & Compress (P&C) (Schwarz et al., 2018) and Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017), state-of-the-art methods for reducing catastrophic forgetting that, unlike replay, assume that the boundaries between different tasks are known (Figure 6).

In this case, we use CLEAR with a 75-25 balance of new-replay experience. We find that we obtain comparable performance to P&C and better performance than EWC, despite CLEAR being significantly simpler and agnostic to the boundaries between tasks.

We have described a simple but powerful approach for preventing catastrophic forgetting in continual learning settings. CLEAR uses on-policy learning on fresh experiences to adapt rapidly to new tasks, while using off-policy learning with behavioral cloning on replay experience to maintain and modestly enhance performance on past tasks.

Our method is simple, scalable, and practical; it takes advantage of the general abundance of memory and storage in modern computers and computing facilities. We believe that the broad applicability and simplicity of the approach make CLEAR a candidate "first line of defense" against catastrophic forgetting in many RL contexts.

A training episode (_unroll_) is generated and inserted into the actor's buffer. Reservoir sampling is used (see further details below) if the buffer has reached its maximum capacity. The actor then samples another unroll from the buffer. The new unroll and replay unroll are both fed into a queue of examples that are read by the learner. The actor waits until its last example in the queue has been read before creating another.

Each element of a batch is a pair (new unroll, replay unroll) from the queue provided by actors. Thus, the number of new unrolls and the number of replay unrolls both equal the entire batch size. Depending on the buffer utilization hyperparameter (see Figure 3), the learner uses a balance of new and replay examples, taking either the new unroll or the replay unroll from each pair.

Our replay buffer stores all information necessary for the V-Trace algorithm, namely the input presented by the environment, the output logits of the network, the value function output by the network, the action taken, and the reward obtained. Leveraging the distributed setup, the buffer is split among all actors equally, so that, for example, if the total buffer size were one million across a hundred actors, then each actor would have buffer capacity of ten thousand. All buffer sizes are measured in environment frames (not in numbers of unrolls), in keeping with the \(x\)-axis of our training plots. For baseline experiments, no buffer was used, while all other parameters and the network remained constant. Unless otherwise specified, the replay buffer was capped at half the number of environment frames on which the network is trained. This is by design - to show that even past the buffer capacity, replay continues to prevent catastrophic forgetting. When the buffer fills up, then new unrolls are added by reservoir sampling, so that the buffer at any given point contains a uniformly random sample of all unrolls up until the present time. Reservoir sampling is implemented as in Isele & Cosgun (2018) by having each unroll associated with a random number between 0 and 1. A threshold is initialized to 0 and rises with time so that the number of unrolls above the threshold is fixed at the capacity of the buffer. Each unroll is either stored or abandoned in its entirety; no unroll is partially stored, as this would preclude training.

We observe that, unlike in the original experiment (Figure 5), the performance obtained on the probe task natlab_varying_map_randomized deteriorates markedly as it appears later in the sequence of tasks. For later positions in the sequence, the probe task comprises a smaller percentage of replay experience, thereby impeding purely off-policy learning. This result underlines why CLEAR uses new experience, as well as replay, to allow rapid learning of new tasks.

Input: Figure 5: The DMLab task natlab varying map randomize (brown line) is presented at different positions within a cyclically repeating sequence of three other DMLab tasks. We find that (1) performance on the probe task is independent of its position in the sequence, (2) the effectiveness of CLEAR does not degrade as more experiences and tasks are introduced into the buffer.

Input: Figure 14: Alternative plot of the experiments shown in Figure 6, showing that CLEAR attains comparable or better performance than the more complicated methods Progress & Compress (P&C) and Elastic Weight Consolidation (EWC), which also require information about task boundaries, unlike CLEAR.

Input: Figure 13: Quantitative comparison of the final cumulative performance between baseline (standard training), CLEAR, Elastic Weight Consolidation (EWC), and Progress & Compress (P&C) (see Figure 14 below). Overall, CLEAR performs comparably to or better than P&C, and significantly better than EWC and baseline.

Input: Figure 1: Separate, simultaneous, and sequential training: the x-axis denotes environment steps summed across all tasks and the y-axis episode score. In “Sequential”, thick line segments are used to denote the task currently being trained, while thin segments are plotted by evaluating performance without learning. In simultaneous training, performance on explore object locations small is higher than in separate training, an example of modest constructive interference. In sequential training, tasks that are not currently being learned exhibit very dramatic catastrophic forgetting. (See Appendix C for a different plot of these data.)

Input: Figure 12: Alternative plot of the experiments shown in Figure 4, showing that reduced-size buffers still allow CLEAR to achieve essentially the same performance.

Input: Figure 11: Alternative plot of the experiments shown in Figure 3, comparing performance between using CLEAR with 75-25 new-replay experience, 50-50 new-replay experience, and 100% replay experience. An equal balance of new and replay experience seems to represent a good tradeoff between stability and plasticity, while 100% replay reduces forgetting but lowers performance overall.

Input: Figure 10: Alternative plot of the experiments shown in Figure 2, showing how applying CLEAR when training on sequentially presented tasks gives almost the same results as training on all tasks simultaneously (compare to sequential and simultaneous training in Figure 9 above). Applying CLEAR without behavioral cloning also yields decent results.

Input: Figure 2: Demonstration of CLEAR on three DMLab tasks, which are trained cyclically in sequence. CLEAR reduces catastrophic forgetting so significantly that sequential tasks train almost as well as simultaneous tasks (see Figure 1). When the behavioral cloning loss terms are ablated, there is still reduced forgetting from off-policy replay alone. As above, thicker line segments are used to denote the task that is currently being trained. (See Appendix C for a different plot of these data.)

Input: Figure 6: Comparison of CLEAR to Progress & Compress (P&C) and Elastic Weight Consolidation (EWC). We find that CLEAR demonstrates comparable or greater performance than these methods, despite being significantly simpler and not requiring any knowledge of boundaries between tasks. (See Appendix C for a different plot of these data.)

Input: Figure 3: Comparison between different proportions of new and replay examples while cycling training among three tasks. We observe that with a 75-25 new-replay split, CLEAR eliminates most, but not all, catastrophic forgetting. At the opposite extreme, 100% replay prevents forgetting at the expense of reduced overall performance (with an especially noticeable reduction in early performance on the task rooms keys doors puzzle). A 50-50 split represents a good tradeoff between these extremes. (See Appendix C for a different plot of these data.)

Input: Figure 14: Alternative plot of the experiments shown in Figure 6, showing that CLEAR attains comparable or better performance than the more complicated methods Progress & Compress (P&C) and Elastic Weight Consolidation (EWC), which also require information about task boundaries, unlike CLEAR.

Input: Figure 13: Quantitative comparison of the final cumulative performance between baseline (standard training), CLEAR, Elastic Weight Consolidation (EWC), and Progress & Compress (P&C) (see Figure 14 below). Overall, CLEAR performs comparably to or better than P&C, and significantly better than EWC and baseline.

Input: Figure 12: Alternative plot of the experiments shown in Figure 4, showing that reduced-size buffers still allow CLEAR to achieve essentially the same performance.

Input: Figure 11: Alternative plot of the experiments shown in Figure 3, comparing performance between using CLEAR with 75-25 new-replay experience, 50-50 new-replay experience, and 100% replay experience. An equal balance of new and replay experience seems to represent a good tradeoff between stability and plasticity, while 100% replay reduces forgetting but lowers performance overall.

Input: Figure 10: Alternative plot of the experiments shown in Figure 2, showing how applying CLEAR when training on sequentially presented tasks gives almost the same results as training on all tasks simultaneously (compare to sequential and simultaneous training in Figure 9 above). Applying CLEAR without behavioral cloning also yields decent results.

Input: Figure 8: Quantitative comparison of the final cumulative performance between standard training (“Sequential (no CLEAR)”) and various versions of CLEAR (see Figures 9, 10, 11, and 12 below) on a cyclically repeating sequence of DMLab tasks. We also include the results of training on each individual task with a separate network (“Separate”) and on all tasks simultaneously (“Simultaneous”) instead of sequentially. As described in Section 4.1, these situations represent no-forgetting scenarios and thus present upper bounds on the performance expected in a continual learning setting, where tasks are presented sequentially. Remarkably, CLEAR achieves performance comparable to “Separate” and “Simultaneous”, demonstrating that forgetting is virtually eliminated.

Input: Figure 4: Comparison of the performance of training with different buffer sizes, using reservoir sampling to ensure that each buffer stores a uniform sample from all past experience. We observe only minimal difference in performance, with the smallest buffer (storing 1 in 200 experiences) demonstrating some catastrophic forgetting. (See Appendix C for a different plot of these data.)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review, the claim "We here demonstrate the surprising power of a simple approach: Continual Learning with Experience And Replay (CLEAR)" appears somewhat overstated. While the evidence shows CLEAR is effective at reducing catastrophic forgetting and performs comparably or better than existing methods like P&C and EWC, describing it as having "surprising power" overstates its capabilities. The results demonstrate solid performance but not necessarily surprising or revolutionary improvements. The method combines known techniques (replay, behavioral cloning) in a straightforward way. The evidence supports CLEAR being a practical and effective approach, but the dramatic language in the claim isn't fully justified by the results shown.

<score>3</score>