Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We then analyze the weakness of existing approaches, and propose a novel regularization method named "Activity Regularization" (AR), which alleviates forgetting meanwhile keeping model's plasticity to acquire new knowledge.

The review comment to be evaluated is:

The authors proposed a new regularizer for continual learning to tackle the catastrophic forgetting problem. The proposed method minimizes the KL-divergence between the prediction of previous models and current models on the stored samples of previous tasks. The idea is straightforward and sounds technical. Experiments show the effectiveness of the methods compared to state-of-the-art. Although the idea sounds interesting and the experiments look promising, the novelty of the paper seems to be limited. In addition, the experiments setting needs to be improved as well. In the following, you have detailed comments.

1. It seems to be an extension of Learning without Forgetting (LwF) Li & Hoiem 2017 with simply on the examples of previous tasks in the memory. LwF only regularizes on the current task. It is not clear what is the difference between the proposed method and LwF except this.
2. The authors fixed many critical hyper-parameters: temperature(5), learning rate(0.05), epochs(10). The author should report the results for all methods with these hyper-parameters chosen on the validation set.
3. The authors presented how they split the training and testing data. Please be clear how you split the validation set.
4. The authors argued that sample quality does not affect the proposed methods. Then the authors should show the variance from different random sampling.

The evidence to be evaluated is:To avoid forgetting previous knowledge, when learning task \(\mathcal{T}_{t}\), the learner \(\mathbf{\theta}\) minimize the multi-task learning objective as \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\lambda_{t-1}\mathcal{L}_{t-1}(\mathbf{\theta},\mathcal{D}_{t-1})+\ldots+ \lambda_{1}\mathcal{L}_{1}(\mathbf{\theta},\mathcal{D}_{1}), \tag{1}\] where the loss \(\mathcal{L}_{i}(\mathbf{\theta},\mathcal{D}_{i})\) is usually the negative log-likelihood \(-\sum_{\mathbf{x},y\in\mathcal{D}_{i}}\log p(y|\mathbf{x};\mathbf{\theta})\) of the corresponding training data and \(\lambda_{i}\) regulates the importance of task \(\mathcal{T}_{i}\) to all the tasks observed so far. In continual learning, at time \(t\), we do not have access to the data \(\mathcal{D}_{k<t}\) to evaluate the loss \(\mathcal{L}_{k<t}\), thus, we need to approximate the past losses \(\mathcal{L}_{k<t}(\mathbf{\theta},\mathcal{D}_{k})\) when minimizing \(\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_{t})\).

\[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\sum_{i<t}\lambda_{k}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}(\bm {\theta}-\mathbf{\theta}_{i}^{*}), \tag{3}\]

For example, Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes a form of distillation loss by minimizing the KL-divergence of past models and current model on the newly observed tasks. By changing the losses \(\mathcal{L}_{k<t}\), we can recover different solutions for continual learning in the literature. While EWC and GEM are two popular methods for continual learning, we argue that they are sensitive to misclassified samples selected in the memory or used to estimate the Fisher information.

In general, if we have too many misclassified samples or label noise, the decision boundary on the previous task might be altered despite minimizing the EWC or GEM objective. This phenomenon happens based on an assumption that we usually do not train the model to have 0 error on the training set to avoid overfitting and we only use a relatively small data portion to estimate the Fisher or store as memory. Therefore, EWC and GEM might not correctly penalize the current model from changing the past tasks' optimized decision boundaries. To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks. This can also be interpreted as distilling the knowledge from \(\mathbf{\theta}_{i}^{*}\) to \(\mathbf{\theta}\) on the corresponding data \(\mathcal{D}_{i}\). In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models. Moreover, the number of regularizers for each task is equal to the number of previously observed tasks, which may considerably restrict the model's _plasticity_ to acquire new knowledge. That is, by enforcing too many constraints on the current model for reducing forgetting, we sacrifice its ability to learn new information.

```**Algorithm 1** Deterministic Activity Regularization (DAR) algorithm for Continual Learning. To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory.

As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability.

The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\) 9:\(\mathbf{\theta}\leftarrow\mathbf{\theta}-\alpha\cdot\text{SGD}(\mathbf{\theta},g_{\mathbf{ \theta}})\) 10:return\(\mathbf{\theta}\) ```**Algorithm 2** Stochastic Activity Regularization (SAR) algorithm for Continual Learning.

The goal of continual learning is to build a learner that can continuously learn from a stream of tasks while still maintain its previously acquired knowledge. Prior works can be broadly categorized into 4 main approaches: (1) structural regularization, (2) functional regularization, (3) Bayesian learning and (4) ensemble learning. **Structural regularization** approaches employ penalties on the network's parameters, encourage the important parameters of previous tasks to not change when learning the new task. The parameter importance can be estimated by calculating the Fisher information as in EWC (Kirkpatrick et al., 2017) or directly measure how changes in each parameter will affect the change in the loss function as proposed in SI (Zenke et al., 2017).

**Functional regularization** methods employ the regularizer on the model's output rather than its parameters. Different from structural regularization, functional regularization aims to preserve the learned input-output mapping function of the network. Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes to minimize the difference in KL-divergence of the current and previous models on the new tasks, which is a form of knowledge distillation penalty. Similarly, Less-forgetting learning (LF) (Jung et al., 2019) proposes to minimize the \(\ell_{2}\) norm of the two prediction in stead of KL-divergence as LwF. Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation. Different from LwF that distills previous models on the current data, which contradicts the goal of knowledge distillation since past models are not trained on the new data, they provide noise to the current model. Our approaches, DAR and SAR, distills past models on their correctly trained data, thus maintaining the past knowledge for new models.

Finally, **ensemble learning** techniques addresses catastrophic forgetting by having a dedicated sub-network for each task. The learning of sub-networks can either be explicit or implicit. In explicit ensemble learning, each sub-network is either newly initialized whenever a new tasks arrives as in Progressive Neural Network (Rusu et al., 2016) or searched from a big network as proposed in Pathner (Fernando et al., 2017). After training a task, the corresponding sub-network is frozen and will not allowed to change, thus completely immune to catastrophic forgetting. In contrast, implicit ensemble learning can be considered as a hybrid method of explicit ensemble with other types of regularization. Dynamically expandable networks (Yoon et al., 2018) proposes to learn sub-networks in a pre-initialized network through sparsity, adding new neurons whenever needed. Catastrophic forgetting is avoided by using a form of structural regularizer. Ensemble learning approaches are usually resistant to catastrophic forgetting as the cost of either unbounded growth of model's size or cannot be applied to different network architectures. In this work, we propose an approximated multi-task learning framework to unify both structural and functional regularization, in which our proposed activity regularization is a special case.

From the results, EWC perform the worst although requiring the most "memory unit". SI performs better than EWC thanks to its Fisher approximation strategy on the whole data. GEM attains 91.8% accuracy, higher than both EWC, SI and VCL due the its strong constraints.

Both DAR and SAR outperforms the baselines by large margins, achieving accuracy of 94.9% and 94.8% respectively. We also investigate the scalability of our methods when more samples are available in the memory.

Both DAR and SAR are consistently better than GEM at all memory size.

At 2500 samples, both DAR and SAR achieves over 96% accuracy, showing great scalability. Across all methods, performance consistently improves when the memory size increases. When only a small amount of samples are available in the memory, e.g. 200 and 500, GEM may overfit to these samples, leading to the performance degrade on the whole past data sets. As more samples are stored, the memory can better represent the past tasks, results in performance improvement.

noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks.

Finally, both DAR and VAR consistently perform better than the baselines, achieving 97.6% and 97.7% accuracy respectively.

We report the averaged accuracy after 10 tasks in fig. 1(a), where EWC and SI are not as effective as alleviating forgetting as in previous experiments because this benchmark is more challenging.

We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework. We further analyze the weakness of two popular methods: EWC and GEM, showing that they are sensitive to mistakes and noise in the Fisher or memory. To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

We further improve DAR with a stochastic sampling regularizer (SAR), which balances the model's stability and plasticity, allowing the model to acquire new knowledge, alleviate forgetting and has greater scalability.

Finally, we show that DAR and SAR can be applied on both MLP and CNN, effectively alleviate catastrophic forgetting.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim, evidence, and review comments, I'll evaluate whether the claim is well-stated or overstated.

Summary of evaluation:
The claim states that the authors analyze weaknesses of existing approaches and propose Activity Regularization (AR) to alleviate forgetting while maintaining plasticity. The evidence shows detailed analysis of EWC and GEM's weaknesses regarding noise sensitivity and memory issues. The experimental results demonstrate DAR and SAR's effectiveness across different settings. However, as noted in the review comments, the novelty appears limited as it seems to be an extension of LwF, and critical experimental details like hyperparameter selection and validation split are missing.

Justification:
1. The paper does analyze weaknesses of existing approaches with mathematical and empirical evidence
2. The proposed AR method shows improved performance in experiments
3. However, the claim may be somewhat overstated since:
   - The method appears to be an incremental improvement over LwF rather than completely novel
   - Some experimental rigor is lacking (fixed hyperparameters, validation split details missing)
   - Claims about plasticity are not thoroughly validated with varying parameters

<score>3</score>

The claim receives a score of 3 (moderate overstatement) because while the core technical contributions are supported by evidence, the novelty and comprehensiveness of the analysis are somewhat overstated given the limitations pointed out in the review.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting.

The evidence to be evaluated is:To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability.

Although more sophisticated methods such as the greedy \(K\)-center algorithm can be used to select data points that spread through out the input space (Nguyen et al., 2018), in this work, even random sampling works well with our methods and can achieve state-of-the-art results.

Table 1 reports the averaged test accuracies on all 10 tasks at the end of training. From the results, EWC perform the worst although requiring the most "memory unit". SI performs better than EWC thanks to its Fisher approximation strategy on the whole data. GEM attains 91.8% accuracy, higher than both EWC, SI and VCL due the its strong constraints.

Both DAR and SAR outperforms the baselines by large margins, achieving accuracy of 94.9% and 94.8% respectively.

Both DAR and SAR are consistently better than GEM at all memory size.

At 2500 samples, both DAR and SAR achieves over 96% accuracy, showing great scalability.

In both cases, our methods show advantage over GEM, maintaining performance on previous task by enforcing model's consistency.

Table 2 reports the final averaged accuracy after learning five tasks.

This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks. As a result, SI performs significantly better than both EWC, SI and VCL alone, achieving 94% accuracy. When combined with a coreset VCL further improves the performance to 96%. Finally, both DAR and VAR consistently perform better than the baselines, achieving 97.6% and 97.7% accuracy respectively.

We report the averaged accuracy after 10 tasks in fig. 1(a), where EWC and SI are not as effective as alleviating forgetting as in previous experiments because this benchmark is more challenging. DAR and SAR achieve slightly higher averaged accuracy compare to GEM.

Except the first task where all models are the same, SAR achieves slightly higher accuracy than DAR on 6 out of 9 remaining tasks. Overall SAR performs better than DAR by a small margin, showing that it can achieve similar results compare to DAR and has more scalability. We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

Through extensive experiments on popular continual learning benchmarks, we show that DAR and SAR consistently outperforms state-of-the-art methods. We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance. Finally, we show that DAR and SAR can be applied on both MLP and CNN, effectively alleviate catastrophic forgetting. \begin{table}
\begin{tabular}{c c c c c} \hline \hline Method & Memory type & Memory Size & Setting & Accuracy \\ \hline EWC & model & 1.8m & \(\lambda=100,N=200\) & 0.795 \\ SI & model & 0.2m & \(c=0.5\) & 0.860 \\ GEM & sample & 1.5m & - & 0.918 \\ VCL* & none & 0 & - & 0.900 \\ VCL + Coreset* & sample & 1.5m & - & 0.930 \\
**DAR** & **sample** & **1.5m** & \(\lambda=10,\tau=5\) & **0.949** \\
**SAR** & **sample** & **1.5m** & \(\lambda=100,\tau=5\) & **0.948** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

 \begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline Method & EWC & GEM & SI & VCL* & VCL+Coreset* & **DAR** & **SAR** \\ \hline Accuracy & 0.710 & 0.810 & 0.940 & 0.920 & 0.960 & **0.976** & **0.977** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

Input: Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Figure 1: Comparison of different memory size on the permuted MNIST experiments.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim "Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting" to be moderately overstated. While the evidence shows DAR/SAR performing better than baselines across multiple experiments (Tables 1&2, Figures 1&2), the improvements are often modest (1-3% gains). The term "effectively overcomes" is too strong since the methods show some performance degradation over tasks (Fig 1). The experiments, while comprehensive across different datasets, don't fully justify such definitive language.

<score>3</score>

Justification:
1. The evidence shows consistent but modest improvements:
- Table 1: 94.9%/94.8% vs 91.8% for GEM
- Table 2: 97.6%/97.7% vs 96% for VCL+Coreset

2. "Effectively overcomes forgetting" is overstated:
- Figure 1 shows declining performance across tasks for both DAR/SAR
- Performance degradation is reduced but not "overcome"

3. "Extensive experiments" is reasonably supported:
- Tests on MNIST, notMNIST, CIFAR-100
- Multiple baseline comparisons
- Various memory sizes tested

4. More measured claims about "reducing" rather than "overcoming" forgetting would better match the evidence.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting.

The review comment to be evaluated is:

This paper proposes an approach to mitigate catastrophic forgetting in supervised learning by regularizing activations. The paper views previous techniques (EWC, SI, and GEM) under a multi-task learning lens, and then proposes an additional loss term to minimise the KL between activations from previous and current models, on previous tasks - this is based on a memory which stores some previous samples and their corresponding activations.

I think it is a simple and intuitive approach and a well-written paper. Unfortunately I have a number of concerns that I think preclude publication in the current state.

First, in terms of related work, I believe this is very similar to Learning without forgetting (LwF), with the difference that the KL-divergence is computed on samples kept from the previous tasks. This is briefly mentioned in the paper, but I think it needs to be made more explicit, and LwF should be a baseline in the experiments to clearly indicate the benefit of keeping this data. There is also a relationship to EWC: given the connection between the Fisher information and KL, it can be viewed as minimising the KL divergence in parameter space, rather than in activation space (which is the case here). Also note that EWC uses the true Fisher rather than the empirical, contrary to the derivation in equation (2).
There are also a number of papers that haven’t been cited in the related work [1][2][3][4].

Second, I think the motivation in Section 3.1 could be more convincing. Most importantly, it’s not clear to me that the decision boundary *shouldn’t* change for previously misclassified examples, as this could be an opportunity for backwards transfer.
Further, I don’t think the point in the last paragraph about having a small data portion is relevant, since they are from the same data distribution, and we would expect misclassified samples to be in the same (low) frequency in Fisher estimation as overall. I think the point of this paragraph is just that it is important to consider the entire predictive distribution of previous tasks rather than the probability of the correct class, so this should be stated more clearly and then justified.

Finally, I think the experimental justification could be improved as well. Beyond permuted MNIST (which it has been argued is not as useful as other baselines [4]), only the final performance on split notMNIST / CIFAR-100 is reported. Some comments and questions:
- The accuracies of EWC (and possibly SI) in the table are worse than reported in previous work (eg. [1]), so I think this needs to be examined.
- What is the fine-tuning baseline (I don't believe it is actually clearly defined)? How can it be so low in figure 2a but better in 2b?
- I think plots over time (performance on all tasks) would be much more useful than the final performance in Table 2 and Fig 2.
- Errors and error bars would be beneficial for all results.
- Table 1 should also include the references provided.

Some other comments and questions:
- Compared to eqn (2), eqn (6) is missing the ½ constant.
- Typos in section 5.3: "SI performs better than SI", and VAR instead of SAR.
- Section 2, unclear of meaning of "coined with the likelihood" (should this be “coincide”?)
- The first line should be “Humans have the ability to learn...” In general, I think the introduction could use another proofread for grammar and readability as I saw a few minor things.

[1] Nguyen, Cuong V., et al. "Variational Continual Learning." ICLR, 2018.
[2] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." ICML, 2018.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." NIPS, 2017.
[4] Farquhar, Sebastian, and Yarin Gal. "Towards Robust Evaluations of Continual Learning." arXiv, 2018.

The evidence to be evaluated is:To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability.

Although more sophisticated methods such as the greedy \(K\)-center algorithm can be used to select data points that spread through out the input space (Nguyen et al., 2018), in this work, even random sampling works well with our methods and can achieve state-of-the-art results.

Table 1 reports the averaged test accuracies on all 10 tasks at the end of training. From the results, EWC perform the worst although requiring the most "memory unit". SI performs better than EWC thanks to its Fisher approximation strategy on the whole data. GEM attains 91.8% accuracy, higher than both EWC, SI and VCL due the its strong constraints.

Both DAR and SAR outperforms the baselines by large margins, achieving accuracy of 94.9% and 94.8% respectively.

Both DAR and SAR are consistently better than GEM at all memory size.

At 2500 samples, both DAR and SAR achieves over 96% accuracy, showing great scalability.

In both cases, our methods show advantage over GEM, maintaining performance on previous task by enforcing model's consistency.

Table 2 reports the final averaged accuracy after learning five tasks.

This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks. As a result, SI performs significantly better than both EWC, SI and VCL alone, achieving 94% accuracy. When combined with a coreset VCL further improves the performance to 96%. Finally, both DAR and VAR consistently perform better than the baselines, achieving 97.6% and 97.7% accuracy respectively.

We report the averaged accuracy after 10 tasks in fig. 1(a), where EWC and SI are not as effective as alleviating forgetting as in previous experiments because this benchmark is more challenging. DAR and SAR achieve slightly higher averaged accuracy compare to GEM.

Except the first task where all models are the same, SAR achieves slightly higher accuracy than DAR on 6 out of 9 remaining tasks. Overall SAR performs better than DAR by a small margin, showing that it can achieve similar results compare to DAR and has more scalability. We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

Through extensive experiments on popular continual learning benchmarks, we show that DAR and SAR consistently outperforms state-of-the-art methods. We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance. Finally, we show that DAR and SAR can be applied on both MLP and CNN, effectively alleviate catastrophic forgetting. \begin{table}
\begin{tabular}{c c c c c} \hline \hline Method & Memory type & Memory Size & Setting & Accuracy \\ \hline EWC & model & 1.8m & \(\lambda=100,N=200\) & 0.795 \\ SI & model & 0.2m & \(c=0.5\) & 0.860 \\ GEM & sample & 1.5m & - & 0.918 \\ VCL* & none & 0 & - & 0.900 \\ VCL + Coreset* & sample & 1.5m & - & 0.930 \\
**DAR** & **sample** & **1.5m** & \(\lambda=10,\tau=5\) & **0.949** \\
**SAR** & **sample** & **1.5m** & \(\lambda=100,\tau=5\) & **0.948** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

 \begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline Method & EWC & GEM & SI & VCL* & VCL+Coreset* & **DAR** & **SAR** \\ \hline Accuracy & 0.710 & 0.810 & 0.940 & 0.920 & 0.960 & **0.976** & **0.977** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

Input: Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Figure 1: Comparison of different memory size on the permuted MNIST experiments.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review of the claim "Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting," I find this claim to be moderately overstated based on the evidence and reviewer comments.

While the evidence shows that DAR and SAR do achieve higher accuracy than baseline methods on several benchmarks (e.g., 94.8-94.9% vs 91.8% for GEM on permuted MNIST), the improvements are relatively modest. The reviewer raises valid concerns about:
- Limited experimental validation beyond permuted MNIST
- Missing error bars and statistical significance
- Lack of comparison to similar methods like LwF
- Lower baseline performances than reported in previous work
- Missing temporal performance plots

The evidence shows improvement but doesn't fully support "effectively overcomes" catastrophic forgetting.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting.

The review comment to be evaluated is:

The paper addresses the problem of continual learning from a sequence of supervised tasks. The main contribution of the paper are the following. The work:
* puts the problem in uniform general framework in which many of the state-of-the-art methods fit
* identifies some drawbacks of some the state of the art methods (namely EWC- Kirkpatrick et al., 2017 and GEM-Lopez-Paz et al., 2017) .
* proposes two versions of an approach to address these drawbacks
The main identified problem in EWC and GEM is that both methods result in a zero penalty when both the previous and current models misclassify a sample, even if they make different mistakes.
To solve this problem, the authors propose to keep a memory of randomly sampled data from the previous task, and distil the knowledge from the optima of the previous tasks to the current one using a KL penalty. The first version considers computing this penalty from the whole sequence of tasks, while the second randomly selects one task at each iteration and uses it to compute the penalty, paying some overall accuracy for plasticity.

While the paper is clear and well written, I have some concerns about it's quality, originality and significance.

Originality: The work seems to me to be very related to LwF. The main difference is that while LwF uses only the new data for the KL penalty, this paper keeps a memory of previously seen data to compute this loss.

Significance: While the way the authors approached the problem seems well structured and motivated, and is based on a sound observation, the authors limited themselves to experiments where the task data have similar structure. I am not sure how significant the improvement of the method would be in the more challenging and realistic setting where the data comes from different domains.  I am more specifically skeptical about the stochastic version of the algorithm in that case.

Quality:
* The proposed algorithm stores some data from previous tasks along with the outputs of the corresponding optimal model. While knowledge distillation as proposed would result in a non zero penalty when the new model makes different prediction, it still doesn't take advantage of the new information to probably correct the previous models prediction when it is wrong. Why not keeping the ground truth labels instead? This won't increase the memory requirement, and may give better results. It would be interesting to compare against such a method.
* I think selecting the samples to keep in memory randomly could to be suboptimal.  Other selection methods can be considered. A previous work:  iCaRL: Incremental Classifier and Representation Learning, Rebuffi et al. 2017, gives way to select representative samples. It would be interesting to see the effect of such a selection on the results.

Overall, while the paper doesn't present any significant flaw, it doesn't add much to the continual learning literature either, which explains my rating.

The evidence to be evaluated is:To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability.

Although more sophisticated methods such as the greedy \(K\)-center algorithm can be used to select data points that spread through out the input space (Nguyen et al., 2018), in this work, even random sampling works well with our methods and can achieve state-of-the-art results.

Table 1 reports the averaged test accuracies on all 10 tasks at the end of training. From the results, EWC perform the worst although requiring the most "memory unit". SI performs better than EWC thanks to its Fisher approximation strategy on the whole data. GEM attains 91.8% accuracy, higher than both EWC, SI and VCL due the its strong constraints.

Both DAR and SAR outperforms the baselines by large margins, achieving accuracy of 94.9% and 94.8% respectively.

Both DAR and SAR are consistently better than GEM at all memory size.

At 2500 samples, both DAR and SAR achieves over 96% accuracy, showing great scalability.

In both cases, our methods show advantage over GEM, maintaining performance on previous task by enforcing model's consistency.

Table 2 reports the final averaged accuracy after learning five tasks.

This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks. As a result, SI performs significantly better than both EWC, SI and VCL alone, achieving 94% accuracy. When combined with a coreset VCL further improves the performance to 96%. Finally, both DAR and VAR consistently perform better than the baselines, achieving 97.6% and 97.7% accuracy respectively.

We report the averaged accuracy after 10 tasks in fig. 1(a), where EWC and SI are not as effective as alleviating forgetting as in previous experiments because this benchmark is more challenging. DAR and SAR achieve slightly higher averaged accuracy compare to GEM.

Except the first task where all models are the same, SAR achieves slightly higher accuracy than DAR on 6 out of 9 remaining tasks. Overall SAR performs better than DAR by a small margin, showing that it can achieve similar results compare to DAR and has more scalability. We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

Through extensive experiments on popular continual learning benchmarks, we show that DAR and SAR consistently outperforms state-of-the-art methods. We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance. Finally, we show that DAR and SAR can be applied on both MLP and CNN, effectively alleviate catastrophic forgetting. \begin{table}
\begin{tabular}{c c c c c} \hline \hline Method & Memory type & Memory Size & Setting & Accuracy \\ \hline EWC & model & 1.8m & \(\lambda=100,N=200\) & 0.795 \\ SI & model & 0.2m & \(c=0.5\) & 0.860 \\ GEM & sample & 1.5m & - & 0.918 \\ VCL* & none & 0 & - & 0.900 \\ VCL + Coreset* & sample & 1.5m & - & 0.930 \\
**DAR** & **sample** & **1.5m** & \(\lambda=10,\tau=5\) & **0.949** \\
**SAR** & **sample** & **1.5m** & \(\lambda=100,\tau=5\) & **0.948** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

 \begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline Method & EWC & GEM & SI & VCL* & VCL+Coreset* & **DAR** & **SAR** \\ \hline Accuracy & 0.710 & 0.810 & 0.940 & 0.920 & 0.960 & **0.976** & **0.977** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

Input: Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Figure 1: Comparison of different memory size on the permuted MNIST experiments.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my analysis of the evidence and reviewer comments, the claim "Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting" is somewhat overstated.

While the results do show improvements over baselines in specific scenarios (e.g., permuted MNIST and notMNIST), the improvements are often modest (1-3% better) and come with caveats. The reviewer raises valid concerns about limited experimental settings, lack of cross-domain testing, and suboptimal sample selection methods. The claim of "effectively overcomes" catastrophic forgetting is particularly strong given that performance still degrades over tasks (as shown in Figure 1) and the reviewer notes potential limitations in more challenging scenarios.

<score>4</score>

Justification:
1. The experiments, while comprehensive within their scope, are limited to similar domain tasks (as noted by reviewer)
2. Performance improvements are modest:
- Table 1: 94.9% vs 91.8% for GEM
- Table 2: 97.7% vs 96.0% for VCL+Coreset
3. Figure 1 shows performance still declines over tasks, indicating forgetting isn't fully "overcome"
4. Reviewer highlights important limitations:
- Limited to similar domain tasks
- Random sampling may be suboptimal
- Lack of testing on cross-domain scenarios
5. The wording "effectively overcomes" implies complete solution rather than incremental improvement


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting.

The review comment to be evaluated is:

The authors proposed a new regularizer for continual learning to tackle the catastrophic forgetting problem. The proposed method minimizes the KL-divergence between the prediction of previous models and current models on the stored samples of previous tasks. The idea is straightforward and sounds technical. Experiments show the effectiveness of the methods compared to state-of-the-art. Although the idea sounds interesting and the experiments look promising, the novelty of the paper seems to be limited. In addition, the experiments setting needs to be improved as well. In the following, you have detailed comments.

1. It seems to be an extension of Learning without Forgetting (LwF) Li & Hoiem 2017 with simply on the examples of previous tasks in the memory. LwF only regularizes on the current task. It is not clear what is the difference between the proposed method and LwF except this.
2. The authors fixed many critical hyper-parameters: temperature(5), learning rate(0.05), epochs(10). The author should report the results for all methods with these hyper-parameters chosen on the validation set.
3. The authors presented how they split the training and testing data. Please be clear how you split the validation set.
4. The authors argued that sample quality does not affect the proposed methods. Then the authors should show the variance from different random sampling.

The evidence to be evaluated is:To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability.

Although more sophisticated methods such as the greedy \(K\)-center algorithm can be used to select data points that spread through out the input space (Nguyen et al., 2018), in this work, even random sampling works well with our methods and can achieve state-of-the-art results.

Table 1 reports the averaged test accuracies on all 10 tasks at the end of training. From the results, EWC perform the worst although requiring the most "memory unit". SI performs better than EWC thanks to its Fisher approximation strategy on the whole data. GEM attains 91.8% accuracy, higher than both EWC, SI and VCL due the its strong constraints.

Both DAR and SAR outperforms the baselines by large margins, achieving accuracy of 94.9% and 94.8% respectively.

Both DAR and SAR are consistently better than GEM at all memory size.

At 2500 samples, both DAR and SAR achieves over 96% accuracy, showing great scalability.

In both cases, our methods show advantage over GEM, maintaining performance on previous task by enforcing model's consistency.

Table 2 reports the final averaged accuracy after learning five tasks.

This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks. As a result, SI performs significantly better than both EWC, SI and VCL alone, achieving 94% accuracy. When combined with a coreset VCL further improves the performance to 96%. Finally, both DAR and VAR consistently perform better than the baselines, achieving 97.6% and 97.7% accuracy respectively.

We report the averaged accuracy after 10 tasks in fig. 1(a), where EWC and SI are not as effective as alleviating forgetting as in previous experiments because this benchmark is more challenging. DAR and SAR achieve slightly higher averaged accuracy compare to GEM.

Except the first task where all models are the same, SAR achieves slightly higher accuracy than DAR on 6 out of 9 remaining tasks. Overall SAR performs better than DAR by a small margin, showing that it can achieve similar results compare to DAR and has more scalability. We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

Through extensive experiments on popular continual learning benchmarks, we show that DAR and SAR consistently outperforms state-of-the-art methods. We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance. Finally, we show that DAR and SAR can be applied on both MLP and CNN, effectively alleviate catastrophic forgetting. \begin{table}
\begin{tabular}{c c c c c} \hline \hline Method & Memory type & Memory Size & Setting & Accuracy \\ \hline EWC & model & 1.8m & \(\lambda=100,N=200\) & 0.795 \\ SI & model & 0.2m & \(c=0.5\) & 0.860 \\ GEM & sample & 1.5m & - & 0.918 \\ VCL* & none & 0 & - & 0.900 \\ VCL + Coreset* & sample & 1.5m & - & 0.930 \\
**DAR** & **sample** & **1.5m** & \(\lambda=10,\tau=5\) & **0.949** \\
**SAR** & **sample** & **1.5m** & \(\lambda=100,\tau=5\) & **0.948** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

 \begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline Method & EWC & GEM & SI & VCL* & VCL+Coreset* & **DAR** & **SAR** \\ \hline Accuracy & 0.710 & 0.810 & 0.940 & 0.920 & 0.960 & **0.976** & **0.977** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Table 1: Average test accuracy of different methods at the end of training on 10 permuted MNIST. Asterisk denotes result is collected from the author’s paper. Hyphen denotes default setting.

Input: Table 2: Average test accuracy on the split notMNIST benchmark.

Input: Figure 1: Comparison of different memory size on the permuted MNIST experiments.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review of the claim and evidence:

Summary: While the provided experimental results do show that DAR/SAR methods outperform baselines on several benchmarks (permuted MNIST, notMNIST, CIFAR-100), the claim is moderately overstated. The improvements, though consistent, are often modest (e.g., 94.9% vs 93% for VCL+Coreset on MNIST). The reviewer points out valid limitations: fixed hyperparameters, unclear validation splits, and limited novelty beyond LwF. The phrase "effectively overcomes catastrophic forgetting" is particularly strong given that performance still degrades over tasks (Fig 1). The claim would be better stated with more measured language acknowledging these limitations.

<score>4</score>

Justification:
1. While performance improvements are real, they are often incremental (1-3% gains)
2. Reviewer highlights important methodological limitations not addressed in the claim
3. The degrading performance in Fig 1 contradicts "effectively overcomes" catastrophic forgetting
4. Language like "extensive experiments" and "outperforms" lacks important qualifiers about experimental conditions and magnitude of improvements
5. No acknowledgment of limitations or experimental constraints in the claim


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

While both shed light on the overall behavior of a model, we show that, when it comes to explaining the prediction on a single input in isolation, i.e., _instance-wise explanations_, the two perspectives lead to fundamentally different explanations.

The evidence to be evaluated is:The most common instance-wise explanatory methods are _feature-based_, i.e., they explain a prediction in terms of the input unit-features (e.g., tokens for text and super-pixels for images). Among the feature-based explainers, there are two major types of explanations: (i) feature-additive: provide signed weights for each input feature, proportional to the contributions of the features to the model's prediction (Ribeiro et al., 2016; Lundberg and Lee, 2017; Shrikumar et al., 2017; Arras et al., 2017), and (ii) feature-selective: provide a (potentially ranked) subset of features responsible for the prediction (Chen et al., 2018; Ribeiro et al., 2018; Carter et al., 2018).

Other types of explanations are (iii) example-based (Koh and Liang, 2017): identify the most relevant instances in the training set that influenced the model's prediction on the current input, and (iv) human-level explanations (Camburu et al., 2018; Park et al., 2018; Kim et al., 2018; Bekele et al., 2018): explanations that are similar to what humans provide in real-world, both in terms of arguments (human-biases) and form (full-sentence natural language). In this work, we focus on verifying feature-based explainers, since they represent the majority of current works.

As mentioned before, current explanatory methods adhere to two major perspectives of explanations: _Perspective 1 (Feature-additivity)_: For a model \(f\) and an instance \(x\), the explanation of the prediction \(f(x)\) consists of a set of contributions \(\{w_{i}^{x}(f)\}_{i}\) for each feature \(i\) of \(x\) such that the sum of the contributions of the features in \(x\) approximates \(f(x)\), i.e., \(\sum_{i}w_{i}^{x}(f)\approx f(x)\). Many explanatory methods adhere to this perspective (Arras et al., 2017; Shrikumar et al., 2017; Ribeiro et al., 2016). For example, LIME (Ribeiro et al., 2016) learns the weights via a linear regression on the neighborhood (explained below) of the instance. Lundberg and Lee (2017) unified this class of methods by showing that the only set of feature-additive contributions that verify three desired constraints (local accuracy, missingness, and consistency--we refer to their paper for details) are given by the Shapley values from game theory: \[w_{i}^{x}(f)=\sum_{x^{\prime}\in x\setminus\{i\}}\frac{|x^{\prime}|![(|x|-|x^{ \prime}|-1)!}{|x|!}[f(x^{\prime}\cup\{i\})-f(x^{\prime})]\,, \tag{1}\] where the sum enumerates over all subsets \(x^{\prime}\) of features in \(x\) that do not include the feature \(i\), and \(|\cdot|\) denotes the number of features of its argument.

_Perspective 2 (Feature-selection)_: For a model \(f\) and an instance \(x\), the explanation of \(f(x)\) consists of a sufficient (ideally small) subset \(S(x)\) of (potentially ranked) features that alone lead to (almost) the same prediction as the original one, i.e., \(f(S(x))\approx f(x)\). Chen et al. (2018); Carter et al. (2018), and Ribeiro et al. (2018) adhere to this perspective. For example, L2X (Chen et al., 2018) learns \(S(x)\) by maximizing the mutual information between \(S(x)\) and the prediction. However, it assumes that the number of important features per instance, i.e., \(|S(x)|\), is known, which is usually not the case in practice. A downside of this perspective is that it may not always be true that the model relied only on a (small) subset of features, as opposed to using all the features. However, this can be the case for certain tasks, such as sentiment analysis.

Thus, we see that the feature-additive perspective aims to provide an _average explanation_ of the model on a _neighborhood_ of the instance, while the feature-selective perspective aims to tell us the pointwise features used by the model on the instance _in isolation_, such as "nice" for instance \(x_{1}\). An even more pronounced difference between the two perspectives is visible on instance \(x_{2}\), where the ranking of features is now different. The feature-selective explanation ranks "good" and "nice" as the two most important features, while on the instance \(x_{2}\) in isolation, the model relied on the tokens "very" and "good", that the feature-selection perspective would aim to provide. Therefore, we see that, while both perspectives of explanations give insights into the model's behavior, one perspective might be preferred over the other in different real-world use-cases. In the rest of the paper, we propose a verification framework for the _feature-selection_ perspective of instance-wise explanations.

We further prune the original dataset on which the RCNN had been trained to ensure that, for each datapoint \(x\), there exists a set of tokens that have zero contribution (irrelevant features) and a set of tokens that have a significant contribution (clearly relevant features) to RCNN's prediction on \(x\). We further introduce a set of metrics that measure how explainers fail to rank the irrelevant tokens lower than the clearly revelant ones.

The RCNN (Lei et al., 2016) consists of two modules: a generator followed by an encoder, both instantiated with recurrent convolutional neural networks (Lei et al., 2015). The generator is a bidirectional network that takes as input a piece of text \(x\) and, for each of its tokens, outputs the parameter of a Bernoulli distribution. According to this distribution, the RCNN selects a subset of tokens from \(x\), called \(\mathcal{S}_{x}=\text{generator}(x)\), and passes it to the encoder, which makes the final prediction solely as a function of \(\mathcal{S}_{x}\).

While Lei et al. (2016) aimed for \(\mathcal{S}_{x}\) to be the sufficient rationals for each prediction, the model might have learned an internal (emergent) communication protocol (Foerster et al., 2016) that encodes information from the non-selected via the selected tokens, which we call a _handshake_. For example, the RCNN could learn a handshake such as the one in Figure 2, where the feature "good" was important in all three cases, but not selected in the first two.

On our example in Figure 2, on the instance "The movie was very good.", the model selects "very" and predicts a score of \(1\). However, if we input the instance consisting of just "very", the model will not select anything2 and would return a score of \(0.5\).

There might be tokens (e.g., _the_ or \(a\) at the ends of the selection sequence(s)) that might have been selected in the original instance \(x\) and that become non-selected in the instance formed by \(\mathcal{S}_{x}\) without significantly changing the actual prediction.

In fact, it is possible that some tokens (such as "the" or "a") are actually noise, but have been selected only to ensure that the selection is a contiguous sequence (as we mentioned, the RCNN was penalized during training for selecting disconnected tokens). Since we do not want to penalize explainers for not differentiating between noise and zero-contribution features, we further prune the dataset such that there exists at least one selected token which is, without any doubt, _clearly relevant_ for the prediction.

We evaluate explainers that provide a ranking over the features. We denote by \(r_{1}(x),r_{2}(x),\ldots,r_{n}(x)\) the ranking (in decreasing order of importance) given by an explainer on the \(n=|x|\) features in the instance \(x\). Under our two properties above, we define the following error metrics:* **Percentage of instances for which the most important token provided by the explainer is among the non-selected tokens:** \[\text{\textasci{1.5}{\%}}_{\text{first}}=\tfrac{1}{|\mathcal{D}_{a}|}\sum \limits_{x\in\mathcal{D}_{a}}\mathbb{1}_{\{r_{1}(x)\in\mathcal{N}_{x}\}}\,,\] (4) where \(\mathbb{1}\) is the indicator function.

Metric (A) shows the most dramatic failure: the percentage of times when the explainer tells us that the most relevant token is one of the zero contribution ones. Metric (B) shows the percentage of instances for which there is at least an error in the explanation. Finally, metric (C) quantifies the number of zero-contribution features that were ranked higher than any clearly relevant feature.

As mentioned in Section 3, LIME and SHAP adhere to the feature-additivity perspective, hence our evaluation is not directly targeting these explainers.

Firstly, we notice that both explainers are prone to attributing importance to non-selected tokens, with LIME and SHAP even ranking the tokens "mouthfel" and "lacing" belonging to \(\mathcal{N}_{x}\) as first two (most important). Further, "gorgeous", the only relevant word used by the model, did not even make it in top 13 tokens for L2X.

In this work, we first shed light on an important distinction between two widely used perspectives of explanations.

We presented the error rates on different metrics for three popular explanatory methods to raise awareness of the types of failures that these explainers can produce, such as incorrectly predicting even the most relevant token.

We note that if there is a handshake in the instance \(x\), i.e., at least one non-selected token \(x_{k}\in\mathcal{N}_{x}\) is actually influencing the final prediction via an internal encoding of its information into the selected tokens, then the model should have a different prediction when \(x_{k}\) is eliminated from the instance, i.e., \(\text{RCNN}(x)\neq\text{RCNN}(x-x_{k})\). Equivalently, if \(\text{RCNN}(x-x_{k})=\text{RCNN}(x)\), then \(x_{k}\) could not have been part of a handshake. Thus, if the RCNN gives the same prediction when eliminating _all_ the non-selected tokens, it means that there was no handshake for the instance \(x\), and hence the tokens in \(\mathcal{N}_{x}\) have indeed zero contribution.

\[\text{RCNN}(x-\mathcal{N}_{x})=\text{RCNN}(x)\implies\text{no handshake in }x\,. \tag{8}\] Since \(x-\mathcal{N}_{x}=\mathcal{S}_{x}\), Equation 8 rewrites as: \[\text{RCNN}(\mathcal{S}_{x})=\text{RCNN}(x)\implies\text{no handshake in }x\,. \tag{9}\]

Input: Figure 2: Example of handshake.

Input: Figure 6: Explainers’ rankings (with the top 5 features on the right-hand side) on an instance from the aroma aspect in our evaluation.

Input: Figure 4: Explainers’ rankings (with top 5 features on the right-hand side) on an instance from the palate aspect in our evaluation dataset.

Input: Figure 1: Examples on which the two perspectives give different instance-wise explanations.

Input: Figure 5: Explainers’ rankings (with the top 5 features on the right-hand side) on an instance from the appearance aspect in our evaluation.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that feature-additive and feature-selective perspectives lead to "fundamentally different explanations" for instance-wise cases is well supported. The evidence provides clear theoretical frameworks for both perspectives and demonstrates concrete examples (like in Figure 1) where they produce distinctly different explanations for the same instances. The paper shows how feature-additive methods provide weighted contributions that sum to the prediction, while feature-selective methods identify sufficient subsets of features. The differences are illustrated through practical examples and mathematical formulations. The figures and examples effectively demonstrate these fundamental differences.

<score>1</score>

Justification:
1. The evidence provides thorough theoretical grounding explaining the distinct mathematical bases of each approach (feature-additive vs. feature-selective).
2. Figure 1 and other examples clearly demonstrate cases where the two perspectives generate different explanations for the same input.
3. The paper extensively documents the practical implications of these differences through multiple examples and evaluation metrics.
4. The claim is carefully worded ("when it comes to explaining the prediction on a single input in isolation") and avoids overgeneralization.
5. The evidence directly supports the specific scope of the claim regarding instance-wise explanations.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

While both shed light on the overall behavior of a model, we show that, when it comes to explaining the prediction on a single input in isolation, i.e., _instance-wise explanations_, the two perspectives lead to fundamentally different explanations.

The review comment to be evaluated is:

Summary -

The paper proposes a verification method for instance wise feature explanations. The verification framework uses an RCNN to identify two types of tokens a) the tokens that are not predictive of outcome b) the subset of clearly relevant tokens for prediction. The data used for RCNN is a pruned version of the data used to train the  black-box. The pruning eliminates data points to ensure that the tokens not selected by RCNN have no contribution to the outcome and that the model does not exhibit suffer from learning "handshakes". A handshake is defined as the set of tokens that may be spuriously missed because their information is encoded in another relevant token. This proof to identify such data points is shown and the RNN is therefore expected to be able to reliably identify 3 kinds of tokens a) Those that have zero contribution to the outcome. b) Those that definitely have some contribution to the outcome and c) those that could be relevant or noisy. Three instance-wise feature selection methods are compared. Results are provided on 3 metrics. a) % instances for which the most important tokes provided by the explainer is among the non-selected tokens, b) % of instances for which at least one non-selected token is ranked higher than a relevant token, and c) Average number of non-selected tokens ranked higher than any obviously relevant tokens.

Clarifications and concerns:
1. For the dataset considered here, I would like to see the distribution of the irrelevant, clearly relevant and unsure if they are relevant tokens as detected by the RCNN. How does this change if I further prune the dataset after ensuring that handshake and other issues have been eliminated. The main concern I have is the idea of verifying other explanations using a neural network itself. I can train the RCNN neural network with half the data (and satisfy the properties the authors mention) and my evaluation would change significantly. From the appendix I see that most of the tokens could be in the set $SDR_x$.

2. What if the set of tokens don't overlap between the RCNN and the black-box to be verified. That said, I think the assumptions of the framework should be much more explicitly mentioned.

3. The std deviations in the experiments are very high. Can the authors justify this and how it is still okay to use this framework for evaluating feature importance based explanations.

Minor:
1. You have cited the "Anchors" paper twice?
2. Page 3 - typo - "....explainer should provide different explanations for the trained model on real data than when the data..."
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update -

I have read the authors response.
If the pruned dataset is created using an RCNN, then it is not clear if the RCNN is used to just explain itself or all other methods as well. Like I said, if we just train the model on a slightly different distribution of labels, or half the data randomly sampled irrespective of labels, the explanations will change because the pruned dataset i.e. the ground truth may significantly change. I am still not convinced how this makes for a good verification framework to asses other explainers.

It is also unclear how generalizable this verification process is to other domains. I will therefore not be updating my score.

The evidence to be evaluated is:The most common instance-wise explanatory methods are _feature-based_, i.e., they explain a prediction in terms of the input unit-features (e.g., tokens for text and super-pixels for images). Among the feature-based explainers, there are two major types of explanations: (i) feature-additive: provide signed weights for each input feature, proportional to the contributions of the features to the model's prediction (Ribeiro et al., 2016; Lundberg and Lee, 2017; Shrikumar et al., 2017; Arras et al., 2017), and (ii) feature-selective: provide a (potentially ranked) subset of features responsible for the prediction (Chen et al., 2018; Ribeiro et al., 2018; Carter et al., 2018).

Other types of explanations are (iii) example-based (Koh and Liang, 2017): identify the most relevant instances in the training set that influenced the model's prediction on the current input, and (iv) human-level explanations (Camburu et al., 2018; Park et al., 2018; Kim et al., 2018; Bekele et al., 2018): explanations that are similar to what humans provide in real-world, both in terms of arguments (human-biases) and form (full-sentence natural language). In this work, we focus on verifying feature-based explainers, since they represent the majority of current works.

As mentioned before, current explanatory methods adhere to two major perspectives of explanations: _Perspective 1 (Feature-additivity)_: For a model \(f\) and an instance \(x\), the explanation of the prediction \(f(x)\) consists of a set of contributions \(\{w_{i}^{x}(f)\}_{i}\) for each feature \(i\) of \(x\) such that the sum of the contributions of the features in \(x\) approximates \(f(x)\), i.e., \(\sum_{i}w_{i}^{x}(f)\approx f(x)\). Many explanatory methods adhere to this perspective (Arras et al., 2017; Shrikumar et al., 2017; Ribeiro et al., 2016). For example, LIME (Ribeiro et al., 2016) learns the weights via a linear regression on the neighborhood (explained below) of the instance. Lundberg and Lee (2017) unified this class of methods by showing that the only set of feature-additive contributions that verify three desired constraints (local accuracy, missingness, and consistency--we refer to their paper for details) are given by the Shapley values from game theory: \[w_{i}^{x}(f)=\sum_{x^{\prime}\in x\setminus\{i\}}\frac{|x^{\prime}|![(|x|-|x^{ \prime}|-1)!}{|x|!}[f(x^{\prime}\cup\{i\})-f(x^{\prime})]\,, \tag{1}\] where the sum enumerates over all subsets \(x^{\prime}\) of features in \(x\) that do not include the feature \(i\), and \(|\cdot|\) denotes the number of features of its argument.

_Perspective 2 (Feature-selection)_: For a model \(f\) and an instance \(x\), the explanation of \(f(x)\) consists of a sufficient (ideally small) subset \(S(x)\) of (potentially ranked) features that alone lead to (almost) the same prediction as the original one, i.e., \(f(S(x))\approx f(x)\). Chen et al. (2018); Carter et al. (2018), and Ribeiro et al. (2018) adhere to this perspective. For example, L2X (Chen et al., 2018) learns \(S(x)\) by maximizing the mutual information between \(S(x)\) and the prediction. However, it assumes that the number of important features per instance, i.e., \(|S(x)|\), is known, which is usually not the case in practice. A downside of this perspective is that it may not always be true that the model relied only on a (small) subset of features, as opposed to using all the features. However, this can be the case for certain tasks, such as sentiment analysis.

Thus, we see that the feature-additive perspective aims to provide an _average explanation_ of the model on a _neighborhood_ of the instance, while the feature-selective perspective aims to tell us the pointwise features used by the model on the instance _in isolation_, such as "nice" for instance \(x_{1}\). An even more pronounced difference between the two perspectives is visible on instance \(x_{2}\), where the ranking of features is now different. The feature-selective explanation ranks "good" and "nice" as the two most important features, while on the instance \(x_{2}\) in isolation, the model relied on the tokens "very" and "good", that the feature-selection perspective would aim to provide. Therefore, we see that, while both perspectives of explanations give insights into the model's behavior, one perspective might be preferred over the other in different real-world use-cases. In the rest of the paper, we propose a verification framework for the _feature-selection_ perspective of instance-wise explanations.

We further prune the original dataset on which the RCNN had been trained to ensure that, for each datapoint \(x\), there exists a set of tokens that have zero contribution (irrelevant features) and a set of tokens that have a significant contribution (clearly relevant features) to RCNN's prediction on \(x\). We further introduce a set of metrics that measure how explainers fail to rank the irrelevant tokens lower than the clearly revelant ones.

The RCNN (Lei et al., 2016) consists of two modules: a generator followed by an encoder, both instantiated with recurrent convolutional neural networks (Lei et al., 2015). The generator is a bidirectional network that takes as input a piece of text \(x\) and, for each of its tokens, outputs the parameter of a Bernoulli distribution. According to this distribution, the RCNN selects a subset of tokens from \(x\), called \(\mathcal{S}_{x}=\text{generator}(x)\), and passes it to the encoder, which makes the final prediction solely as a function of \(\mathcal{S}_{x}\).

While Lei et al. (2016) aimed for \(\mathcal{S}_{x}\) to be the sufficient rationals for each prediction, the model might have learned an internal (emergent) communication protocol (Foerster et al., 2016) that encodes information from the non-selected via the selected tokens, which we call a _handshake_. For example, the RCNN could learn a handshake such as the one in Figure 2, where the feature "good" was important in all three cases, but not selected in the first two.

On our example in Figure 2, on the instance "The movie was very good.", the model selects "very" and predicts a score of \(1\). However, if we input the instance consisting of just "very", the model will not select anything2 and would return a score of \(0.5\).

There might be tokens (e.g., _the_ or \(a\) at the ends of the selection sequence(s)) that might have been selected in the original instance \(x\) and that become non-selected in the instance formed by \(\mathcal{S}_{x}\) without significantly changing the actual prediction.

In fact, it is possible that some tokens (such as "the" or "a") are actually noise, but have been selected only to ensure that the selection is a contiguous sequence (as we mentioned, the RCNN was penalized during training for selecting disconnected tokens). Since we do not want to penalize explainers for not differentiating between noise and zero-contribution features, we further prune the dataset such that there exists at least one selected token which is, without any doubt, _clearly relevant_ for the prediction.

We evaluate explainers that provide a ranking over the features. We denote by \(r_{1}(x),r_{2}(x),\ldots,r_{n}(x)\) the ranking (in decreasing order of importance) given by an explainer on the \(n=|x|\) features in the instance \(x\). Under our two properties above, we define the following error metrics:* **Percentage of instances for which the most important token provided by the explainer is among the non-selected tokens:** \[\text{\textasci{1.5}{\%}}_{\text{first}}=\tfrac{1}{|\mathcal{D}_{a}|}\sum \limits_{x\in\mathcal{D}_{a}}\mathbb{1}_{\{r_{1}(x)\in\mathcal{N}_{x}\}}\,,\] (4) where \(\mathbb{1}\) is the indicator function.

Metric (A) shows the most dramatic failure: the percentage of times when the explainer tells us that the most relevant token is one of the zero contribution ones. Metric (B) shows the percentage of instances for which there is at least an error in the explanation. Finally, metric (C) quantifies the number of zero-contribution features that were ranked higher than any clearly relevant feature.

As mentioned in Section 3, LIME and SHAP adhere to the feature-additivity perspective, hence our evaluation is not directly targeting these explainers.

Firstly, we notice that both explainers are prone to attributing importance to non-selected tokens, with LIME and SHAP even ranking the tokens "mouthfel" and "lacing" belonging to \(\mathcal{N}_{x}\) as first two (most important). Further, "gorgeous", the only relevant word used by the model, did not even make it in top 13 tokens for L2X.

In this work, we first shed light on an important distinction between two widely used perspectives of explanations.

We presented the error rates on different metrics for three popular explanatory methods to raise awareness of the types of failures that these explainers can produce, such as incorrectly predicting even the most relevant token.

We note that if there is a handshake in the instance \(x\), i.e., at least one non-selected token \(x_{k}\in\mathcal{N}_{x}\) is actually influencing the final prediction via an internal encoding of its information into the selected tokens, then the model should have a different prediction when \(x_{k}\) is eliminated from the instance, i.e., \(\text{RCNN}(x)\neq\text{RCNN}(x-x_{k})\). Equivalently, if \(\text{RCNN}(x-x_{k})=\text{RCNN}(x)\), then \(x_{k}\) could not have been part of a handshake. Thus, if the RCNN gives the same prediction when eliminating _all_ the non-selected tokens, it means that there was no handshake for the instance \(x\), and hence the tokens in \(\mathcal{N}_{x}\) have indeed zero contribution.

\[\text{RCNN}(x-\mathcal{N}_{x})=\text{RCNN}(x)\implies\text{no handshake in }x\,. \tag{8}\] Since \(x-\mathcal{N}_{x}=\mathcal{S}_{x}\), Equation 8 rewrites as: \[\text{RCNN}(\mathcal{S}_{x})=\text{RCNN}(x)\implies\text{no handshake in }x\,. \tag{9}\]

Input: Figure 2: Example of handshake.

Input: Figure 6: Explainers’ rankings (with the top 5 features on the right-hand side) on an instance from the aroma aspect in our evaluation.

Input: Figure 4: Explainers’ rankings (with top 5 features on the right-hand side) on an instance from the palate aspect in our evaluation dataset.

Input: Figure 1: Examples on which the two perspectives give different instance-wise explanations.

Input: Figure 5: Explainers’ rankings (with the top 5 features on the right-hand side) on an instance from the appearance aspect in our evaluation.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me evaluate the claim about the fundamental differences between feature-additive and feature-selective explanations.

Based on the evidence presented, particularly Figure 1 which provides concrete examples, the claim appears well-stated. The evidence demonstrates clear differences between the two perspectives in how they explain individual predictions:

1. Feature-additive methods provide weighted contributions that sum to approximate the prediction (shown through numerical weights in Figure 1)
2. Feature-selective methods identify specific subsets of features responsible for the prediction (shown through selected tokens)
3. The evidence shows these approaches can lead to different rankings and selections of important features for the same input

The review comments don't dispute this fundamental distinction between the approaches. The paper adequately supports this specific claim about the differences between these two perspectives for instance-wise explanations, without overstating their divergence.

<score>1</score>