Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning.

The review comment to be evaluated is:

The paper addresses the problem of continual learning from a sequence of supervised tasks. The main contribution of the paper are the following. The work:
* puts the problem in uniform general framework in which many of the state-of-the-art methods fit
* identifies some drawbacks of some the state of the art methods (namely EWC- Kirkpatrick et al., 2017 and GEM-Lopez-Paz et al., 2017) .
* proposes two versions of an approach to address these drawbacks
The main identified problem in EWC and GEM is that both methods result in a zero penalty when both the previous and current models misclassify a sample, even if they make different mistakes.
To solve this problem, the authors propose to keep a memory of randomly sampled data from the previous task, and distil the knowledge from the optima of the previous tasks to the current one using a KL penalty. The first version considers computing this penalty from the whole sequence of tasks, while the second randomly selects one task at each iteration and uses it to compute the penalty, paying some overall accuracy for plasticity.

While the paper is clear and well written, I have some concerns about it's quality, originality and significance.

Originality: The work seems to me to be very related to LwF. The main difference is that while LwF uses only the new data for the KL penalty, this paper keeps a memory of previously seen data to compute this loss.

Significance: While the way the authors approached the problem seems well structured and motivated, and is based on a sound observation, the authors limited themselves to experiments where the task data have similar structure. I am not sure how significant the improvement of the method would be in the more challenging and realistic setting where the data comes from different domains.  I am more specifically skeptical about the stochastic version of the algorithm in that case.

Quality:
* The proposed algorithm stores some data from previous tasks along with the outputs of the corresponding optimal model. While knowledge distillation as proposed would result in a non zero penalty when the new model makes different prediction, it still doesn't take advantage of the new information to probably correct the previous models prediction when it is wrong. Why not keeping the ground truth labels instead? This won't increase the memory requirement, and may give better results. It would be interesting to compare against such a method.
* I think selecting the samples to keep in memory randomly could to be suboptimal.  Other selection methods can be considered. A previous work:  iCaRL: Incremental Classifier and Representation Learning, Rebuffi et al. 2017, gives way to select representative samples. It would be interesting to see the effect of such a selection on the results.

Overall, while the paper doesn't present any significant flaw, it doesn't add much to the continual learning literature either, which explains my rating.

The evidence to be evaluated is:To avoid forgetting previous knowledge, when learning task \(\mathcal{T}_{t}\), the learner \(\mathbf{\theta}\) minimize the multi-task learning objective as \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\lambda_{t-1}\mathcal{L}_{t-1}(\mathbf{\theta},\mathcal{D}_{t-1})+\ldots+ \lambda_{1}\mathcal{L}_{1}(\mathbf{\theta},\mathcal{D}_{1}), \tag{1}\] where the loss \(\mathcal{L}_{i}(\mathbf{\theta},\mathcal{D}_{i})\) is usually the negative log-likelihood \(-\sum_{\mathbf{x},y\in\mathcal{D}_{i}}\log p(y|\mathbf{x};\mathbf{\theta})\) of the corresponding training data and \(\lambda_{i}\) regulates the importance of task \(\mathcal{T}_{i}\) to all the tasks observed so far. In continual learning, at time \(t\), we do not have access to the data \(\mathcal{D}_{k<t}\) to evaluate the loss \(\mathcal{L}_{k<t}\), thus, we need to approximate the past losses \(\mathcal{L}_{k<t}(\mathbf{\theta},\mathcal{D}_{k})\) when minimizing \(\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_{t})\). A common choice is to employ Laplace propagation (Eskin et al., 2004) of the negative log posterior \[-\log p(\mathbf{\theta}|\mathcal{D}_{i})\approx-\log p(\mathbf{\theta}_{i}^{*}| \mathcal{D}_{i})+\frac{1}{2}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}( \mathbf{\theta}-\mathbf{\theta}_{i}^{*}), \tag{2}\] where \(\mathbf{\theta}_{i}^{*}\) is a mode of the log posterior and \(\mathbf{F}_{i}\) is the empirical Fisher information matrix evaluated at \(\mathbf{\theta}=\mathbf{\theta}_{i}^{*}\). If we assume an uniform prior on \(\mathbf{\theta}\), the posterior coined with the likelihood 1, thus by substituting eqn. 2 to 1, the objective for training task \(\mathcal{T}_{t}\) becomes

\[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\sum_{i<t}\lambda_{k}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}(\bm {\theta}-\mathbf{\theta}_{i}^{*}), \tag{3}\] where the terms \(-\log p(\mathbf{\theta}_{i}^{*}|\mathcal{D}_{i})\) are omitted because they are constant with respect to \(\mathbf{\theta}\).

Eqn. 3 results in a wide range of methods for continual learning, each of which proposes a different method to approximate the Fisher information matrix. For example, Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017) assumes the Fisher is diagonal and approximates it using the identity: \[\mathbf{F}_{t}\approx\text{diag}\left(\sum_{n=1}^{N}\left.\left(\nabla_{\mathbf{ \theta}}\log p(\mathbf{y}_{t}^{(n)}|\mathbf{x}_{t}^{(n)};\mathbf{\theta})\right)^{2} \right|_{\mathbf{\theta}=\mathbf{\theta}_{T}}\right) \tag{4}\] Instead of using the Fisher's identity, Synaptic Intelligence (SI) (Zenke et al., 2017) directly measures the sensitivity of a parameter \(\mathbf{\theta}_{t}^{k}\) to the loss function at task \(\mathcal{T}_{t}\) through out the whole training trajectory. Zenke et al. (2017) also shows that this estimation is equivalent to measuring the full Fisher matrix on whole data set under certain choices of the loss function. Another approach to achieve the approximated multi-task learning objective in eqn. 1 is based on the observation that at the beginning of task \(\mathcal{T}_{t}\), we initialize \(\mathbf{\theta}=\mathbf{\theta}_{t-1}^{*}\), whose value already minimized the lost \(\tilde{\mathcal{L}}_{t-1}\). From this, if we can consider eqn. 1 as minimizing the loss \(\mathcal{L}_{t}(\mathbf{\theta})\) with the constraints that previous losses are not allowed to increase after every mini-batch update, which is the GEM objective. In particular, GEM optimize the following problem \[\begin{array}{ll}\text{Minimize}_{\mathbf{\theta},\mathcal{D}_{t}}&\mathcal{L}_ {t}(\mathbf{\theta})\\ \text{subject to}&\mathcal{L}_{k}(\mathbf{\theta},\mathcal{M}_{k})\leq\mathcal{L}_ {k}(\mathbf{\theta}_{t-1}^{*},\mathcal{M}_{k}),\forall k<t,\end{array} \tag{5}\]where the loss \(\mathcal{L}_{k}\) is evaluated at the memory \(\mathcal{M}_{k}\) that stores some of the training samples from task \(\mathcal{T}_{k}\). GEM further converts the constraints in eqn. 5 into the gradient constraints of the losses and solve them using quadratic programming. Finally, the type of losses \(\mathcal{L}_{k<t}\) can be flexibly changed to other loss rather than the negative log-likelihood. For example, Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes a form of distillation loss by minimizing the KL-divergence of past models and current model on the newly observed tasks. By changing the losses \(\mathcal{L}_{k<t}\), we can recover different solutions for continual learning in the literature.

To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks. This can also be interpreted as distilling the knowledge from \(\mathbf{\theta}_{i}^{*}\) to \(\mathbf{\theta}\) on the corresponding data \(\mathcal{D}_{i}\). In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

We refer to this algorithm as "Deterministic Activity Regularization" (DAR) which imposes regularization constraints for all the previous tasks at every learning iteration.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models.

7:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{D }}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{i<t}\sum_{\mathbf{x}\in\mathcal{M} _{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory. For SAR, we instead randomly sample an index i of the previous tasks and apply the AR only on that memory. As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability. For DAR, we omit the sampling index i in step 7 and calculate the KL divergence on all the memory as in eqn. 7. The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\) 9:\(\mathbf{\theta}\leftarrow\mathbf{\theta}-\alpha\cdot\text{SGD}(\mathbf{\theta},g_{\mathbf{ \theta}})\) 10:return\(\mathbf{\theta}\) ```**Algorithm 2** Stochastic Activity Regularization (SAR) algorithm for Continual Learning.

Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes to minimize the difference in KL-divergence of the current and previous models on the new tasks, which is a form of knowledge distillation penalty. Similarly, Less-forgetting learning (LF) (Jung et al., 2019) proposes to minimize the \(\ell_{2}\) norm of the two prediction in stead of KL-divergence as LwF. Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation.

While there are some overlapping between Bayesian learning and our approximated multi-task learning framework, the main difference is that our objective allows us to actively control the contribution of previous tasks by setting the importance value \(\lambda_{i}\) for each loss \(\mathcal{L}_{i}\).

In this work, we propose an approximated multi-task learning framework to unify both structural and functional regularization, in which our proposed activity regularization is a special case.

We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

We further improve DAR with a stochastic sampling regularizer (SAR), which balances the model's stability and plasticity, allowing the model to acquire new knowledge, alleviate forgetting and has greater scalability.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the claim "We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning" is well-supported by the evidence. The paper demonstrates:

1. A mathematical framework showing how different loss functions in the approximated multi-task learning objective (Equation 1) lead to different continual learning approaches
2. Explicit examples of how existing methods (EWC, SI, GEM, LwF) fit into this framework
3. Clear derivations showing how varying the losses recovers different methods

While the reviewer points out limitations in other aspects of the paper, their comments don't dispute this specific claim about unifying existing approaches under one framework.

<score>1</score>

Justification:
- The evidence directly demonstrates how various existing methods can be derived from their framework by modifying the loss functions (EWC through Fisher approximation, GEM through gradient constraints, LwF through KL-divergence)
- Mathematical formulations clearly show the connections between different approaches
- The claim is carefully worded to focus specifically on showing how existing methods can be unified, without overstating broader implications
- Even the critical reviewer acknowledges the work "puts the problem in uniform general framework in which many of the state-of-the-art methods fit"


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning.

The review comment to be evaluated is:

The authors proposed a new regularizer for continual learning to tackle the catastrophic forgetting problem. The proposed method minimizes the KL-divergence between the prediction of previous models and current models on the stored samples of previous tasks. The idea is straightforward and sounds technical. Experiments show the effectiveness of the methods compared to state-of-the-art. Although the idea sounds interesting and the experiments look promising, the novelty of the paper seems to be limited. In addition, the experiments setting needs to be improved as well. In the following, you have detailed comments.

1. It seems to be an extension of Learning without Forgetting (LwF) Li & Hoiem 2017 with simply on the examples of previous tasks in the memory. LwF only regularizes on the current task. It is not clear what is the difference between the proposed method and LwF except this.
2. The authors fixed many critical hyper-parameters: temperature(5), learning rate(0.05), epochs(10). The author should report the results for all methods with these hyper-parameters chosen on the validation set.
3. The authors presented how they split the training and testing data. Please be clear how you split the validation set.
4. The authors argued that sample quality does not affect the proposed methods. Then the authors should show the variance from different random sampling.

The evidence to be evaluated is:To avoid forgetting previous knowledge, when learning task \(\mathcal{T}_{t}\), the learner \(\mathbf{\theta}\) minimize the multi-task learning objective as \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\lambda_{t-1}\mathcal{L}_{t-1}(\mathbf{\theta},\mathcal{D}_{t-1})+\ldots+ \lambda_{1}\mathcal{L}_{1}(\mathbf{\theta},\mathcal{D}_{1}), \tag{1}\] where the loss \(\mathcal{L}_{i}(\mathbf{\theta},\mathcal{D}_{i})\) is usually the negative log-likelihood \(-\sum_{\mathbf{x},y\in\mathcal{D}_{i}}\log p(y|\mathbf{x};\mathbf{\theta})\) of the corresponding training data and \(\lambda_{i}\) regulates the importance of task \(\mathcal{T}_{i}\) to all the tasks observed so far. In continual learning, at time \(t\), we do not have access to the data \(\mathcal{D}_{k<t}\) to evaluate the loss \(\mathcal{L}_{k<t}\), thus, we need to approximate the past losses \(\mathcal{L}_{k<t}(\mathbf{\theta},\mathcal{D}_{k})\) when minimizing \(\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_{t})\). A common choice is to employ Laplace propagation (Eskin et al., 2004) of the negative log posterior \[-\log p(\mathbf{\theta}|\mathcal{D}_{i})\approx-\log p(\mathbf{\theta}_{i}^{*}| \mathcal{D}_{i})+\frac{1}{2}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}( \mathbf{\theta}-\mathbf{\theta}_{i}^{*}), \tag{2}\] where \(\mathbf{\theta}_{i}^{*}\) is a mode of the log posterior and \(\mathbf{F}_{i}\) is the empirical Fisher information matrix evaluated at \(\mathbf{\theta}=\mathbf{\theta}_{i}^{*}\). If we assume an uniform prior on \(\mathbf{\theta}\), the posterior coined with the likelihood 1, thus by substituting eqn. 2 to 1, the objective for training task \(\mathcal{T}_{t}\) becomes

\[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\sum_{i<t}\lambda_{k}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}(\bm {\theta}-\mathbf{\theta}_{i}^{*}), \tag{3}\] where the terms \(-\log p(\mathbf{\theta}_{i}^{*}|\mathcal{D}_{i})\) are omitted because they are constant with respect to \(\mathbf{\theta}\).

Eqn. 3 results in a wide range of methods for continual learning, each of which proposes a different method to approximate the Fisher information matrix. For example, Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017) assumes the Fisher is diagonal and approximates it using the identity: \[\mathbf{F}_{t}\approx\text{diag}\left(\sum_{n=1}^{N}\left.\left(\nabla_{\mathbf{ \theta}}\log p(\mathbf{y}_{t}^{(n)}|\mathbf{x}_{t}^{(n)};\mathbf{\theta})\right)^{2} \right|_{\mathbf{\theta}=\mathbf{\theta}_{T}}\right) \tag{4}\] Instead of using the Fisher's identity, Synaptic Intelligence (SI) (Zenke et al., 2017) directly measures the sensitivity of a parameter \(\mathbf{\theta}_{t}^{k}\) to the loss function at task \(\mathcal{T}_{t}\) through out the whole training trajectory. Zenke et al. (2017) also shows that this estimation is equivalent to measuring the full Fisher matrix on whole data set under certain choices of the loss function. Another approach to achieve the approximated multi-task learning objective in eqn. 1 is based on the observation that at the beginning of task \(\mathcal{T}_{t}\), we initialize \(\mathbf{\theta}=\mathbf{\theta}_{t-1}^{*}\), whose value already minimized the lost \(\tilde{\mathcal{L}}_{t-1}\). From this, if we can consider eqn. 1 as minimizing the loss \(\mathcal{L}_{t}(\mathbf{\theta})\) with the constraints that previous losses are not allowed to increase after every mini-batch update, which is the GEM objective. In particular, GEM optimize the following problem \[\begin{array}{ll}\text{Minimize}_{\mathbf{\theta},\mathcal{D}_{t}}&\mathcal{L}_ {t}(\mathbf{\theta})\\ \text{subject to}&\mathcal{L}_{k}(\mathbf{\theta},\mathcal{M}_{k})\leq\mathcal{L}_ {k}(\mathbf{\theta}_{t-1}^{*},\mathcal{M}_{k}),\forall k<t,\end{array} \tag{5}\]where the loss \(\mathcal{L}_{k}\) is evaluated at the memory \(\mathcal{M}_{k}\) that stores some of the training samples from task \(\mathcal{T}_{k}\). GEM further converts the constraints in eqn. 5 into the gradient constraints of the losses and solve them using quadratic programming. Finally, the type of losses \(\mathcal{L}_{k<t}\) can be flexibly changed to other loss rather than the negative log-likelihood. For example, Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes a form of distillation loss by minimizing the KL-divergence of past models and current model on the newly observed tasks. By changing the losses \(\mathcal{L}_{k<t}\), we can recover different solutions for continual learning in the literature.

To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks. This can also be interpreted as distilling the knowledge from \(\mathbf{\theta}_{i}^{*}\) to \(\mathbf{\theta}\) on the corresponding data \(\mathcal{D}_{i}\). In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

We refer to this algorithm as "Deterministic Activity Regularization" (DAR) which imposes regularization constraints for all the previous tasks at every learning iteration.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models.

7:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{D }}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{i<t}\sum_{\mathbf{x}\in\mathcal{M} _{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory. For SAR, we instead randomly sample an index i of the previous tasks and apply the AR only on that memory. As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability. For DAR, we omit the sampling index i in step 7 and calculate the KL divergence on all the memory as in eqn. 7. The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\) 9:\(\mathbf{\theta}\leftarrow\mathbf{\theta}-\alpha\cdot\text{SGD}(\mathbf{\theta},g_{\mathbf{ \theta}})\) 10:return\(\mathbf{\theta}\) ```**Algorithm 2** Stochastic Activity Regularization (SAR) algorithm for Continual Learning.

Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes to minimize the difference in KL-divergence of the current and previous models on the new tasks, which is a form of knowledge distillation penalty. Similarly, Less-forgetting learning (LF) (Jung et al., 2019) proposes to minimize the \(\ell_{2}\) norm of the two prediction in stead of KL-divergence as LwF. Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation.

While there are some overlapping between Bayesian learning and our approximated multi-task learning framework, the main difference is that our objective allows us to actively control the contribution of previous tasks by setting the importance value \(\lambda_{i}\) for each loss \(\mathcal{L}_{i}\).

In this work, we propose an approximated multi-task learning framework to unify both structural and functional regularization, in which our proposed activity regularization is a special case.

We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

We further improve DAR with a stochastic sampling regularizer (SAR), which balances the model's stability and plasticity, allowing the model to acquire new knowledge, alleviate forgetting and has greater scalability.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning."

Analysis:
The evidence strongly supports this claim through:
1. Mathematical derivation showing how the approximated multi-task learning objective (Equation 1-3) can be transformed into different forms
2. Concrete examples demonstrating how existing methods fit into this framework:
- EWC through Fisher matrix approximation (Equation 4)
- GEM through constraint optimization (Equation 5)
- LwF through KL-divergence
- Various other methods (MAS, SNI, LF) are shown to be special cases

The reviewer's comments don't dispute this foundational claim about unifying different approaches, focusing instead on the novelty of the proposed methods (DAR/SAR).

<score>1</score>

The claim is well-stated and accurately reflects what the evidence demonstrates without exaggeration. The authors provide detailed mathematical formulations and clear examples showing how different approaches can be derived from their framework.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Then, we analyze the weakness of two popular approaches, EWC and GEM, and show that label noise or mistakes in the memory can result in dramatic changes in the optimal decision boundary of previous tasks, leading to catastrophic forgetting.

The evidence to be evaluated is:While EWC and GEM are two popular methods for continual learning, we argue that they are sensitive to misclassified samples selected in the memory or used to estimate the Fisher information. Let us consider a scenario of learning two tasks \(\mathcal{T}_{1}\) and \(\mathcal{T}_{2}\) where we store the first optimal model \(\mathbf{\theta}_{1}^{*}\) and use a sample \(\{\mathbf{x}_{1},\mathbf{y}_{1}\}\) to estimate the Fisher information or storing in the memory. For EWC, by applying eqn. 2 with this scenario, we have: \[(\mathbf{\theta}-\mathbf{\theta}_{1}^{*})^{T}\mathbf{F}_{1}(\mathbf{\theta}-\mathbf{\theta}_{1}^{ *})\approx\log p(\mathbf{\theta}_{1}^{*}|\mathbf{x}_{1})-\log p(\mathbf{\theta}|\mathbf{x}_{1}), \tag{6}\] again we assume uniform prior on \(\mathbf{\theta}\), thus minimizing LHS of eqn. 6 is equivalent to finding \(\mathbf{\theta}\) that has the same log-likelihood as \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). For GEM, the constraint is to penalize whenever \(\mathbf{\theta}\) that has higher negative log-likelihood than \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). Now assume \(\mathbf{y}_{1}=[1,0,0]\) and the predictions of the two models \(\mathbf{\theta}_{1}^{*}\) and \(\mathbf{\theta}\) are \([0.1,0.9,0.1]\) and \([0.1,0.1,0.9]\), that is, both models make mistakes on \(\mathbf{x}_{1}\).

In general, if we have too many misclassified samples or label noise, the decision boundary on the previous task might be altered despite minimizing the EWC or GEM objective. This phenomenon happens based on an assumption that we usually do not train the model to have 0 error on the training set to avoid overfitting and we only use a relatively small data portion to estimate the Fisher or store as memory. Therefore, EWC and GEM might not correctly penalize the current model from changing the past tasks' optimized decision boundaries.

**Structural regularization** approaches employ penalties on the network's parameters, encourage the important parameters of previous tasks to not change when learning the new task. The parameter importance can be estimated by calculating the Fisher information as in EWC (Kirkpatrick et al., 2017) or directly measure how changes in each parameter will affect the change in the loss function as proposed in SI (Zenke et al., 2017). Exact estimation of the full Fisher is intractable. Therefore, both EWC and SI estimates only its diagonal, namely, how much each parameter contribute to the total loss assuming that the parameters are independent from each other.

Compare to SI, EWC and GEM perform considerably worse due the presence of label noise. This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks.

We further analyze the weakness of two popular methods: EWC and GEM, showing that they are sensitive to mistakes and noise in the Fisher or memory.

We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me evaluate the claim based on the provided evidence:

Summary:
The claim states that EWC and GEM show weakness when dealing with label noise/mistakes, leading to catastrophic forgetting. The evidence provides:
1. Mathematical analysis showing how misclassified samples can affect model predictions
2. Theoretical explanation of why noisy data impacts EWC and GEM's performance
3. Empirical evidence comparing EWC and GEM to SI, showing worse performance with noisy data
4. Concrete example with notMNIST dataset demonstrating the negative impact

The claim appears well-stated as it:
- Accurately reflects the mathematical and empirical evidence
- Makes assertions that are directly supported by the provided analysis
- Doesn't overextend beyond what the evidence demonstrates
- Uses appropriate qualifying language ("can result in")

<score>1</score>

Justification:
1. The mathematical analysis in equations 6 clearly demonstrates the mechanism by which misclassified samples affect the models
2. The evidence provides both theoretical framework and practical examples supporting the claim
3. The comparison with SI provides a control reference demonstrating the specific weakness
4. The evidence with notMNIST dataset provides concrete validation
5. The claim's language matches the strength of the evidence without overstating the findings
6. The limitations and conditions are properly acknowledged (e.g., "if we have too many misclassified samples")


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Then, we analyze the weakness of two popular approaches, EWC and GEM, and show that label noise or mistakes in the memory can result in dramatic changes in the optimal decision boundary of previous tasks, leading to catastrophic forgetting.

The review comment to be evaluated is:

This paper proposes an approach to mitigate catastrophic forgetting in supervised learning by regularizing activations. The paper views previous techniques (EWC, SI, and GEM) under a multi-task learning lens, and then proposes an additional loss term to minimise the KL between activations from previous and current models, on previous tasks - this is based on a memory which stores some previous samples and their corresponding activations.

I think it is a simple and intuitive approach and a well-written paper. Unfortunately I have a number of concerns that I think preclude publication in the current state.

First, in terms of related work, I believe this is very similar to Learning without forgetting (LwF), with the difference that the KL-divergence is computed on samples kept from the previous tasks. This is briefly mentioned in the paper, but I think it needs to be made more explicit, and LwF should be a baseline in the experiments to clearly indicate the benefit of keeping this data. There is also a relationship to EWC: given the connection between the Fisher information and KL, it can be viewed as minimising the KL divergence in parameter space, rather than in activation space (which is the case here). Also note that EWC uses the true Fisher rather than the empirical, contrary to the derivation in equation (2).
There are also a number of papers that haven’t been cited in the related work [1][2][3][4].

Second, I think the motivation in Section 3.1 could be more convincing. Most importantly, it’s not clear to me that the decision boundary *shouldn’t* change for previously misclassified examples, as this could be an opportunity for backwards transfer.
Further, I don’t think the point in the last paragraph about having a small data portion is relevant, since they are from the same data distribution, and we would expect misclassified samples to be in the same (low) frequency in Fisher estimation as overall. I think the point of this paragraph is just that it is important to consider the entire predictive distribution of previous tasks rather than the probability of the correct class, so this should be stated more clearly and then justified.

Finally, I think the experimental justification could be improved as well. Beyond permuted MNIST (which it has been argued is not as useful as other baselines [4]), only the final performance on split notMNIST / CIFAR-100 is reported. Some comments and questions:
- The accuracies of EWC (and possibly SI) in the table are worse than reported in previous work (eg. [1]), so I think this needs to be examined.
- What is the fine-tuning baseline (I don't believe it is actually clearly defined)? How can it be so low in figure 2a but better in 2b?
- I think plots over time (performance on all tasks) would be much more useful than the final performance in Table 2 and Fig 2.
- Errors and error bars would be beneficial for all results.
- Table 1 should also include the references provided.

Some other comments and questions:
- Compared to eqn (2), eqn (6) is missing the ½ constant.
- Typos in section 5.3: "SI performs better than SI", and VAR instead of SAR.
- Section 2, unclear of meaning of "coined with the likelihood" (should this be “coincide”?)
- The first line should be “Humans have the ability to learn...” In general, I think the introduction could use another proofread for grammar and readability as I saw a few minor things.

[1] Nguyen, Cuong V., et al. "Variational Continual Learning." ICLR, 2018.
[2] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." ICML, 2018.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." NIPS, 2017.
[4] Farquhar, Sebastian, and Yarin Gal. "Towards Robust Evaluations of Continual Learning." arXiv, 2018.

The evidence to be evaluated is:While EWC and GEM are two popular methods for continual learning, we argue that they are sensitive to misclassified samples selected in the memory or used to estimate the Fisher information. Let us consider a scenario of learning two tasks \(\mathcal{T}_{1}\) and \(\mathcal{T}_{2}\) where we store the first optimal model \(\mathbf{\theta}_{1}^{*}\) and use a sample \(\{\mathbf{x}_{1},\mathbf{y}_{1}\}\) to estimate the Fisher information or storing in the memory. For EWC, by applying eqn. 2 with this scenario, we have: \[(\mathbf{\theta}-\mathbf{\theta}_{1}^{*})^{T}\mathbf{F}_{1}(\mathbf{\theta}-\mathbf{\theta}_{1}^{ *})\approx\log p(\mathbf{\theta}_{1}^{*}|\mathbf{x}_{1})-\log p(\mathbf{\theta}|\mathbf{x}_{1}), \tag{6}\] again we assume uniform prior on \(\mathbf{\theta}\), thus minimizing LHS of eqn. 6 is equivalent to finding \(\mathbf{\theta}\) that has the same log-likelihood as \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). For GEM, the constraint is to penalize whenever \(\mathbf{\theta}\) that has higher negative log-likelihood than \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). Now assume \(\mathbf{y}_{1}=[1,0,0]\) and the predictions of the two models \(\mathbf{\theta}_{1}^{*}\) and \(\mathbf{\theta}\) are \([0.1,0.9,0.1]\) and \([0.1,0.1,0.9]\), that is, both models make mistakes on \(\mathbf{x}_{1}\).

In general, if we have too many misclassified samples or label noise, the decision boundary on the previous task might be altered despite minimizing the EWC or GEM objective. This phenomenon happens based on an assumption that we usually do not train the model to have 0 error on the training set to avoid overfitting and we only use a relatively small data portion to estimate the Fisher or store as memory. Therefore, EWC and GEM might not correctly penalize the current model from changing the past tasks' optimized decision boundaries.

**Structural regularization** approaches employ penalties on the network's parameters, encourage the important parameters of previous tasks to not change when learning the new task. The parameter importance can be estimated by calculating the Fisher information as in EWC (Kirkpatrick et al., 2017) or directly measure how changes in each parameter will affect the change in the loss function as proposed in SI (Zenke et al., 2017). Exact estimation of the full Fisher is intractable. Therefore, both EWC and SI estimates only its diagonal, namely, how much each parameter contribute to the total loss assuming that the parameters are independent from each other.

Compare to SI, EWC and GEM perform considerably worse due the presence of label noise. This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks.

We further analyze the weakness of two popular methods: EWC and GEM, showing that they are sensitive to mistakes and noise in the Fisher or memory.

We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll assess the claim based on the provided evidence and review comments:

Summary:
The claim argues that EWC and GEM are sensitive to label noise/mistakes leading to catastrophic forgetting. While the evidence provides theoretical analysis and some experimental observations showing their weakness with noisy data, the reviewer points out several limitations:
1. The experimental justification is limited (missing error bars, comparisons with other baselines)
2. The reported accuracies for EWC/SI are worse than in previous work
3. The reviewer questions whether decision boundary changes are necessarily negative (could enable beneficial backward transfer)
4. The motivation regarding small data portions needs stronger justification

The claim is moderately overstated (score 3) as it makes strong assertions about "dramatic changes" and "catastrophic forgetting" without fully addressing limitations noted by the reviewer or providing comprehensive experimental validation.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Then, we analyze the weakness of two popular approaches, EWC and GEM, and show that label noise or mistakes in the memory can result in dramatic changes in the optimal decision boundary of previous tasks, leading to catastrophic forgetting.

The review comment to be evaluated is:

The paper addresses the problem of continual learning from a sequence of supervised tasks. The main contribution of the paper are the following. The work:
* puts the problem in uniform general framework in which many of the state-of-the-art methods fit
* identifies some drawbacks of some the state of the art methods (namely EWC- Kirkpatrick et al., 2017 and GEM-Lopez-Paz et al., 2017) .
* proposes two versions of an approach to address these drawbacks
The main identified problem in EWC and GEM is that both methods result in a zero penalty when both the previous and current models misclassify a sample, even if they make different mistakes.
To solve this problem, the authors propose to keep a memory of randomly sampled data from the previous task, and distil the knowledge from the optima of the previous tasks to the current one using a KL penalty. The first version considers computing this penalty from the whole sequence of tasks, while the second randomly selects one task at each iteration and uses it to compute the penalty, paying some overall accuracy for plasticity.

While the paper is clear and well written, I have some concerns about it's quality, originality and significance.

Originality: The work seems to me to be very related to LwF. The main difference is that while LwF uses only the new data for the KL penalty, this paper keeps a memory of previously seen data to compute this loss.

Significance: While the way the authors approached the problem seems well structured and motivated, and is based on a sound observation, the authors limited themselves to experiments where the task data have similar structure. I am not sure how significant the improvement of the method would be in the more challenging and realistic setting where the data comes from different domains.  I am more specifically skeptical about the stochastic version of the algorithm in that case.

Quality:
* The proposed algorithm stores some data from previous tasks along with the outputs of the corresponding optimal model. While knowledge distillation as proposed would result in a non zero penalty when the new model makes different prediction, it still doesn't take advantage of the new information to probably correct the previous models prediction when it is wrong. Why not keeping the ground truth labels instead? This won't increase the memory requirement, and may give better results. It would be interesting to compare against such a method.
* I think selecting the samples to keep in memory randomly could to be suboptimal.  Other selection methods can be considered. A previous work:  iCaRL: Incremental Classifier and Representation Learning, Rebuffi et al. 2017, gives way to select representative samples. It would be interesting to see the effect of such a selection on the results.

Overall, while the paper doesn't present any significant flaw, it doesn't add much to the continual learning literature either, which explains my rating.

The evidence to be evaluated is:While EWC and GEM are two popular methods for continual learning, we argue that they are sensitive to misclassified samples selected in the memory or used to estimate the Fisher information. Let us consider a scenario of learning two tasks \(\mathcal{T}_{1}\) and \(\mathcal{T}_{2}\) where we store the first optimal model \(\mathbf{\theta}_{1}^{*}\) and use a sample \(\{\mathbf{x}_{1},\mathbf{y}_{1}\}\) to estimate the Fisher information or storing in the memory. For EWC, by applying eqn. 2 with this scenario, we have: \[(\mathbf{\theta}-\mathbf{\theta}_{1}^{*})^{T}\mathbf{F}_{1}(\mathbf{\theta}-\mathbf{\theta}_{1}^{ *})\approx\log p(\mathbf{\theta}_{1}^{*}|\mathbf{x}_{1})-\log p(\mathbf{\theta}|\mathbf{x}_{1}), \tag{6}\] again we assume uniform prior on \(\mathbf{\theta}\), thus minimizing LHS of eqn. 6 is equivalent to finding \(\mathbf{\theta}\) that has the same log-likelihood as \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). For GEM, the constraint is to penalize whenever \(\mathbf{\theta}\) that has higher negative log-likelihood than \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). Now assume \(\mathbf{y}_{1}=[1,0,0]\) and the predictions of the two models \(\mathbf{\theta}_{1}^{*}\) and \(\mathbf{\theta}\) are \([0.1,0.9,0.1]\) and \([0.1,0.1,0.9]\), that is, both models make mistakes on \(\mathbf{x}_{1}\).

In general, if we have too many misclassified samples or label noise, the decision boundary on the previous task might be altered despite minimizing the EWC or GEM objective. This phenomenon happens based on an assumption that we usually do not train the model to have 0 error on the training set to avoid overfitting and we only use a relatively small data portion to estimate the Fisher or store as memory. Therefore, EWC and GEM might not correctly penalize the current model from changing the past tasks' optimized decision boundaries.

**Structural regularization** approaches employ penalties on the network's parameters, encourage the important parameters of previous tasks to not change when learning the new task. The parameter importance can be estimated by calculating the Fisher information as in EWC (Kirkpatrick et al., 2017) or directly measure how changes in each parameter will affect the change in the loss function as proposed in SI (Zenke et al., 2017). Exact estimation of the full Fisher is intractable. Therefore, both EWC and SI estimates only its diagonal, namely, how much each parameter contribute to the total loss assuming that the parameters are independent from each other.

Compare to SI, EWC and GEM perform considerably worse due the presence of label noise. This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks.

We further analyze the weakness of two popular methods: EWC and GEM, showing that they are sensitive to mistakes and noise in the Fisher or memory.

We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me assess the claim based on the provided evidence and review:

1. Summary and Evaluation:
The evidence provides theoretical analysis and experimental examples demonstrating how EWC and GEM can be affected by misclassified samples and label noise. The claim is supported by:
- Mathematical analysis showing how both models can fail when samples are misclassified
- Experimental observations comparing performance with SI
- Demonstration with notMNIST dataset
- Review acknowledges the sound observation of the weakness

However, some limitations exist:
- Limited experimental settings (noted in review)
- Focuses mainly on similar data structure scenarios
- Could benefit from more diverse experimental validation

The claim is well-stated (score 1) as it:
- Makes specific, evidence-backed assertions
- Uses measured language ("can result" rather than absolute statements)
- Directly reflects the demonstrated weaknesses
- Aligns with both the mathematical proof and experimental results
- Doesn't overextend beyond what the evidence shows

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Then, we analyze the weakness of two popular approaches, EWC and GEM, and show that label noise or mistakes in the memory can result in dramatic changes in the optimal decision boundary of previous tasks, leading to catastrophic forgetting.

The review comment to be evaluated is:

The authors proposed a new regularizer for continual learning to tackle the catastrophic forgetting problem. The proposed method minimizes the KL-divergence between the prediction of previous models and current models on the stored samples of previous tasks. The idea is straightforward and sounds technical. Experiments show the effectiveness of the methods compared to state-of-the-art. Although the idea sounds interesting and the experiments look promising, the novelty of the paper seems to be limited. In addition, the experiments setting needs to be improved as well. In the following, you have detailed comments.

1. It seems to be an extension of Learning without Forgetting (LwF) Li & Hoiem 2017 with simply on the examples of previous tasks in the memory. LwF only regularizes on the current task. It is not clear what is the difference between the proposed method and LwF except this.
2. The authors fixed many critical hyper-parameters: temperature(5), learning rate(0.05), epochs(10). The author should report the results for all methods with these hyper-parameters chosen on the validation set.
3. The authors presented how they split the training and testing data. Please be clear how you split the validation set.
4. The authors argued that sample quality does not affect the proposed methods. Then the authors should show the variance from different random sampling.

The evidence to be evaluated is:While EWC and GEM are two popular methods for continual learning, we argue that they are sensitive to misclassified samples selected in the memory or used to estimate the Fisher information. Let us consider a scenario of learning two tasks \(\mathcal{T}_{1}\) and \(\mathcal{T}_{2}\) where we store the first optimal model \(\mathbf{\theta}_{1}^{*}\) and use a sample \(\{\mathbf{x}_{1},\mathbf{y}_{1}\}\) to estimate the Fisher information or storing in the memory. For EWC, by applying eqn. 2 with this scenario, we have: \[(\mathbf{\theta}-\mathbf{\theta}_{1}^{*})^{T}\mathbf{F}_{1}(\mathbf{\theta}-\mathbf{\theta}_{1}^{ *})\approx\log p(\mathbf{\theta}_{1}^{*}|\mathbf{x}_{1})-\log p(\mathbf{\theta}|\mathbf{x}_{1}), \tag{6}\] again we assume uniform prior on \(\mathbf{\theta}\), thus minimizing LHS of eqn. 6 is equivalent to finding \(\mathbf{\theta}\) that has the same log-likelihood as \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). For GEM, the constraint is to penalize whenever \(\mathbf{\theta}\) that has higher negative log-likelihood than \(\mathbf{\theta}_{1}^{*}\) on \(\mathbf{x}_{1}\). Now assume \(\mathbf{y}_{1}=[1,0,0]\) and the predictions of the two models \(\mathbf{\theta}_{1}^{*}\) and \(\mathbf{\theta}\) are \([0.1,0.9,0.1]\) and \([0.1,0.1,0.9]\), that is, both models make mistakes on \(\mathbf{x}_{1}\).

In general, if we have too many misclassified samples or label noise, the decision boundary on the previous task might be altered despite minimizing the EWC or GEM objective. This phenomenon happens based on an assumption that we usually do not train the model to have 0 error on the training set to avoid overfitting and we only use a relatively small data portion to estimate the Fisher or store as memory. Therefore, EWC and GEM might not correctly penalize the current model from changing the past tasks' optimized decision boundaries.

**Structural regularization** approaches employ penalties on the network's parameters, encourage the important parameters of previous tasks to not change when learning the new task. The parameter importance can be estimated by calculating the Fisher information as in EWC (Kirkpatrick et al., 2017) or directly measure how changes in each parameter will affect the change in the loss function as proposed in SI (Zenke et al., 2017). Exact estimation of the full Fisher is intractable. Therefore, both EWC and SI estimates only its diagonal, namely, how much each parameter contribute to the total loss assuming that the parameters are independent from each other.

Compare to SI, EWC and GEM perform considerably worse due the presence of label noise. This result is consistent with our analysis in section 3.1: noisy memory can easily cause catastrophic forgetting in EWC and GEM while SI is more resistant to noise since it approximates the Fisher on all observed samples of past tasks.

We further analyze the weakness of two popular methods: EWC and GEM, showing that they are sensitive to mistakes and noise in the Fisher or memory.

We demonstrate that label noise data such as notMNIST can be extremely destructive for EWC and GEM in the continual learning setting while our methods can still maintain good performance.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me assess the claim based on the provided evidence:

The claim argues that EWC and GEM show weakness due to label noise/mistakes leading to catastrophic forgetting. Here's my evaluation:

The evidence provides:
1. Theoretical analysis showing how misclassified samples can affect EWC and GEM's objectives
2. Specific examples demonstrating the impact of incorrect predictions
3. Experimental evidence showing EWC and GEM perform worse compared to SI in presence of label noise
4. Demonstration with notMNIST dataset showing sensitivity to noise

While the evidence supports the core argument about these methods' sensitivity to noise, the claim of "dramatic changes" and "catastrophic forgetting" appears somewhat overstated given that:
1. The analysis primarily focuses on specific scenarios/examples
2. The relative performance degradation is shown but not necessarily "dramatic"
3. The reviewer points out need for more robust experimental validation including variance analysis

<score>3</score>

Justification:
- The claim has valid foundational evidence but uses strong language ("dramatic changes", "catastrophic") that isn't fully supported by the quantitative evidence provided
- The theoretical analysis is sound but the empirical validation could be more comprehensive, as noted in review comments
- The evidence demonstrates sensitivity to noise but doesn't conclusively show the extent of "dramatic changes" in all cases
- The reviewer's comments about need for validation across different random samplings suggests the conclusions might be overstated without showing consistency across different scenarios


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To overcome the above problem, we propose a novel technique called "Activity Regularization" (AR) that penalizes changes in the model's prediction on its learned information.

The evidence to be evaluated is:To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks.

In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

We refer to this algorithm as "Deterministic Activity Regularization" (DAR) which imposes regularization constraints for all the previous tasks at every learning iteration.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models. Moreover, the number of regularizers for each task is equal to the number of previously observed tasks, which may considerably restrict the model's _plasticity_ to acquire new knowledge. That is, by enforcing too many constraints on the current model for reducing forgetting, we sacrifice its ability to learn new information.

7:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{D }}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{i<t}\sum_{\mathbf{x}\in\mathcal{M} _{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

```**Algorithm 1** Deterministic Activity Regularization (DAR) algorithm for Continual Learning. To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory. For SAR, we instead randomly sample an index i of the previous tasks and apply the AR only on that memory. As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability. For DAR, we omit the sampling index i in step 7 and calculate the KL divergence on all the memory as in eqn. 7. The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation.

To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim about Activity Regularization based on the provided evidence.

Summary:
The claim is moderately well-stated, as the evidence demonstrates that AR is indeed a technique that uses KL-divergence to penalize changes in model predictions on previously learned tasks. The evidence thoroughly explains the mathematical formulation and implementation details of both DAR and SAR variants. The experimental results from Figure 2 support the effectiveness of the approach, showing competitive performance compared to other methods. However, the claim could be more precise by mentioning the KL-divergence mechanism and acknowledging the limitations discussed in the evidence.

Key points supporting this assessment:
1. The evidence clearly describes the technical implementation through equations and algorithms
2. Results demonstrate effectiveness but also discuss limitations (computational cost, plasticity constraints)
3. The claim is accurate but could be more specific about the mechanism

<score>2</score>

Justification:
While the claim is generally accurate, it receives a score of 2 (slightly more than well-stated) because:
- It correctly describes the core purpose of AR
- The evidence substantiates the basic claim through mathematical formulation and experimental results
- However, it could be more precise by mentioning:
  - The specific use of KL-divergence
  - The distinction between DAR and SAR variants
  - The trade-offs involved in terms of computational cost and plasticity
- The experimental results support the claim's validity without showing signs of exaggeration


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To overcome the above problem, we propose a novel technique called "Activity Regularization" (AR) that penalizes changes in the model's prediction on its learned information.

The review comment to be evaluated is:

This paper proposes an approach to mitigate catastrophic forgetting in supervised learning by regularizing activations. The paper views previous techniques (EWC, SI, and GEM) under a multi-task learning lens, and then proposes an additional loss term to minimise the KL between activations from previous and current models, on previous tasks - this is based on a memory which stores some previous samples and their corresponding activations.

I think it is a simple and intuitive approach and a well-written paper. Unfortunately I have a number of concerns that I think preclude publication in the current state.

First, in terms of related work, I believe this is very similar to Learning without forgetting (LwF), with the difference that the KL-divergence is computed on samples kept from the previous tasks. This is briefly mentioned in the paper, but I think it needs to be made more explicit, and LwF should be a baseline in the experiments to clearly indicate the benefit of keeping this data. There is also a relationship to EWC: given the connection between the Fisher information and KL, it can be viewed as minimising the KL divergence in parameter space, rather than in activation space (which is the case here). Also note that EWC uses the true Fisher rather than the empirical, contrary to the derivation in equation (2).
There are also a number of papers that haven’t been cited in the related work [1][2][3][4].

Second, I think the motivation in Section 3.1 could be more convincing. Most importantly, it’s not clear to me that the decision boundary *shouldn’t* change for previously misclassified examples, as this could be an opportunity for backwards transfer.
Further, I don’t think the point in the last paragraph about having a small data portion is relevant, since they are from the same data distribution, and we would expect misclassified samples to be in the same (low) frequency in Fisher estimation as overall. I think the point of this paragraph is just that it is important to consider the entire predictive distribution of previous tasks rather than the probability of the correct class, so this should be stated more clearly and then justified.

Finally, I think the experimental justification could be improved as well. Beyond permuted MNIST (which it has been argued is not as useful as other baselines [4]), only the final performance on split notMNIST / CIFAR-100 is reported. Some comments and questions:
- The accuracies of EWC (and possibly SI) in the table are worse than reported in previous work (eg. [1]), so I think this needs to be examined.
- What is the fine-tuning baseline (I don't believe it is actually clearly defined)? How can it be so low in figure 2a but better in 2b?
- I think plots over time (performance on all tasks) would be much more useful than the final performance in Table 2 and Fig 2.
- Errors and error bars would be beneficial for all results.
- Table 1 should also include the references provided.

Some other comments and questions:
- Compared to eqn (2), eqn (6) is missing the ½ constant.
- Typos in section 5.3: "SI performs better than SI", and VAR instead of SAR.
- Section 2, unclear of meaning of "coined with the likelihood" (should this be “coincide”?)
- The first line should be “Humans have the ability to learn...” In general, I think the introduction could use another proofread for grammar and readability as I saw a few minor things.

[1] Nguyen, Cuong V., et al. "Variational Continual Learning." ICLR, 2018.
[2] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." ICML, 2018.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." NIPS, 2017.
[4] Farquhar, Sebastian, and Yarin Gal. "Towards Robust Evaluations of Continual Learning." arXiv, 2018.

The evidence to be evaluated is:To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks.

In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

We refer to this algorithm as "Deterministic Activity Regularization" (DAR) which imposes regularization constraints for all the previous tasks at every learning iteration.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models. Moreover, the number of regularizers for each task is equal to the number of previously observed tasks, which may considerably restrict the model's _plasticity_ to acquire new knowledge. That is, by enforcing too many constraints on the current model for reducing forgetting, we sacrifice its ability to learn new information.

7:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{D }}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{i<t}\sum_{\mathbf{x}\in\mathcal{M} _{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

```**Algorithm 1** Deterministic Activity Regularization (DAR) algorithm for Continual Learning. To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory. For SAR, we instead randomly sample an index i of the previous tasks and apply the AR only on that memory. As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability. For DAR, we omit the sampling index i in step 7 and calculate the KL divergence on all the memory as in eqn. 7. The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation.

To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, this appears to be a modestly overstated claim. While the evidence shows that Activity Regularization (AR) is indeed proposed as a technique to address catastrophic forgetting by penalizing changes in model predictions, several key points make the claim somewhat overstated:

1. The reviewer correctly points out similarity to existing work (LwF), suggesting the "novel" aspect is overstated
2. The evidence shows limitations of DAR that aren't acknowledged in the claim (computational cost, restricted plasticity)
3. The empirical results in Fig 2 show modest improvements over baselines
4. The review highlights that experimental validation could be more thorough

The claim would be better stated by acknowledging these limitations and similarities to existing approaches.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To overcome the above problem, we propose a novel technique called "Activity Regularization" (AR) that penalizes changes in the model's prediction on its learned information.

The review comment to be evaluated is:

The paper addresses the problem of continual learning from a sequence of supervised tasks. The main contribution of the paper are the following. The work:
* puts the problem in uniform general framework in which many of the state-of-the-art methods fit
* identifies some drawbacks of some the state of the art methods (namely EWC- Kirkpatrick et al., 2017 and GEM-Lopez-Paz et al., 2017) .
* proposes two versions of an approach to address these drawbacks
The main identified problem in EWC and GEM is that both methods result in a zero penalty when both the previous and current models misclassify a sample, even if they make different mistakes.
To solve this problem, the authors propose to keep a memory of randomly sampled data from the previous task, and distil the knowledge from the optima of the previous tasks to the current one using a KL penalty. The first version considers computing this penalty from the whole sequence of tasks, while the second randomly selects one task at each iteration and uses it to compute the penalty, paying some overall accuracy for plasticity.

While the paper is clear and well written, I have some concerns about it's quality, originality and significance.

Originality: The work seems to me to be very related to LwF. The main difference is that while LwF uses only the new data for the KL penalty, this paper keeps a memory of previously seen data to compute this loss.

Significance: While the way the authors approached the problem seems well structured and motivated, and is based on a sound observation, the authors limited themselves to experiments where the task data have similar structure. I am not sure how significant the improvement of the method would be in the more challenging and realistic setting where the data comes from different domains.  I am more specifically skeptical about the stochastic version of the algorithm in that case.

Quality:
* The proposed algorithm stores some data from previous tasks along with the outputs of the corresponding optimal model. While knowledge distillation as proposed would result in a non zero penalty when the new model makes different prediction, it still doesn't take advantage of the new information to probably correct the previous models prediction when it is wrong. Why not keeping the ground truth labels instead? This won't increase the memory requirement, and may give better results. It would be interesting to compare against such a method.
* I think selecting the samples to keep in memory randomly could to be suboptimal.  Other selection methods can be considered. A previous work:  iCaRL: Incremental Classifier and Representation Learning, Rebuffi et al. 2017, gives way to select representative samples. It would be interesting to see the effect of such a selection on the results.

Overall, while the paper doesn't present any significant flaw, it doesn't add much to the continual learning literature either, which explains my rating.

The evidence to be evaluated is:To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks.

In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

We refer to this algorithm as "Deterministic Activity Regularization" (DAR) which imposes regularization constraints for all the previous tasks at every learning iteration.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models. Moreover, the number of regularizers for each task is equal to the number of previously observed tasks, which may considerably restrict the model's _plasticity_ to acquire new knowledge. That is, by enforcing too many constraints on the current model for reducing forgetting, we sacrifice its ability to learn new information.

7:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{D }}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{i<t}\sum_{\mathbf{x}\in\mathcal{M} _{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

```**Algorithm 1** Deterministic Activity Regularization (DAR) algorithm for Continual Learning. To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task. Algorithm 1 summarizes the proposed SAR algorithm for continual learning.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory. For SAR, we instead randomly sample an index i of the previous tasks and apply the AR only on that memory. As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability. For DAR, we omit the sampling index i in step 7 and calculate the KL divergence on all the memory as in eqn. 7. The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation.

To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

Input: Figure 2: Performance results on the split CIFAR-100 benchmark.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and reviewer comments, I will assess the claim "To overcome the above problem, we propose a novel technique called 'Activity Regularization' (AR) that penalizes changes in the model's prediction on its learned information."

The evaluation reveals:
- The evidence shows AR does address continual learning challenges through KL-divergence regularization
- Results in Fig 2 demonstrate modest improvements over baseline methods
- The reviewer notes limited originality (similarities to LwF) and significance (only tested on similar domain data)
- The technique builds incrementally on existing approaches rather than being completely novel
- The word "novel" appears somewhat overstated given the similarities to existing methods

<score>3</score>

Justification:
The claim is moderately overstated primarily due to:
1. Using "novel" when the approach builds incrementally on existing methods
2. The reviewer pointing out strong similarities to LwF, with the main difference being memory storage
3. Limited testing scope (similar domain data only) questioning broad applicability
4. While the method shows improvements in results, they are modest rather than transformative
5. The core technical contribution (KL-divergence regularization) is sound but an evolution of existing approaches rather than a completely new technique