Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We also analyze how aligned natural language concepts evolve as they get represented in deeper layers.

The review comment to be evaluated is:

This paper describes a method for identifying linguistic components ("concepts") to which individual units of convolutional networks are sensitive, by selecting the sentences that most activate the given unit and then quantifying the activation of those units in response to subparts of those sentences that have been isolated and repeated. The paper reports analyses of the sensitivities of different units as well as the evolution of sensitivity across network layers, finding interesting patterns of sensitivity to specific words as well as higher-level categories.

I think this paper provides some useful insights into the specialization of hidden layer units in these networks.  There are some places where I think the analysis could go deeper / some questions that I'm left with (see comments below), but on the whole I think that the paper sheds useful light on the finer-grained picture of what these models learn internally. I like the fact that the analysis is able to identify a lack of substantial change between middle and deeper layers of the translation model, which inspires a prediction - subsequently borne out - that decreasing the number of layers will not substantially reduce task performance.

The paper is overall written pretty clearly (though some of the questions below could likely be attributed to sub-optimal clarity), and to my knowledge the analyses and insights that it contributes are original. Overall, I think this is a solid paper with some interesting contributions to neural network interpretability.

Comments/questions:

-I'm wondering about the importance of repeating the “concepts” to reach the average sentence length. Do the units not respond adequately with just one instance of the concept (eg "the ball" rather than "the ball the ball the ball")? What is the contribution of repetition alone?

-Did you experiment with any other values for M (number of aligned candidate concepts per unit)? It seems that this is a non-trivial modeling decision, as it has bearing on the interesting question of how broadly selective a unit is.

-You give examples of units that have interpretable sensitivity patterns - can you give a sense of what proportion of units do *not* respond in an interpretable way, based on your analysis?

-What exactly is plotted on the y-axis of Figure 5? Is it number of units, or number of concepts? How does it pool over different instances of a category (different morphemes, different words, etc)? What is the relationship between that measure and the number of distinct words/morphemes etc that produce sensitivity?

-I'm interested in the units that cluster members of certain syntactic and semantic categories, and it would be nice to be able to get a broader sense of the scope of these sensitivities. What examples of these categories are captured? Is it clear why certain categories are selected over others? Are they obviously the most optimal categories for task performance?

-p7 typo: "morhpeme"

The evidence to be evaluated is:Compared with these previous studies, we focus on discovering a much wider variety of _natural language concepts_, including any morphemes, words, and phrases all found in the training data.

By extending the scope of detected concepts to meaningful building blocks of natural language, we provide insights into how various linguistic features are encoded by the hidden units of deep representations.

(1) By identifying the role of the individual units, rather than analyzing the representation as a whole, we provide more fine-grained understanding of how the representations encode informative factors in training data.

We first train a CNN model for each natural language task (_e.g._ translation and classification) and retrieve training sentences that highly activate specific units. Interestingly, we discover morphemes, words and phrases that appear dominantly within these retrieved sentences, implying that those concepts have a significant impact on the activation value of the unit. Then, we find a set of concepts which attribute a lot to the unit activation by measuring activation value of each replicated candidate concept, and align them to unit.

Interestingly, some natural language patterns such as morphemes, words, phrases frequently appear in the retrieved sentences (see Figure 1), implying that those concepts might have a large attribution to the activation value of that unit.

For constructing candidate concepts, we parse each of top \(K\) sentences with a constituency parser (Kitaev and Klein, 2018). Within the constituency-based parse tree, we define candidate concepts as all terminal and non-terminal nodes (_e.g._ from sentence _John hit the balls_, we obtain candidate concepts as {_John, hit, the balls, hit the balls, John hit the balls_}). We also break each word into morphemes using a morphological analysis tool (Virpioja et al., 2013) and add them to candidate concepts (_e.g._ from word _balls_, we obtain morphemes {_ball, s_}). We repeat this process for every top \(K\) sentence and build a set of candidate concepts for unit \(u\), which is denoted as \(\mathcal{C}_{u}=\{c_{1},...,c_{N}\}\), where \(N\) is the number of candidate concepts of the unit.

Finally, the degree of alignment (DoA) between a candidate concept \(c_{n}\) and a unit \(u\) is defined as follows: \[\mathsf{DoA}_{u,c_{n}}=a_{u}(r_{n}) \tag{1}\] In short, the \(\mathsf{DoA}\)1 measures the extent to unit \(u\)'s activation is sensitive to the presence of candidate concept \(c_{n}\). If a candidate concept \(c_{n}\) appears in the top \(K\) sentences and unit's activation value \(a_{u}\) is responsive to \(c_{n}\) a lot, then \(\mathsf{DoA}_{u,c_{n}}\) gets large, suggesting that candidate concept \(c_{n}\) is strongly aligned to unit \(u\).

Finally, for each unit \(u\), we define a set of its aligned concepts \(\mathcal{C}_{u}^{*}=\{c_{1}^{*},...,c_{M}^{*}\}\) as \(M\) candidate concepts with the largest DoA values in \(C_{u}\). Depending on how we set \(M\), we can detect different numbers of concepts per unit.

We then focus on the representations in each encoder layer of ByteNet and convolutional layer of VDCNN, because as Mou et al. (2016) pointed out, the representation of the decoder (the output layer in the case of classification) is specialized for predicting the output of the target task rather than for learning the semantics of the input text. To quantitatively evaluate how well our approach aligns concepts, we measure how selectively each unit responds to the aligned concept. Motivated by Morcos et al. (2018), we define the **concept selectivity** of a unit \(u\), to a set of concepts \(\mathcal{C}_{u}^{*}\) that our alignment method detects, as follows: \[\mathsf{Sel}_{u}=\frac{\mu_{+}-\mu_{-}}{\max_{s\in\mathcal{S}}a_{u}(s)-\min_{s \in\mathcal{S}}a_{u}(s)} \tag{2}\] where \(\mathcal{S}\) denotes all sentences in training set, and \(\mu_{+}=\frac{1}{|\mathcal{S}_{+}|}\sum_{s\in\mathcal{S}_{+}}a_{u}(s)\) is the average value of unit activation when forwarding a set of sentences \(\mathcal{S}_{+}\), which is defined as one of the following: * _replicate_: \(\mathcal{S}_{+}\) contains the sentences created by replicating each concept in \(\mathcal{C}_{u}^{*}\). As before, the sentence length is set as the average length of all training sentences for fair comparison.

More interestingly, we discover that many units could capture specific meanings or syntactic roles beyond superficial, low-level patterns. For example, unit 690 of the 14th layer in ByteNet captures (_what, who, where_) concepts, all of which play a similar grammatical role. On the other hand, unit 224 of the 14th layer in ByteNet and unit 53 of the 0th layer in VDCNN each captures semantically similar concepts, with the ByteNet unit detecting the meaning of certainty in knowledge (_sure, know, aware_) and the VDCNN unit detecting years (_1999, 1969, 1992_). This suggests that, although we train character-level CNNs with feeding sentences as the form of discrete symbols (_i.e._ character indices), individual units could capture natural language concepts sharing a similar semantic or grammatical role.

We note that there are units that detect concepts more abstract than just morphemes, words, or phrases, and for these units, our method tends to align relevant lower-level concepts. For example, in unit 244 of the 3rd layer in VDCNN, while each aligned concept emerges only once in the top \(K\) sentences, all top \(K\) sentences have similar _nucances_ like positive sentiments. In this case, our method does capture relevant phrase-level concepts (e.g., _very disappointing, absolute worst place_), indicating that the higher-level _nucance_ (e.g., negativity) is indirectly captured.

Our approach thus tends to find concepts that are frequent in training data or considered as more important than others for solving the target task. Overall, these results suggest how input sentences are represented in the hidden layers of the CNN: * Several units in the CNN learned on NLP tasks respond selectively to specific natural language concepts, rather than getting activated in an uninterpretable way. This means that these units can serve as detectors for specific natural language concepts.

Using the concept alignments found earlier, we can visualize how concepts are distributed across layers. Figure 4 shows the concepts of the units in the 0th, 1st, 3rd layer of VDCNN learned on AG-News dataset, and 0th, 4th, and 14th layer of the ByteNet encoder learned on English-to-German Europarl dataset with their number of aligned units. For each layer, we sort concepts in decreasing order by the number of aligned units and show 30 concepts most aligned.

Overall, we find that data and task-specific concepts are likely to be aligned to many units. In AG News, since the task is to classify given sentences into following categories; _World_, _Sports_, _Business_ and _Science/Tech_, concepts related to these topics commonly emerge. Similarly, we can see that units learned for Europarl dataset focus to encode some key words (_e.g. vote_, _propose_, _environment_) in the training corpus.

To confirm that it also holds for representations learned in NLP tasks, we divide granularity of natural language concepts to the morpheme, word and \(N\)-gram phrase (\(N=2,3,4,5\)), and observe the number of units that they are aligned in different layers.

This trend coincides with the result from Figure 5 that the evolution of concept granularity stops around middle-to-higher layers.

In deeper models, this may suggest that the middle layer's representation may be already informative enough to encode the input text, and our result may partly coincide with that of Mou et al. (2016), which shows that representation of intermediate layers is more transferable than that of deeper layers in language tasks, unlike in computer vision where deeper layers are usually more useful and discriminative.

We observe that the concepts do not appear uniformly; some concepts are aligned to many units, while others are aligned to few or even no units.

First, the concepts with a higher frequency in training data may be aligned to more units. Figure 7-(a) shows the correlation between the frequency of each concept in the training corpus and the number of units where each concept is aligned in the last layer of the topic classification model learned on AG News dataset. Second, the concepts that have more influence on the objective function (expected loss) may be aligned to more units.

We proposed a simple but highly effective concept alignment method for character-level CNNs to confirm that each unit of the hidden layers serves as detectors of natural language concepts. Using this method, we analyzed the characteristics of units with multiple datasets on classification and translation tasks. Consequently, we shed light on how deep representations capture the natural language, and how they vary with various conditions.

In particular, combining our definition of concepts with the attention mechanism (_e.g._ Bahdanau et al. (2015)) could be a promising direction, because it can reveal how the representations are attended by the model to capture concepts, helping us better understand the decision-making process of popular deep models.

In section 3.2, we define Degree of Alignment (DoA) between concept \(c_{n}\) and unit \(u\) as activation value of unit \(u\) for replication of \(c_{n}\).

We define the **degree of alignment (DoA)** between a concept \(c\in\mathcal{C}\) and a unit \(u\) as: \[\text{DoA}_{u,c_{n}}=\frac{1}{Z}\sum_{s\in\mathcal{S}}\mathbb{1}(c_{n}\in s)( a_{u}(s)-a_{u}(\text{Occ}_{c_{n}}(s))) \tag{6}\] where \(Z\) is a normalizing factor, and \(a_{u}\) indicates the mean activation of unit \(u\), \(\text{Occ}_{c_{n}}(s)\) is a sentence \(s\) where candidate concept \(c_{n}\) is occluded, and \(\mathbb{1}(c_{n}\in s)\) is an indicator of whether \(c_{n}\) is included in the sentence \(s\). In short, the DoA measures how much a candidate concept contributes to the activation of the unit's top \(K\) sentences. If a candidate concept \(c_{n}\) appears in the topsentences \(\mathcal{S}\) and greatly reduces the activation of unit \(u\), then \(\mathsf{DoA}_{u,c_{n}}\) gets large, implying that the \(c_{n}\) is strongly aligned to unit \(u\).

In our experiments, we analyze representations of each convolutional block layer.

We here refer _concept cluster_ to the concepts that are aligned to the same unit and have similar semantics or grammatical roles. We analyze how clusters are formed in the units and how they vary with the target task and layer depth.

Since the concepts in the x, y-axes are ordered by the clustering result, if the diagonal blocks (concept clusters) emerge more strongly, the concepts in the same unit are more likely to have the similar meanings. In Figure 11, the units learned in the classification tasks tend to have stronger concept clusters than those learned in the translation tasks. Particularly, the concept clusters are highly evident in the units learned in DBpedia and AG News dataset.

We analyze how concept clusters change by layer in each task. We compute the averaged pairwise distance between the concepts in each layer.

In section 4.4, we visualized how concepts are distributed across layers, where the model is trained on AG News dataset and English-to-German Europarl dataset.

In the classification tasks, we expect to find more concepts that are directly related to predicting the output label, as opposed to the translation tasks where the representations may have to include information on most of the words for an accurate translation. While our goal is not to relate each concept to one of the labels, we find several concepts that are more predictive to a particular label than others. Consistent with section 4.4, there are data-specific and task-specific concepts aligned in each layer; _i.e._ {_worst, 2 stars, awful_} at Yelp Review, {_film, ship, school_} at DBpedia, and some key words at translation datasets.

Input: Figure 4: 30 concepts selected by the number of aligned units in three encoding layers of ByteNet learned on the Europarl translation dataset (left column) and VDCNN learned on AG-News (right column). [#] symbol denotes morpheme concepts. See section 4.4 for details.

Input: Figure 10: Examples of non-interpretable units, their concepts and top 5 activated sentences. Units are from representations learned on english-to-french translation dataset. See section D for details.

Input: Figure 1: We discover the most activated sentences and aligned concepts to the units in hidden representations of deep convolutional networks. Aligned concepts appear frequently in most activated sentences, implying that those units respond selectively to specific natural language concepts.

Input: Figure 14: The Pearson correlations between the number of units per concept and (i) document frequency (top row), (ii) delta of expected loss (bottom row). They are measured at the final layer representation.

Input: Figure 15: 30 concepts selected by the number of aligned units in four encoding layers in VDCNN learned on Yelp Review dataset and DBpedia ontology dataset, and ByteNet learned on the English-to-German, English-to-French, and English-to-Czech parallel corpus. [#] symbol denotes morpheme concept.

Input: Figure 12: Averaged pairwise distances of concepts in each layer per task. Top and bottom row show the concept distances of translation models and classification models, respectively. For projecting concepts into the embedding space, we use three pretrained embedding: Glove, ConceptNet, FastText. See Appendix E.2 for more details.

Input: Figure 13: The number of unique concepts in each layer. It increases with the layer depth, which implies that the units in a deeper layer represent more diverse concepts.

Input: Figure 16: Aligned concepts per each task and their number of occurrences over multiple layers. See Appendix H for more details.

Input: Figure 11: Concept clusters of the last layer representations learned in each task. The more distinct the diagonal blocks are, the stronger the tendency that concepts aligned to the same unit share similar meanings or semantics. See Appendix E for details.

Input: Figure 7: Correlations between the number of units per concept and (a) document frequency and (b) delta of expected loss. Pearson correlation coefficients are measured at the final layer of the AG News sentence classification task. Some concepts aligned to many units are annotated. Results on other tasks are available in Appendix F.

Input: Figure 11: Concept clusters of the last layer representations learned in each task. The more distinct the diagonal blocks are, the stronger the tendency that concepts aligned to the same unit share similar meanings or semantics. See Appendix E for details.

Input: Figure 3: Examples of top activated sentences and aligned concepts to some units in several encoding layers of ByteNet and VDCNN. For each unit, concepts and their presence in top K sentences are shown in the same color. [#] symbol denotes morpheme concepts. See section 4.3 for details.

Input: Figure 15: 30 concepts selected by the number of aligned units in four encoding layers in VDCNN learned on Yelp Review dataset and DBpedia ontology dataset, and ByteNet learned on the English-to-German, English-to-French, and English-to-Czech parallel corpus. [#] symbol denotes morpheme concept.

Input: Figure 12: Averaged pairwise distances of concepts in each layer per task. Top and bottom row show the concept distances of translation models and classification models, respectively. For projecting concepts into the embedding space, we use three pretrained embedding: Glove, ConceptNet, FastText. See Appendix E.2 for more details.

Input: Figure 13: The number of unique concepts in each layer. It increases with the layer depth, which implies that the units in a deeper layer represent more diverse concepts.

Input: Figure 16: Aligned concepts per each task and their number of occurrences over multiple layers. See Appendix H for more details.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence, the claim "We also analyze how aligned natural language concepts evolve as they get represented in deeper layers" appears to be well-stated. The paper provides extensive analysis of how concepts are distributed and evolve across network layers, supported by:

- Multiple visualizations showing concept distributions across layers (Figures 4, 12, 13, 15)
- Quantitative analysis of concept clustering and distances between layers
- Clear demonstration of how concept granularity and diversity changes with layer depth
- Thorough examination across different tasks and model architectures

The claim accurately reflects the scope and depth of the analysis without exaggeration.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

As part of our analysis, we show that our interpretation of learned representations could be utilized at designing network architectures with fewer parameters but with comparable performance to baseline models.

The evidence to be evaluated is:(1) By identifying the role of the individual units, rather than analyzing the representation as a whole, we provide more fine-grained understanding of how the representations encode informative factors in training data.

Compared to deep architectures based on fully connected layers, CNNs are natural candidates for unit-level analysis because their channel-level representations are reported to work as templates for detecting concepts (Bau et al., 2017).

We first train a CNN model for each natural language task (_e.g._ translation and classification) and retrieve training sentences that highly activate specific units. Interestingly, we discover morphemes, words and phrases that appear dominantly within these retrieved sentences, implying that those concepts have a significant impact on the activation value of the unit.

More interestingly, we discover that many units could capture specific meanings or syntactic roles beyond superficial, low-level patterns. For example, unit 690 of the 14th layer in ByteNet captures (_what, who, where_) concepts, all of which play a similar grammatical role. On the other hand, unit 224 of the 14th layer in ByteNet and unit 53 of the 0th layer in VDCNN each captures semantically similar concepts, with the ByteNet unit detecting the meaning of certainty in knowledge (_sure, know, aware_) and the VDCNN unit detecting years (_1999, 1969, 1992_). This suggests that, although we train character-level CNNs with feeding sentences as the form of discrete symbols (_i.e._ character indices), individual units could capture natural language concepts sharing a similar semantic or grammatical role.

Our approach thus tends to find concepts that are frequent in training data or considered as more important than others for solving the target task. Overall, these results suggest how input sentences are represented in the hidden layers of the CNN: * Several units in the CNN learned on NLP tasks respond selectively to specific natural language concepts, rather than getting activated in an uninterpretable way. This means that these units can serve as detectors for specific natural language concepts.

One possibility is that the capacity of the network is large enough so that the representations in the middle layers could be sufficiently informative to solve the task.

The performance of the translation model does not change much with more than six encoder layers, but it significantly drops at the models with fewer than 4 encoder layers.

This shared pattern suggests that about six encoder layers are enough to encode informative factors in the given datasets to perform optimally on the translation task. In deeper models, this may suggest that the middle layer's representation may be already informative enough to encode the input text, and our result may partly coincide with that of Mou et al. (2016), which shows that representation of intermediate layers is more transferable than that of deeper layers in language tasks, unlike in computer vision where deeper layers are usually more useful and discriminative.

We proposed a simple but highly effective concept alignment method for character-level CNNs to confirm that each unit of the hidden layers serves as detectors of natural language concepts. Using this method, we analyzed the characteristics of units with multiple datasets on classification and translation tasks. Consequently, we shed light on how deep representations capture the natural language, and how they vary with various conditions.

In our experiments, we analyze representations of each convolutional block layer.

For each of the AG News, Yelp Reviews and DBpedia datasets, a VDCNN was learned with the same structure and hyperparameters.

Input: Figure 6: BLEU scores on the validation data for three translation models. We train ByteNet from scratch on each dataset by varying the number of encoder layers.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be somewhat overstated. While the evidence shows that 6 encoder layers are sufficient for optimal translation performance (as demonstrated in the graph), and that CNN units can detect specific language concepts, the evidence does not directly demonstrate that this understanding was "utilized at designing network architectures with fewer parameters but with comparable performance." The evidence shows that fewer layers are possible, but doesn't explicitly connect concept interpretation to architecture design decisions or compare parameter counts between models. The claim makes a stronger causal connection than the evidence supports.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

As part of our analysis, we show that our interpretation of learned representations could be utilized at designing network architectures with fewer parameters but with comparable performance to baseline models.

The review comment to be evaluated is:

The paper is well written and structured, presenting the problem clearly and accurately. It contains considerable relevant references and enough background knowledge. It nicely motivates the proposed approach, locates the contributions in the state-of-the-art and reviews related work. It is also very honest in terms of how it differs on the technical level from existing approaches.
The paper presents interesting and novel findings to further state-of-the-art’s understanding on how language concepts are represented in the intermediate layers of deep convolutional neural networks, showing that channels in convolutional representations are selectively sensitive to specific natural language concepts. It also nicely discusses how concepts granularity evolves with layers’ deepness in the case of natural language tasks.
What I am missing, however, is an empirical study of concepts coverage over multiple layers, studying the multiple occurrences of single concepts at different layers, and a deeper dive on the rather noisy elements of natural language and the layers’ activation dynamics towards such elements.
Overall, however, the ideas presented in the paper are interesting and original, and the experimental section is convincing. My recommendation is to accept this submission.

The evidence to be evaluated is:(1) By identifying the role of the individual units, rather than analyzing the representation as a whole, we provide more fine-grained understanding of how the representations encode informative factors in training data.

Compared to deep architectures based on fully connected layers, CNNs are natural candidates for unit-level analysis because their channel-level representations are reported to work as templates for detecting concepts (Bau et al., 2017).

We first train a CNN model for each natural language task (_e.g._ translation and classification) and retrieve training sentences that highly activate specific units. Interestingly, we discover morphemes, words and phrases that appear dominantly within these retrieved sentences, implying that those concepts have a significant impact on the activation value of the unit.

More interestingly, we discover that many units could capture specific meanings or syntactic roles beyond superficial, low-level patterns. For example, unit 690 of the 14th layer in ByteNet captures (_what, who, where_) concepts, all of which play a similar grammatical role. On the other hand, unit 224 of the 14th layer in ByteNet and unit 53 of the 0th layer in VDCNN each captures semantically similar concepts, with the ByteNet unit detecting the meaning of certainty in knowledge (_sure, know, aware_) and the VDCNN unit detecting years (_1999, 1969, 1992_). This suggests that, although we train character-level CNNs with feeding sentences as the form of discrete symbols (_i.e._ character indices), individual units could capture natural language concepts sharing a similar semantic or grammatical role.

Our approach thus tends to find concepts that are frequent in training data or considered as more important than others for solving the target task. Overall, these results suggest how input sentences are represented in the hidden layers of the CNN: * Several units in the CNN learned on NLP tasks respond selectively to specific natural language concepts, rather than getting activated in an uninterpretable way. This means that these units can serve as detectors for specific natural language concepts.

One possibility is that the capacity of the network is large enough so that the representations in the middle layers could be sufficiently informative to solve the task.

The performance of the translation model does not change much with more than six encoder layers, but it significantly drops at the models with fewer than 4 encoder layers.

This shared pattern suggests that about six encoder layers are enough to encode informative factors in the given datasets to perform optimally on the translation task. In deeper models, this may suggest that the middle layer's representation may be already informative enough to encode the input text, and our result may partly coincide with that of Mou et al. (2016), which shows that representation of intermediate layers is more transferable than that of deeper layers in language tasks, unlike in computer vision where deeper layers are usually more useful and discriminative.

We proposed a simple but highly effective concept alignment method for character-level CNNs to confirm that each unit of the hidden layers serves as detectors of natural language concepts. Using this method, we analyzed the characteristics of units with multiple datasets on classification and translation tasks. Consequently, we shed light on how deep representations capture the natural language, and how they vary with various conditions.

In our experiments, we analyze representations of each convolutional block layer.

For each of the AG News, Yelp Reviews and DBpedia datasets, a VDCNN was learned with the same structure and hyperparameters.

Input: Figure 6: BLEU scores on the validation data for three translation models. We train ByteNet from scratch on each dataset by varying the number of encoder layers.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be somewhat overstated. While the evidence shows that CNN representations can detect specific language concepts and that performance plateaus after 6 encoder layers (suggesting some parameter efficiency), there is no direct experimental evidence demonstrating that this understanding was actually utilized to design more efficient architectures. The paper shows analysis of existing architectures but doesn't provide concrete examples of using these insights to create new, more efficient networks while maintaining comparable performance.

The review comment, while positive about the analysis aspects, also doesn't mention any evidence supporting the practical application of these insights into new architecture design.

The graph shows translation performance across different numbers of encoder layers, which supports the finding about layer efficiency, but doesn't demonstrate the claim about utilizing this knowledge to design better architectures.

<score>4</score>

The score of 4 reflects that while the underlying analysis is valuable and suggests potential for architectural optimization, the claim overstates the actual demonstrated achievements by implying direct utilization of these insights in network design, when the evidence only shows analysis and potential implications rather than implemented results.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

As part of our analysis, we show that our interpretation of learned representations could be utilized at designing network architectures with fewer parameters but with comparable performance to baseline models.

The review comment to be evaluated is:

========== Edit following authors' response  ==========

Thank you for your detailed response and updated version. I think the new revision is significantly improved, mainly in more quantitative analyses and details in several places. I have updated my evaluation accordingly.

See a few more points below.

1. Thank you for clarifying your definition of concepts. I still think that the word "concept" has a strong semantic connotation, while the linguistic elements your analyses capture may do other things. The results in appendix E do show that some semantic clusters arise. It's especially interesting to see the blocks in some of the heat maps, where similar "concepts" are clustered together (like the sports terms in AG); consider commenting on this.

2. The new quantitative analyses are helpful. One other suggestion that I mentioned before is to connect detected concepts to external resources like WordNet or ConceptNet. That would help show that "concepts" are indeed semantic objects.

3. The motivation for replicating as normalizing for length does make sense, although the input would still be unnatural. The comparison to "one instance" is helpful, but it's interesting that the differences between it and replication in figure 2 are not large. It would be good to show results that substantiate your assumption that without replication there will be a bias towards lengthy concepts. Does "one instance" detect more lengthy concepts than replication?

4. The results on frequency and loss difference in 4.5 are very interesting. There is another angle to consider frequency: words that appear frequently often carry less semantic content (e.g. function words), so one might conjecture that they would require less units. It may be interesting to look at which concepts are detected at each frequency bin.

5. Minor points: section 2.2 still mentions "regression" where it should be "classification".

6. A few remaining grammar issues:
- "one concept has a less activation value.." - rephrase
- end of section 3.3: "this experiments" -> "these experiments"

========== Original review follows ==========

Summary:
=======
This paper analyzes individual units in CNN models for text classification and translation tasks. It defines a measure of sensitivity for a unit and evaluates how sensitive each unit is to "concepts" in the input text, where concepts are morphemes, words, and phrases. The analysis shows that some units seem to learn semantic concepts, while others capture linguistic elements that are frequent or relevant for the end task. Layer-wise results show some correspondence between layer depth and linguistic element size.

The paper studies an important question that is relatively under-studies in NLP compared to the computer vision community. The motivation for the work is quite convincing.
I found some of the results and analysis interesting, but overall felt that the work can be made much stronger by more quantitative evaluations. I am also worried that the notion of "concept" is misleading here. See below for this and other comments. I am willing to reconsider my evaluation pending response to the below issues.

Main comments:
=============
1. Concepts:
- morphemes, words, and phrases - are these "concepts"? They are indeed "fundamental building blocks of natural language" (2.2), but "concepts" has a more semantic connotation that I'm not sure these units target at.
- Some of the results do suggest that units learn concepts, as the analysis in 4.2 shows a "unit detecting the meaning of certainty in knowledge" and later units that have similar sentiments. It would be informative to quantify this in some way, for example by matching detected concepts to WordNet synsets, sentiment lexicons, etc., or else tagging and classifying them with various NLP tools. This could also reveal if units learn more syntactic or semantic concepts, and so on.
2. Generally, many of the analyses in the paper are qualitative and on a small scale. The results will be more convincing with more automatic aggregate measures.
3. The structure of the paper is confusing. Section3 starts with the approach but then mentions datasets and tasks (3.1). Section 4 is titled experiments, but section 4.1 starts with defining the concept selectivity. I would suggest reorganizing sections 3 and 4, such that section 3 describes all the methods and metrics, while dataset-specific parts are moved to section 4.
4. section 3.2 should provide more details on the sentence representation and how its obtained in the CNN models. A mathematical derivation and/or figure could be helpful. It is also not clear to me what's the motivation for mean-pooling over the l entries of the vector.
5. section 3.3: the use of replicated text for "concept alignment" is puzzling. This is not a natural input to the model, and I think more justification and motivation åre needed for this issue, as well as perhaps comparison with other approaches.
6. I found section 4.4 very interesting. It shows some intuitive results of larger linguistic elements learned at higher layers, but then some results that do not show such a trend. Then, hypothesizing that the middle layers are sufficient AND validating the hypothesis by retraining the model is excellent. It's a very nice demonstration that the analysis can lead to model improvements.
7. Figure 2 seems to be almost caused by construction of the different options for S_+. Is it surprising that the replicate set has the highest sensitivity? Is there a better control setup than comparing with a random set?
8. One concern that I have is the effect of confounding factors like frequency on the results. The papers occasionally attributes importance to concepts (e.g. in 4.2), but I wonder if instead we may be seeing more frequent words. Controlling for the effect of frequency would be useful.

Minor comments:
==============
- Section 2.2, first paragraph: regression should be changed to classification
- The related work is generally relevant, although one could mention a few other papers that analyzed individual neurons in NLP tasks [1, 2]
- section 4.1: the random set may perhaps be denoted by something more neutral, not S_+ as the replicate and inclusion sets.
- section 4.3, last paragraph: listing examples showing that units in Europarl focus on key words would be good.
- Figure 5, y axis label: should this be number of units instead of concepts?
- Appendix A has several interesting points but there is no reference to them from the main paper.

Writing, grammar, etc.:
======================
- Introduction: among them - who is them?
- 2.1: motivated from -> motivated by; In computer vision community -> In the computer vision community
- 2.1: quantifying characteristics of representations in layer-wise -> rephrase
- 3.2: dimension of sentence -> dimension of the/a sentence
- 4.1: to which -> remove "which"
- 4.2: in the several encoding layer -> in several encoding layers
- 4.3: aliged -> aligned
- Capitalize titles in references
- A.2: with following -> with the following; how much candidate -> how much a candidate; consider following -> consider the following
- A.3: induces similar bias -> induces a bias; such phrase -> such a phrase; on very -> on a very
- C: where model -> where the model; In consistent -> Consistent; where model -> where the model

References
==========
[1] Qian et al., Analyzing linguistic knowledge in sequential model of sentence
[2] Shi et al., Why Neural Translations are the Right Length

The evidence to be evaluated is:(1) By identifying the role of the individual units, rather than analyzing the representation as a whole, we provide more fine-grained understanding of how the representations encode informative factors in training data.

Compared to deep architectures based on fully connected layers, CNNs are natural candidates for unit-level analysis because their channel-level representations are reported to work as templates for detecting concepts (Bau et al., 2017).

We first train a CNN model for each natural language task (_e.g._ translation and classification) and retrieve training sentences that highly activate specific units. Interestingly, we discover morphemes, words and phrases that appear dominantly within these retrieved sentences, implying that those concepts have a significant impact on the activation value of the unit.

More interestingly, we discover that many units could capture specific meanings or syntactic roles beyond superficial, low-level patterns. For example, unit 690 of the 14th layer in ByteNet captures (_what, who, where_) concepts, all of which play a similar grammatical role. On the other hand, unit 224 of the 14th layer in ByteNet and unit 53 of the 0th layer in VDCNN each captures semantically similar concepts, with the ByteNet unit detecting the meaning of certainty in knowledge (_sure, know, aware_) and the VDCNN unit detecting years (_1999, 1969, 1992_). This suggests that, although we train character-level CNNs with feeding sentences as the form of discrete symbols (_i.e._ character indices), individual units could capture natural language concepts sharing a similar semantic or grammatical role.

Our approach thus tends to find concepts that are frequent in training data or considered as more important than others for solving the target task. Overall, these results suggest how input sentences are represented in the hidden layers of the CNN: * Several units in the CNN learned on NLP tasks respond selectively to specific natural language concepts, rather than getting activated in an uninterpretable way. This means that these units can serve as detectors for specific natural language concepts.

One possibility is that the capacity of the network is large enough so that the representations in the middle layers could be sufficiently informative to solve the task.

The performance of the translation model does not change much with more than six encoder layers, but it significantly drops at the models with fewer than 4 encoder layers.

This shared pattern suggests that about six encoder layers are enough to encode informative factors in the given datasets to perform optimally on the translation task. In deeper models, this may suggest that the middle layer's representation may be already informative enough to encode the input text, and our result may partly coincide with that of Mou et al. (2016), which shows that representation of intermediate layers is more transferable than that of deeper layers in language tasks, unlike in computer vision where deeper layers are usually more useful and discriminative.

We proposed a simple but highly effective concept alignment method for character-level CNNs to confirm that each unit of the hidden layers serves as detectors of natural language concepts. Using this method, we analyzed the characteristics of units with multiple datasets on classification and translation tasks. Consequently, we shed light on how deep representations capture the natural language, and how they vary with various conditions.

In our experiments, we analyze representations of each convolutional block layer.

For each of the AG News, Yelp Reviews and DBpedia datasets, a VDCNN was learned with the same structure and hyperparameters.

Input: Figure 6: BLEU scores on the validation data for three translation models. We train ByteNet from scratch on each dataset by varying the number of encoder layers.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be moderately overstated. While the evidence shows that reducing encoder layers to about 6 layers maintains reasonable performance (as shown in the BLEU score graph), the claim suggests this insight was derived from their "interpretation of learned representations." However, the evidence primarily shows empirical performance results rather than clear insights from representation analysis leading to architecture decisions. The connection between their representational analysis and the architecture optimization is not strongly substantiated in the provided evidence.

The graph does support that fewer parameters (via reduced layers) can maintain comparable performance, but attributing this finding specifically to their interpretation methods appears to overstate the connection.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

As part of our analysis, we show that our interpretation of learned representations could be utilized at designing network architectures with fewer parameters but with comparable performance to baseline models.

The review comment to be evaluated is:

This paper describes a method for identifying linguistic components ("concepts") to which individual units of convolutional networks are sensitive, by selecting the sentences that most activate the given unit and then quantifying the activation of those units in response to subparts of those sentences that have been isolated and repeated. The paper reports analyses of the sensitivities of different units as well as the evolution of sensitivity across network layers, finding interesting patterns of sensitivity to specific words as well as higher-level categories.

I think this paper provides some useful insights into the specialization of hidden layer units in these networks.  There are some places where I think the analysis could go deeper / some questions that I'm left with (see comments below), but on the whole I think that the paper sheds useful light on the finer-grained picture of what these models learn internally. I like the fact that the analysis is able to identify a lack of substantial change between middle and deeper layers of the translation model, which inspires a prediction - subsequently borne out - that decreasing the number of layers will not substantially reduce task performance.

The paper is overall written pretty clearly (though some of the questions below could likely be attributed to sub-optimal clarity), and to my knowledge the analyses and insights that it contributes are original. Overall, I think this is a solid paper with some interesting contributions to neural network interpretability.

Comments/questions:

-I'm wondering about the importance of repeating the “concepts” to reach the average sentence length. Do the units not respond adequately with just one instance of the concept (eg "the ball" rather than "the ball the ball the ball")? What is the contribution of repetition alone?

-Did you experiment with any other values for M (number of aligned candidate concepts per unit)? It seems that this is a non-trivial modeling decision, as it has bearing on the interesting question of how broadly selective a unit is.

-You give examples of units that have interpretable sensitivity patterns - can you give a sense of what proportion of units do *not* respond in an interpretable way, based on your analysis?

-What exactly is plotted on the y-axis of Figure 5? Is it number of units, or number of concepts? How does it pool over different instances of a category (different morphemes, different words, etc)? What is the relationship between that measure and the number of distinct words/morphemes etc that produce sensitivity?

-I'm interested in the units that cluster members of certain syntactic and semantic categories, and it would be nice to be able to get a broader sense of the scope of these sensitivities. What examples of these categories are captured? Is it clear why certain categories are selected over others? Are they obviously the most optimal categories for task performance?

-p7 typo: "morhpeme"

The evidence to be evaluated is:(1) By identifying the role of the individual units, rather than analyzing the representation as a whole, we provide more fine-grained understanding of how the representations encode informative factors in training data.

Compared to deep architectures based on fully connected layers, CNNs are natural candidates for unit-level analysis because their channel-level representations are reported to work as templates for detecting concepts (Bau et al., 2017).

We first train a CNN model for each natural language task (_e.g._ translation and classification) and retrieve training sentences that highly activate specific units. Interestingly, we discover morphemes, words and phrases that appear dominantly within these retrieved sentences, implying that those concepts have a significant impact on the activation value of the unit.

More interestingly, we discover that many units could capture specific meanings or syntactic roles beyond superficial, low-level patterns. For example, unit 690 of the 14th layer in ByteNet captures (_what, who, where_) concepts, all of which play a similar grammatical role. On the other hand, unit 224 of the 14th layer in ByteNet and unit 53 of the 0th layer in VDCNN each captures semantically similar concepts, with the ByteNet unit detecting the meaning of certainty in knowledge (_sure, know, aware_) and the VDCNN unit detecting years (_1999, 1969, 1992_). This suggests that, although we train character-level CNNs with feeding sentences as the form of discrete symbols (_i.e._ character indices), individual units could capture natural language concepts sharing a similar semantic or grammatical role.

Our approach thus tends to find concepts that are frequent in training data or considered as more important than others for solving the target task. Overall, these results suggest how input sentences are represented in the hidden layers of the CNN: * Several units in the CNN learned on NLP tasks respond selectively to specific natural language concepts, rather than getting activated in an uninterpretable way. This means that these units can serve as detectors for specific natural language concepts.

One possibility is that the capacity of the network is large enough so that the representations in the middle layers could be sufficiently informative to solve the task.

The performance of the translation model does not change much with more than six encoder layers, but it significantly drops at the models with fewer than 4 encoder layers.

This shared pattern suggests that about six encoder layers are enough to encode informative factors in the given datasets to perform optimally on the translation task. In deeper models, this may suggest that the middle layer's representation may be already informative enough to encode the input text, and our result may partly coincide with that of Mou et al. (2016), which shows that representation of intermediate layers is more transferable than that of deeper layers in language tasks, unlike in computer vision where deeper layers are usually more useful and discriminative.

We proposed a simple but highly effective concept alignment method for character-level CNNs to confirm that each unit of the hidden layers serves as detectors of natural language concepts. Using this method, we analyzed the characteristics of units with multiple datasets on classification and translation tasks. Consequently, we shed light on how deep representations capture the natural language, and how they vary with various conditions.

In our experiments, we analyze representations of each convolutional block layer.

For each of the AG News, Yelp Reviews and DBpedia datasets, a VDCNN was learned with the same structure and hyperparameters.

Input: Figure 6: BLEU scores on the validation data for three translation models. We train ByteNet from scratch on each dataset by varying the number of encoder layers.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the claim is moderately well-stated but has some degree of overstatement. The evidence shows that reducing encoder layers to 6 (from 15) maintains comparable performance in translation tasks, as demonstrated by the BLEU scores graph. However, the claim implies this finding is directly derived from their "interpretation of learned representations," when the evidence suggests this was more of an empirical observation after noticing patterns in middle layers. The connection between representation analysis and architecture optimization isn't as strongly established as the claim suggests.

Key evidence:
- Performance remains stable with 6+ encoder layers
- Middle layer representations appear sufficiently informative
- No direct causal link shown between representation analysis and architecture decisions

<score>3</score>

The score of 3 reflects that while the claim has merit based on the empirical results, it somewhat overstates the direct connection between their interpretability methods and architecture optimization decisions. The evidence supports the possibility of using fewer parameters, but the causal relationship with their interpretation method isn't strongly demonstrated.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning.

The evidence to be evaluated is:To avoid forgetting previous knowledge, when learning task \(\mathcal{T}_{t}\), the learner \(\mathbf{\theta}\) minimize the multi-task learning objective as \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\lambda_{t-1}\mathcal{L}_{t-1}(\mathbf{\theta},\mathcal{D}_{t-1})+\ldots+ \lambda_{1}\mathcal{L}_{1}(\mathbf{\theta},\mathcal{D}_{1}), \tag{1}\] where the loss \(\mathcal{L}_{i}(\mathbf{\theta},\mathcal{D}_{i})\) is usually the negative log-likelihood \(-\sum_{\mathbf{x},y\in\mathcal{D}_{i}}\log p(y|\mathbf{x};\mathbf{\theta})\) of the corresponding training data and \(\lambda_{i}\) regulates the importance of task \(\mathcal{T}_{i}\) to all the tasks observed so far. In continual learning, at time \(t\), we do not have access to the data \(\mathcal{D}_{k<t}\) to evaluate the loss \(\mathcal{L}_{k<t}\), thus, we need to approximate the past losses \(\mathcal{L}_{k<t}(\mathbf{\theta},\mathcal{D}_{k})\) when minimizing \(\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_{t})\). A common choice is to employ Laplace propagation (Eskin et al., 2004) of the negative log posterior \[-\log p(\mathbf{\theta}|\mathcal{D}_{i})\approx-\log p(\mathbf{\theta}_{i}^{*}| \mathcal{D}_{i})+\frac{1}{2}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}( \mathbf{\theta}-\mathbf{\theta}_{i}^{*}), \tag{2}\] where \(\mathbf{\theta}_{i}^{*}\) is a mode of the log posterior and \(\mathbf{F}_{i}\) is the empirical Fisher information matrix evaluated at \(\mathbf{\theta}=\mathbf{\theta}_{i}^{*}\). If we assume an uniform prior on \(\mathbf{\theta}\), the posterior coined with the likelihood 1, thus by substituting eqn. 2 to 1, the objective for training task \(\mathcal{T}_{t}\) becomes

\[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\sum_{i<t}\lambda_{k}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}(\bm {\theta}-\mathbf{\theta}_{i}^{*}), \tag{3}\] where the terms \(-\log p(\mathbf{\theta}_{i}^{*}|\mathcal{D}_{i})\) are omitted because they are constant with respect to \(\mathbf{\theta}\).

Eqn. 3 results in a wide range of methods for continual learning, each of which proposes a different method to approximate the Fisher information matrix. For example, Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017) assumes the Fisher is diagonal and approximates it using the identity: \[\mathbf{F}_{t}\approx\text{diag}\left(\sum_{n=1}^{N}\left.\left(\nabla_{\mathbf{ \theta}}\log p(\mathbf{y}_{t}^{(n)}|\mathbf{x}_{t}^{(n)};\mathbf{\theta})\right)^{2} \right|_{\mathbf{\theta}=\mathbf{\theta}_{T}}\right) \tag{4}\] Instead of using the Fisher's identity, Synaptic Intelligence (SI) (Zenke et al., 2017) directly measures the sensitivity of a parameter \(\mathbf{\theta}_{t}^{k}\) to the loss function at task \(\mathcal{T}_{t}\) through out the whole training trajectory. Zenke et al. (2017) also shows that this estimation is equivalent to measuring the full Fisher matrix on whole data set under certain choices of the loss function. Another approach to achieve the approximated multi-task learning objective in eqn. 1 is based on the observation that at the beginning of task \(\mathcal{T}_{t}\), we initialize \(\mathbf{\theta}=\mathbf{\theta}_{t-1}^{*}\), whose value already minimized the lost \(\tilde{\mathcal{L}}_{t-1}\). From this, if we can consider eqn. 1 as minimizing the loss \(\mathcal{L}_{t}(\mathbf{\theta})\) with the constraints that previous losses are not allowed to increase after every mini-batch update, which is the GEM objective. In particular, GEM optimize the following problem \[\begin{array}{ll}\text{Minimize}_{\mathbf{\theta},\mathcal{D}_{t}}&\mathcal{L}_ {t}(\mathbf{\theta})\\ \text{subject to}&\mathcal{L}_{k}(\mathbf{\theta},\mathcal{M}_{k})\leq\mathcal{L}_ {k}(\mathbf{\theta}_{t-1}^{*},\mathcal{M}_{k}),\forall k<t,\end{array} \tag{5}\]where the loss \(\mathcal{L}_{k}\) is evaluated at the memory \(\mathcal{M}_{k}\) that stores some of the training samples from task \(\mathcal{T}_{k}\). GEM further converts the constraints in eqn. 5 into the gradient constraints of the losses and solve them using quadratic programming. Finally, the type of losses \(\mathcal{L}_{k<t}\) can be flexibly changed to other loss rather than the negative log-likelihood. For example, Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes a form of distillation loss by minimizing the KL-divergence of past models and current model on the newly observed tasks. By changing the losses \(\mathcal{L}_{k<t}\), we can recover different solutions for continual learning in the literature.

To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks. This can also be interpreted as distilling the knowledge from \(\mathbf{\theta}_{i}^{*}\) to \(\mathbf{\theta}\) on the corresponding data \(\mathcal{D}_{i}\). In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

We refer to this algorithm as "Deterministic Activity Regularization" (DAR) which imposes regularization constraints for all the previous tasks at every learning iteration.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models.

7:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{D }}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{i<t}\sum_{\mathbf{x}\in\mathcal{M} _{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory. For SAR, we instead randomly sample an index i of the previous tasks and apply the AR only on that memory. As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability. For DAR, we omit the sampling index i in step 7 and calculate the KL divergence on all the memory as in eqn. 7. The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\) 9:\(\mathbf{\theta}\leftarrow\mathbf{\theta}-\alpha\cdot\text{SGD}(\mathbf{\theta},g_{\mathbf{ \theta}})\) 10:return\(\mathbf{\theta}\) ```**Algorithm 2** Stochastic Activity Regularization (SAR) algorithm for Continual Learning.

Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes to minimize the difference in KL-divergence of the current and previous models on the new tasks, which is a form of knowledge distillation penalty. Similarly, Less-forgetting learning (LF) (Jung et al., 2019) proposes to minimize the \(\ell_{2}\) norm of the two prediction in stead of KL-divergence as LwF. Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation.

While there are some overlapping between Bayesian learning and our approximated multi-task learning framework, the main difference is that our objective allows us to actively control the contribution of previous tasks by setting the importance value \(\lambda_{i}\) for each loss \(\mathcal{L}_{i}\).

In this work, we propose an approximated multi-task learning framework to unify both structural and functional regularization, in which our proposed activity regularization is a special case.

We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

We further improve DAR with a stochastic sampling regularizer (SAR), which balances the model's stability and plasticity, allowing the model to acquire new knowledge, alleviate forgetting and has greater scalability.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
1. Evaluation Summary:
The claim "We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning" is well-supported by the evidence. The text demonstrates this by:
- Showing how EWC and SI emerge from different Fisher matrix approximations
- Explaining how GEM arises from constraint-based interpretations
- Demonstrating how LwF emerges from using KL-divergence losses
- Illustrating how their own DAR/SAR approaches fit within this framework

The evidence thoroughly establishes how different loss variations in the multi-task learning objective lead to various existing continual learning approaches, making this claim well-stated and accurately reflecting the evidence.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning.

The review comment to be evaluated is:

This paper proposes an approach to mitigate catastrophic forgetting in supervised learning by regularizing activations. The paper views previous techniques (EWC, SI, and GEM) under a multi-task learning lens, and then proposes an additional loss term to minimise the KL between activations from previous and current models, on previous tasks - this is based on a memory which stores some previous samples and their corresponding activations.

I think it is a simple and intuitive approach and a well-written paper. Unfortunately I have a number of concerns that I think preclude publication in the current state.

First, in terms of related work, I believe this is very similar to Learning without forgetting (LwF), with the difference that the KL-divergence is computed on samples kept from the previous tasks. This is briefly mentioned in the paper, but I think it needs to be made more explicit, and LwF should be a baseline in the experiments to clearly indicate the benefit of keeping this data. There is also a relationship to EWC: given the connection between the Fisher information and KL, it can be viewed as minimising the KL divergence in parameter space, rather than in activation space (which is the case here). Also note that EWC uses the true Fisher rather than the empirical, contrary to the derivation in equation (2).
There are also a number of papers that haven’t been cited in the related work [1][2][3][4].

Second, I think the motivation in Section 3.1 could be more convincing. Most importantly, it’s not clear to me that the decision boundary *shouldn’t* change for previously misclassified examples, as this could be an opportunity for backwards transfer.
Further, I don’t think the point in the last paragraph about having a small data portion is relevant, since they are from the same data distribution, and we would expect misclassified samples to be in the same (low) frequency in Fisher estimation as overall. I think the point of this paragraph is just that it is important to consider the entire predictive distribution of previous tasks rather than the probability of the correct class, so this should be stated more clearly and then justified.

Finally, I think the experimental justification could be improved as well. Beyond permuted MNIST (which it has been argued is not as useful as other baselines [4]), only the final performance on split notMNIST / CIFAR-100 is reported. Some comments and questions:
- The accuracies of EWC (and possibly SI) in the table are worse than reported in previous work (eg. [1]), so I think this needs to be examined.
- What is the fine-tuning baseline (I don't believe it is actually clearly defined)? How can it be so low in figure 2a but better in 2b?
- I think plots over time (performance on all tasks) would be much more useful than the final performance in Table 2 and Fig 2.
- Errors and error bars would be beneficial for all results.
- Table 1 should also include the references provided.

Some other comments and questions:
- Compared to eqn (2), eqn (6) is missing the ½ constant.
- Typos in section 5.3: "SI performs better than SI", and VAR instead of SAR.
- Section 2, unclear of meaning of "coined with the likelihood" (should this be “coincide”?)
- The first line should be “Humans have the ability to learn...” In general, I think the introduction could use another proofread for grammar and readability as I saw a few minor things.

[1] Nguyen, Cuong V., et al. "Variational Continual Learning." ICLR, 2018.
[2] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." ICML, 2018.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." NIPS, 2017.
[4] Farquhar, Sebastian, and Yarin Gal. "Towards Robust Evaluations of Continual Learning." arXiv, 2018.

The evidence to be evaluated is:To avoid forgetting previous knowledge, when learning task \(\mathcal{T}_{t}\), the learner \(\mathbf{\theta}\) minimize the multi-task learning objective as \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\lambda_{t-1}\mathcal{L}_{t-1}(\mathbf{\theta},\mathcal{D}_{t-1})+\ldots+ \lambda_{1}\mathcal{L}_{1}(\mathbf{\theta},\mathcal{D}_{1}), \tag{1}\] where the loss \(\mathcal{L}_{i}(\mathbf{\theta},\mathcal{D}_{i})\) is usually the negative log-likelihood \(-\sum_{\mathbf{x},y\in\mathcal{D}_{i}}\log p(y|\mathbf{x};\mathbf{\theta})\) of the corresponding training data and \(\lambda_{i}\) regulates the importance of task \(\mathcal{T}_{i}\) to all the tasks observed so far. In continual learning, at time \(t\), we do not have access to the data \(\mathcal{D}_{k<t}\) to evaluate the loss \(\mathcal{L}_{k<t}\), thus, we need to approximate the past losses \(\mathcal{L}_{k<t}(\mathbf{\theta},\mathcal{D}_{k})\) when minimizing \(\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_{t})\). A common choice is to employ Laplace propagation (Eskin et al., 2004) of the negative log posterior \[-\log p(\mathbf{\theta}|\mathcal{D}_{i})\approx-\log p(\mathbf{\theta}_{i}^{*}| \mathcal{D}_{i})+\frac{1}{2}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}( \mathbf{\theta}-\mathbf{\theta}_{i}^{*}), \tag{2}\] where \(\mathbf{\theta}_{i}^{*}\) is a mode of the log posterior and \(\mathbf{F}_{i}\) is the empirical Fisher information matrix evaluated at \(\mathbf{\theta}=\mathbf{\theta}_{i}^{*}\). If we assume an uniform prior on \(\mathbf{\theta}\), the posterior coined with the likelihood 1, thus by substituting eqn. 2 to 1, the objective for training task \(\mathcal{T}_{t}\) becomes

\[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta},\mathcal{D}_ {t})+\sum_{i<t}\lambda_{k}(\mathbf{\theta}-\mathbf{\theta}_{i}^{*})^{T}\mathbf{F}_{i}(\bm {\theta}-\mathbf{\theta}_{i}^{*}), \tag{3}\] where the terms \(-\log p(\mathbf{\theta}_{i}^{*}|\mathcal{D}_{i})\) are omitted because they are constant with respect to \(\mathbf{\theta}\).

Eqn. 3 results in a wide range of methods for continual learning, each of which proposes a different method to approximate the Fisher information matrix. For example, Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017) assumes the Fisher is diagonal and approximates it using the identity: \[\mathbf{F}_{t}\approx\text{diag}\left(\sum_{n=1}^{N}\left.\left(\nabla_{\mathbf{ \theta}}\log p(\mathbf{y}_{t}^{(n)}|\mathbf{x}_{t}^{(n)};\mathbf{\theta})\right)^{2} \right|_{\mathbf{\theta}=\mathbf{\theta}_{T}}\right) \tag{4}\] Instead of using the Fisher's identity, Synaptic Intelligence (SI) (Zenke et al., 2017) directly measures the sensitivity of a parameter \(\mathbf{\theta}_{t}^{k}\) to the loss function at task \(\mathcal{T}_{t}\) through out the whole training trajectory. Zenke et al. (2017) also shows that this estimation is equivalent to measuring the full Fisher matrix on whole data set under certain choices of the loss function. Another approach to achieve the approximated multi-task learning objective in eqn. 1 is based on the observation that at the beginning of task \(\mathcal{T}_{t}\), we initialize \(\mathbf{\theta}=\mathbf{\theta}_{t-1}^{*}\), whose value already minimized the lost \(\tilde{\mathcal{L}}_{t-1}\). From this, if we can consider eqn. 1 as minimizing the loss \(\mathcal{L}_{t}(\mathbf{\theta})\) with the constraints that previous losses are not allowed to increase after every mini-batch update, which is the GEM objective. In particular, GEM optimize the following problem \[\begin{array}{ll}\text{Minimize}_{\mathbf{\theta},\mathcal{D}_{t}}&\mathcal{L}_ {t}(\mathbf{\theta})\\ \text{subject to}&\mathcal{L}_{k}(\mathbf{\theta},\mathcal{M}_{k})\leq\mathcal{L}_ {k}(\mathbf{\theta}_{t-1}^{*},\mathcal{M}_{k}),\forall k<t,\end{array} \tag{5}\]where the loss \(\mathcal{L}_{k}\) is evaluated at the memory \(\mathcal{M}_{k}\) that stores some of the training samples from task \(\mathcal{T}_{k}\). GEM further converts the constraints in eqn. 5 into the gradient constraints of the losses and solve them using quadratic programming. Finally, the type of losses \(\mathcal{L}_{k<t}\) can be flexibly changed to other loss rather than the negative log-likelihood. For example, Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes a form of distillation loss by minimizing the KL-divergence of past models and current model on the newly observed tasks. By changing the losses \(\mathcal{L}_{k<t}\), we can recover different solutions for continual learning in the literature.

To tackle the aforementioned problem, we propose to minimize the KL-divergence between the current model and the previous optimal models on the corresponding tasks. This can also be interpreted as distilling the knowledge from \(\mathbf{\theta}_{i}^{*}\) to \(\mathbf{\theta}\) on the corresponding data \(\mathcal{D}_{i}\). In particular, for each task \(\mathcal{T}_{i}\), we store some of the samples \(\mathbf{x}_{i}\) and the prediction of the optimal model \(f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x}_{i})\) in a memory \(\mathcal{M}_{i}\); then the Activity Regularization (AR) objective for task \(\mathcal{T}_{t}\) is \[\tilde{\mathcal{L}}_{t}(\mathbf{\theta})=\mathcal{L}_{t}(\mathbf{\theta})+\sum_{i<t} \sum_{\mathbf{x}\in\mathcal{M}_{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{ \theta}_{i}^{*}}(\mathbf{x})\bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}, \tag{7}\] where \(\tau\) is the temperature used in softmax outputs. Our goal is to ensure that the new model \(\mathbf{\theta}\) behaves similarly with \(\mathbf{\theta}_{i}^{*}\) on the previous task \(\mathcal{T}_{t}\), thus we call this constraint as activity regularization.

We refer to this algorithm as "Deterministic Activity Regularization" (DAR) which imposes regularization constraints for all the previous tasks at every learning iteration.

Similar to approximated multi-task learning methods, DAR employs a regularization constraint on every of the past tasks when learning a new one, which requires calculating the KL-divergence resulting in an additional forward pass on the memory, and thus results in additional computational cost when training the models.

7:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{D }}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{i<t}\sum_{\mathbf{x}\in\mathcal{M} _{i}}\lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\)

To address the weaknesses of DAR, we propose the Stochastic Activity Regularization (SAR), which randomly samples a past task and apply activity regularization on each gradient update when learning a new task.

Then, when learning a new task, for each mini batch to update \(\mathbf{\theta}\), we apply AR on all of the previous memory. For SAR, we instead randomly sample an index i of the previous tasks and apply the AR only on that memory. As a result, SAR is much more robust as it improves the model's plasticity, alleviates catastrophic forgetting and has greater scalability. For DAR, we omit the sampling index i in step 7 and calculate the KL divergence on all the memory as in eqn. 7. The memory \(\mathcal{M}_{i}\) is used to store some training samples of the previous tasks which is used to regularize the current model when learning new tasks.

7: Sample an index i from \([0,\dots,t-1]\) 8:\(g_{\mathbf{\theta}}\leftarrow\nabla_{\mathbf{\theta}}\left[\sum_{\mathbf{x},y\in\mathcal{ D}}\mathcal{L}_{t}(\mathbf{\theta},(\mathbf{x},y))+\sum_{\mathbf{x}\in\mathcal{M}_{i}} \lambda_{i}KL\bigg{(}\frac{1}{\tau}f_{\mathbf{\theta}_{i}^{*}}(\mathbf{x})\ \bigg{\|}\ \frac{1}{\tau}f_{\mathbf{\theta}}(\mathbf{x})\bigg{)}\right]\) 9:\(\mathbf{\theta}\leftarrow\mathbf{\theta}-\alpha\cdot\text{SGD}(\mathbf{\theta},g_{\mathbf{ \theta}})\) 10:return\(\mathbf{\theta}\) ```**Algorithm 2** Stochastic Activity Regularization (SAR) algorithm for Continual Learning.

Learning without Forgetting (LwF) (Li & Hoiem, 2017) proposes to minimize the difference in KL-divergence of the current and previous models on the new tasks, which is a form of knowledge distillation penalty. Similarly, Less-forgetting learning (LF) (Jung et al., 2019) proposes to minimize the \(\ell_{2}\) norm of the two prediction in stead of KL-divergence as LwF. Motivated from Hebbian learning (Hebbs, 1949), MAS (Aljundi et al., 2018a) proposes to penalize the \(\ell_{2}\) norm of the model's prediction. Selfless sequential learning (SNI) (Aljundi et al., 2018b) improves MAS by combining with it sparsity in each layer's activation.

While there are some overlapping between Bayesian learning and our approximated multi-task learning framework, the main difference is that our objective allows us to actively control the contribution of previous tasks by setting the importance value \(\lambda_{i}\) for each loss \(\mathcal{L}_{i}\).

In this work, we propose an approximated multi-task learning framework to unify both structural and functional regularization, in which our proposed activity regularization is a special case.

We propose an approximated multi-task learning framework that can alleviate catastrophic forgetting, an important challenge in continual learning and show that several previous solutions in the literature are special cases of this framework.

To overcome this problem, we propose the deterministic activity regularization for continual learning, DAR, that enforces the new model's behaviour to be consistent with the optimal models on each of the previous tasks.

We further improve DAR with a stochastic sampling regularizer (SAR), which balances the model's stability and plasticity, allowing the model to acquire new knowledge, alleviate forgetting and has greater scalability.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "We show that by varying the losses in the approximated multi-task learning objective, we can recover a family of regularization based approaches for continual learning."

Summary of evaluation:
The evidence strongly supports this claim. The paper demonstrates how different loss functions in their framework correspond to various existing continual learning methods:
1. Using Fisher information approximation recovers EWC
2. Using sensitivity measurements recovers SI
3. Using gradient constraints recovers GEM
4. Using KL-divergence recovers LwF
The paper mathematically shows how these methods emerge from their framework through equations 1-5, and the review comment acknowledges these connections while raising other concerns unrelated to this specific claim.

<score>1</score>