Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To approach that, we add only two subnetworks in the form of Multilayer Perceptron, one for computing the feature compensation and the other for soft gating the compensation, where both are computed based on the context.

The evidence to be evaluated is:In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network.

Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region.

Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that it is moderately well-stated with some minor overstatement. The evidence clearly shows two MLP subnetworks in Figure 1 - one for computing feature compensation (marked as "MLP" producing Δ) and another for soft gating (marked as "MLP" producing W). The evidence thoroughly explains how these networks operate based on context (similarity scores S). However, the claim's use of "only" implies extreme simplicity, while the full system includes additional components like clustering and auto-encoding that are essential to generating the context. The implementation is more complex than suggested.

<score>2</score>

Justification:
1. The evidence supports the basic architectural claim about two MLP subnetworks and their functions
2. The system diagram shows these networks do operate on context-based information (S values)
3. However, the word "only" understates the full complexity, as the system requires:
- An auto-encoder (DEC)
- A clustering mechanism
- Cosine similarity computation
- Additional operations (concatenation, scalar product)
4. The paper's own figures and explanations demonstrate this is more than "only two subnetworks"


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To approach that, we add only two subnetworks in the form of Multilayer Perceptron, one for computing the feature compensation and the other for soft gating the compensation, where both are computed based on the context.

The review comment to be evaluated is:

Strengths:
- The problem addressed by the paper is relevant.
- The methodology is clear and well-described.
- The experiments are fair and show that the method is competitive with SOTA in some cases.

Weaknesses:
1. In the introduction, the authors remark that "due to the overlap between the source domain and the target domain, not every data example needs transferring, and how much transferring is needed for a data example is subject to its position in the overall data distribution". Nonetheless, in their algorithm, clustering is performed using the target data. Thus, the computed similarities will not tell how far a given target example is from the source distribution. Therefore, I don't see how they are useful to assess "how much transferring is needed" for that example. When the source distribution is very different from the target, the method seems inappropriate. The poor results on MIT Indoor-67 reinforce this suspicion.
2. Are the displacements $\Delta$ normalized? Otherwise, the role of the predicted magnitude $W$ is redundant.
3. The experimental results are not particularly convincing. In particular, it is unclear which of the training strategies proposed in section 3.3 is the best one. In addition, it would be interesting to visualize what kind of transformations the model is learning, i.e. if it's learning $\Delta$s that pull representations towards the cluster centroids (or something else).
4. A theoretical analysis of the proposed methodology is missing. It is presented as a mere heuristic without any theoretical guarantees.

The evidence to be evaluated is:In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network.

Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region.

Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and review comments, I'll evaluate the claim about adding "only two subnetworks in the form of Multilayer Perceptron."

The evidence from the figures and text shows that the architecture actually involves several components beyond just two MLPs:
1. An Auto-Encoder (DEC)
2. A clustering mechanism
3. Cosine similarity computation
4. Two MLPs (one for compensation and one for soft gating)
5. Various concatenation and mathematical operations

While the claim focuses on the two MLPs, it understates the complexity of the overall architecture and additional components required for the system to function. The reviewer's comments also highlight concerns about the theoretical foundations and methodology.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To approach that, we add only two subnetworks in the form of Multilayer Perceptron, one for computing the feature compensation and the other for soft gating the compensation, where both are computed based on the context.

The review comment to be evaluated is:

Strengths:
- The paper tackles an important problem of identifying the mismatch between a source domain and a target domain before transfer.
- The paper is, in general, well-aligned with literature in the area.

Weaknesses:
- The paper does not provide explanations / intuitions to most technical decisions. For instance, the proposed method is very quickly explained in roughly half a page (sections 3.1 - 3.2). For example, why is it reasonable to use DEC? Why is compensation necessary - what are the underlying factors that make it necessary once the closest cluster centers have been identified? Why is cosine similarity, and no other measure, applicable in this case?
- It is very unclear to me how the proposed method is actually helping to identify the mismatch between a source domain and a target domain. In several parts of the paper, it is mentioned that the cluster centers are calculated on the full training set, and according to my understanding it is the whole training set of the target domain for which cluster centers are being calculated? I think it would greatly benefit the explanation of the method if it would clearly distinguish which parts of it come from the source domain and which from the target domain.
- In the experiments, I do not see any reasoning as to what the proposed method works in some cases but not in others. In which cases doing this kind of clustering + compensation approach would work better? This kind of description is very important to really see the contribution of the proposed method to the problem of transfer learning.

The evidence to be evaluated is:In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network.

Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region.

Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim moderately overstated. While the evidence shows they do add two MLP subnetworks (one for compensation and one for soft gating), the claim oversimplifies the actual complexity of the approach. The evidence reveals additional components beyond just "two subnetworks" - including clustering (DEC), cosine similarity calculations, and context vector generation. The review comments also highlight that technical decisions like using DEC and cosine similarity lack proper justification. The method appears more complex than suggested in the claim.

<score>4</score>

Justification:
1. The claim states they "just add two subnetworks" but the evidence shows:
- A DEC-based clustering component
- Cosine similarity calculations
- Context vector generation (S)
- Feature concatenation operations
- Both MLPs with specific constraints

2. The reviewer points out valid concerns about:
- Lack of explanation for technical choices (DEC, cosine similarity)
- Unclear distinction between source and target domain processing
- Limited reasoning for when/why the method works

3. Figure 1 clearly shows more components than just "two subnetworks", indicating the claim understates the system's complexity.

The claim would be more accurate if it acknowledged the full scope of components and computational steps involved, rather than minimizing it to "just" two subnetworks.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To approach that, we add only two subnetworks in the form of Multilayer Perceptron, one for computing the feature compensation and the other for soft gating the compensation, where both are computed based on the context.

The review comment to be evaluated is:

### Strength

- The method is interesting. Studying the contextual relation between source and target domains is novel.
- The related work section is sufficient. So much literature is surveyed.
- Experimental datasets and comparison methods are representative.

### Weakness

- Although the idea of learning contextual relation is novel, this approach is not. It remains unclear whether clustering can model the context information, or if the results of clusters and original inputs can solve the mismatch between source and target domains in transfer learning.
- The claim that "this method only adds two MLPs to the backbone network" is not correct. As can be seen from Figure 1, there are more than two MLPs: it still has an autoencoder (DEC), which is much more complicated than MLPs.
- The method part is extremely confusing and not clear, which should be rewritten. I find figure 1 hard to understand, specifically how to compute $\mathbf{X} \textcopyright \mathbf{S}$ and $\mathbf{W} \cdot \Delta$. (I even think that should be $\cdot$ instead of $.$, since you claim it's a product.)
- The success of "compensation" heavily relies on DEC (deep embedding clustering). If DEC does not give the correct clustering results, the method will fail. How to justify this? Additionally, DEC brings extra computations, which should be highlighted in the paper.
- This method will consume a lot of training time than comparison methods, according to Sec. 3.3. We see that there is more than one step of fine-tuning, which is more complicated than existing work. Authors should compare your training and inference time in the experiments.
- In experiments, you should also list the number of training parameters of each method for a more fair study since this method introduces extra neural network modules.
- More importantly, the performance on general fine-tuning tasks of this method is NOT good. According to table 1, This approach only succeeds using Resnet-101, while it does not generate competitive results in Resnet-50 and VGG16. Hence, it is difficult to say that this approach actually works in different backbones.
- No experiments to explain why this method works. Lack of detailed analysis.
- With regards to related work, I think this paper misses two very important references on the same topic:

[1] Jang Y, Lee H, Hwang S J, et al. Learning what and where to transfer[C]//International Conference on Machine Learning. PMLR, 2019: 3030-3039.

[2] Murugesan K, Sadashivaiah V, Luss R, et al. Auto-Transfer: Learning to Route Transferrable Representations[J]. arXiv preprint arXiv:2202.01011, 2022.

The evidence to be evaluated is:In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network.

Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region.

Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim, evidence, and reviewer comments, I find the claim is significantly overstated. Here's why:

1. The claim states "we add only two subnetworks in the form of Multilayer Perceptron," but Figure 1 clearly shows additional components beyond just two MLPs, including an auto-encoder (DEC) which is more complex than MLPs. This is directly contradicted by the reviewer's comments and the architectural diagram.

2. The evidence shows the system requires:
- An Auto-Encoder (DEC)
- Clustering mechanism
- Multiple MLPs (for compensation and gating)
- Cosine similarity computation
- Additional computational components

3. The reviewer correctly points out that the DEC component is crucial for the method's success and adds significant computational overhead, which is not acknowledged in the original claim.

The claim substantially understates the architectural complexity and computational requirements of the approach, making it misleading.

<score>5</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To approach that, we add only two subnetworks in the form of Multilayer Perceptron, one for computing the feature compensation and the other for soft gating the compensation, where both are computed based on the context.

The review comment to be evaluated is:

Strengths:
the paper is well written, methodology is well-described, and the problem addressed is relevant.

Weaknesses.
The author is describing how to use data centers to do data compensation, however, how this is related to data transferring is poorly described. For examples, is data centers computed on target data set? how is its feature vector computed, using the pretrained network? using distance to get compensation factor seems only to be working under a strong assumption that this is used for classification, and it only works for image classification tasks? why using distance to centers work? all these questions are not answered properly.

Based on the experimental setup, it is just finetuning the pretrained network using some downstream tasks. It just barely competes with SOTA methods, seems that with or without such 2 layers of compensation networks does not change the result at all.

The evidence to be evaluated is:In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network.

Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region.

Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim, evidence, and review comments, I will evaluate its accuracy:

The claim that "we add only two subnetworks in the form of Multilayer Perceptron, one for computing the feature compensation and the other for soft gating the compensation, where both are computed based on the context" appears to be well-described by the evidence. Figure 1 clearly shows these two MLP components and their roles. The methodology is thoroughly explained in the text, detailing how each MLP functions - one for feature compensation and one for soft gating based on context vectors derived from cluster similarities. However, the reviewer raises valid concerns about the practical effectiveness and limitations of this approach, noting that experimental results show minimal improvements over baseline methods.

<score>2</score>

While the technical description of the architecture is accurate, the claim would be better stated by acknowledging the limitations raised in the review regarding its practical effectiveness and domain constraints. The "only" in the claim suggests simplicity but understates the complexity of the full system shown in Figure 1, which includes additional components like clustering and similarity computation.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address the aforementioned challenges, we propose GAPS (Guided copy-And-Paste Synthesis), which improves the training of incremental segmentation models by synthesizing fully-annotated images from partially-annotated examples.

The evidence to be evaluated is:Noticeably, a few attempts have been made by recent work to address the semantic background shift problem proposed by Cermelli et al. (2020) via either pseudo-labeling (Cha et al., 2021; Douillard et al., 2021; Yan et al., 2021) or modified loss (Cermelli et al., 2020; Zhang et al., 2022) to train on partially-annotated images of novel classes.

In contrast, our work enables incremental segmentation learning with _few data_ via a guided copy-paste process, which demonstrates promising performance under the few-shot and more impoverished single-instance setting. Furthermore, GAPS is a _model-agnostic data pre-processor_, which is orthogonal to incremental learning techniques such as regularization (Li and Hoiem, 2017).

While there are many works in few-shot incremental image classification (Tao et al., 2020; Cheraghian et al., 2021), relatively fewer works have been done to investigate few-shot incremental segmentation (Tian et al., 2020; Cermelli et al., 2021; Ganea et al., 2021).

However, PIFS (Cermelli et al., 2021) is fine-tuned on only a small number of samples, which leads to sub-optimal performance due to overfitting. In addition, PIFS requires fully-annotated images as input, which hinders its potential for practical applications.

Copy-and-paste is an augmentation technique that copies a subset of objects from one image and pastes onto the other image using their segmentation masks. Many works (Dwibedi et al., 2017; Ghiasi et al., 2021; Dvornik et al., 2018) have been done to investigate how copy-and-paste augmentation can help with various visual tasks.

We extend the augmentation strategy from Ghiasi et al. (2021) and construct an intuitive baseline called Naive copy-Paste Synthesis (NPS) to adapt it to our online task setting. However, as we will demonstrate, such naive adaptation gives unsatisfactory performance in our task setting because of _gaps_ between the static offline learning and continual online learning. In our work, we propose a series of techniques to guide the copy-paste synthesizer to address these gaps, whose effectiveness is evident from the significant improvement from NPS.

Different from standard semantic segmentation, in few-shot incremental segmentation, \(\mathcal{C}\) is expanded over time through two stages. During the _base learning stage_, the model is provided with a base dataset \(\mathcal{D}_{0}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{ \mathcal{C}_{0}}\}\), where \(\mathcal{C}_{0}\) is a set of classes in the base dataset.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\). In few-shot learning, the size of the training sets for the novel tasks is small, _i.e_., \(|D_{j}|\ll|\mathcal{D}_{0}|\).

The goal of incremental learning is to optimize the model performance jointly on both previous tasks and the current task. To enforce the partially-annotated image setting, we follow Cermelli _et al_. and assume that only novel classes are annotated, _i.e_., \(\mathcal{C}_{i}\cap C_{j}=\varnothing\) for all \(i\neq j\).

Fig. 2 illustrates our proposed Guided copy-Paste Synthesis (GAPS) framework for few-shot incremental segmentation. It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

During the incremental learning stage, fully-labeled samples are synthesized by copying from the masked novel objects in \(D_{1},\dots,D_{j}\) and pasting onto base exemplars from the replay buffer \(\hat{D}_{0}\).

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance.

For methods with memory-replaying (e.g., SSUL (Cha et al., 2021)), GAPS can work directly on top of their constructed buffers with minimal modification. For other methods such as PIFS (Cermelli et al., 2021), we propose a diversity-guided exemplar selection process that builds a small yet diverse memory-replay buffer \(\hat{D_{0}}\) from \(\mathcal{D}_{0}\) to mitigate catastrophic forgetting. Selecting diverse examples that are representative of the base dataset helps mitigate catastrophic forgetting, as suggested by Rebuffi et al. (2017). Inspired by Bang et al. (2021), we select samples distributed uniformly along a spectrum from easy to hard for diversity. Here, we present an algorithm (Algorithm 1) to construct \(\hat{D_{0}}\) by exploiting the Masked Average Pooling (Map) function from Cermelli et al. (2021). Intuitively, we approximate the difficulty of every sample by their similarity between the estimated prototype with learned prototypes. Estimated prototypes that are close to the learned prototype are considered easy samples and vice versa. After building a list of base samples sorted by difficulties, we select samples from equally-spaced intervals to ensure samples of all difficulties are selected for diversity.

We hypothesize that synthesizing novel objects onto contextually consistent base images would result in an improved learning process.

We design a context-guided sampling algorithm to select images from \(\hat{D_{0}}\) that are contextually similar to the provided partially-labeled images. GAPS uses a scene embedding network \(h:\mathcal{X}\rightarrow\mathbb{R}^{m}\) to estimate pairwise contextual similarity between any two given images. The embedding \(h\) maps an input image to a scene embedding vector in a metric space where pairwise similarity comparison between two embeddings are possible. In our system, we use an off-the-shelf VGGNet (Simonyan and Zisserman, 2015) and replace its last fully connected layer with a cosine-similarity-based classifier. The network \(h\) is trained using the Places365 (Zhou et al., 2017) dataset, which contains \(365\) scene categories and roughly 1.6 million images in its training split. During training, the scene prediction is done by comparing the predicted embedding with existing prototypes.

To find contextually similar base images to each novel image, we evaluate the cosine similarity of the novel image to each of the examples in \(\hat{D}_{0}\), and constuct a contextually similar subset \(\mathbb{S}\) with \(|\hat{D}_{0}|/10\) most contextually similar examples. When there are multiple novel images, we take a union of selected examples. To allow other base scenes to be sampled to mitigate catastrophic forgetting, we sample from \(\mathbb{S}\) with a probability of \(\alpha\), and sample from \(\hat{D}_{0}\) with a probability of \(1-\alpha\), where \(\alpha\) is a hyperparameter set to \(0.9\) in our implementation.

Now the final question is, given a fully-annotated image \(x_{B}\) and an image of a novel object \(x_{N}\), how frequent should we apply copy-paste? There is a trade-off between oversampling and undersampling. As one extreme, one can follow Ghiasi et al. (2021) and always apply copy-paste augmentation to paste novel objects onto every base image. However, this will lead to oversampling of novel categories in the current task, which we found to hurt the performance of existing classes.

Therefore, to guide copy-paste in the online setting, we design a synthesis strategy called vRFS based on RFS (Repeat Factor Sampling) described by Gupta et al. (2019) to perform synthesis. To apply vRFS, we first need to compute category-wise sampling factor \(r_{c}\) for every \(c\) as in RFS. If \(c\in\mathcal{C}_{0}\), we set \(r_{c}=1\) as since during the construction of \(\hat{D}_{0}\) we already consider class balance by class-wise uniformly sampling. If \(c\in\mathcal{C}_{j}\) with \(j\geq 1\), we first compute its class frequency by \(f_{c}=nShot/|\hat{D}_{0}|\), where \(nShot\) denotes the number of images in \(D_{j}\) with at least one pixel of \(c\). Then, the category-wise sampling factor for \(c\) is given by \(r_{c}=\textsc{Max}(1,\sqrt{t/f_{c}})\). Note that in (Gupta et al., 2019), \(t\) is chosen as a hyperparameter to be tuned. However, we empirically found that setting \(t\) to be the multiplicative inverse of total number of classes, or \(t=1/|\cup_{1,...,j}Y_{j}|\), is enough to yield stable results across different datasets and under different few-shot settings. This eliminates the need to search a hyperparameter for different settings and make our proposed method more robust towards different task settings. During the synthesis process, we first randomly select a novel class \(c_{N}\) from \(\mathcal{C}_{j}\), and another class \(c_{o}\) from \(\cup_{1,...,j}C_{j}\setminus\{c_{N}\}\). We first decide if \(c_{o}\) should be pasted onto \(x_{B}\). To apply vRFS resampling, we hallucinate two _virtual_ samples: in the first sample where copy-paste would not be applied, the image-level sampling factor is given by 1. In the second sample where copy-paste synthesis were to be performed, we would obtain a sample with image-level sampling factor of \(r_{i}=\textsc{Max}_{c\in i}r_{c}=r_{c_{o}}\). Thus, the probability to synthesize class \(c_{o}\) onto \(x_{B}\) is given by \(r_{c_{o}}/(1+r_{c_{o}})\). We then repeat the process for the novel class \(c_{N}\). Note that vRFS synthesis is applied twice for every class, resulting in up to two pasted instances of \(c_{N}\) in the final image.

**GAPS consistently increases performance under few-shot settings.** Methods combined with our proposed data source, GAPS, consistently outperform their un-augmented counterpart on both the base and novel categories performance. It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

**Context-guided sampling steadily improves performance on novel classes**, which is consistent with findings in previous work (Dvornik et al., 2018) that background context is an important factor to consider in copy-paste synthesis. **Frequency-guided probabilistic synthesis boosts results on novel classes**.

Though the semantic background shift proposed by Cermelli et al. (2020) relaxes the requirement to provide full segmentation annotations, it still requires _all novel instances_ in images to be annotated, which can be time-consuming to obtain in cluttered scenes and hinder potential applications. Here we consider a more challenging task setting, which we term _single-instance incremental learning_. Namely, for training images provided in incremental learning stages, if there are multiple instances of a novel class in the image, we assume that only _one instance_ will be annotated. To simulate this setting, we use the instance-level segmentation annotation provided by the COCO dataset to enforce only annotation of one novel instance in every image is available to the model.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images. Our proposed GAPS technique selects representative exemplars in the memory-replay buffer and addresses the problems of class imbalance and contextual mismatch in synthesis. In future work, we are interested in further application of copy-paste as a synthesis technique to cope with the background shifting problem for incremental segmentation. We believe that copy-paste can serve as a promising alternative to pseudo-labeling and modified loss to enable learning on partially-annotated images.

In our implementation, the probability to synthesize novel objects is simply given by \(1/M\), where \(M\) is the sum of total number of base classes and seen novel classes. For every selected base image, the _Aways Paste_ method randomly picks a novel category and always synthesizes an instance of the selected novel category onto the given image. Finally, the _50-50-Paste_ method is an extension of the naive copy-paste method to leverage past novel samples like vRFS. Essentially, it replays samples from learned novel classes with 50% probability, and uses novel class from the current incremental learning task also with 50% probability.

We perform an additional experiment on COCO-20-1 where a long few-shot incremental learning sequence (60 base; 20 novel classes with 20 incremental steps; 5-shot) is used.

PIFS(\(\mathcal{L}_{2}\)), augmented with GAPS, demonstrates more robustness to forgetting with many incremental learning steps than standard PIFS. To validate our approach on the ADE20k dataset (Zhou et al., 2017, 2019), we ran a suite of experiments where we follow the ADE100-5 (11 steps) setup discussed in (Cha et al., 2021) and modify it as 5-shot learning. We tested PIFS(\(\mathcal{L}_{2}\)) and PIFS(\(\mathcal{L}_{2}\))+GAPS on this setting. The results are given in Table 9. We observe a steady increase in both the base IoU and novel IoU in this challenging task setting on methods augmented with GAPS. This further validates the effectiveness of our approach.

Our PIFS(\(\mathcal{L}_{2}\))+GAPS significantly outperforms both baselines.

As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method.

Fig. 5 shows qualitative results of PIFS(\(\mathcal{L}_{2}\)), PIFS(\(\mathcal{L}_{2}\))+NPS (naive copy-paste), and PIFS(\(\mathcal{L}_{2}\))+GAPS on the PASCAL-5-1 split dataset under the 1-shot setting using images on the left column. We can see that PIFS and PIFS+NPS gives reasonably good predictions on classes that are adapted recently (e.g., the cow class in the last row). However, they tend to exhibit catastrophic forgetting for classes learned earlier (e.g., the bus example in PIFS). On the other hand, the model trained using GAPS not only yields good result on recently learned classes, but also maintains promising performance on classes adapted earlier.

We can see that given an image of traffic lights on a railway, our scene embedding model is able to select contextually-similar scenes such as railways and street scenes for subsequent synthesis. Fig. 7 and Fig. 8 demonstrate how PIFS(\(\mathcal{L}_{2}\))+GAPS compare to the baseline PIFS(\(\mathcal{L}_{2}\)) across multiple step.

On the other hand, PIFS(\(\mathcal{L}_{2}\))+GAPS is able to maintain good performance on learned classes as it adapts to new classes.

Input: Table 9: Performance of PIFS(L2) and PIFS(L2)+GAPS on the ADE100-5 disjoint 5-shot setting. Mean and standard deviation over 5 runs are reported.

Input: Figure 4: Performance of PIFS(L2)+GAPS on the COCO-20-1 split under 5-shot setting with varying memory-replay buffer size. Best view in color.

Input: Figure 1: Our proposed method utilizes guided copy-paste augmentation to synthesize diverse training data, using as few as one single novel instance for training. For example, the model encounters an image of many motorcycles, which is novel to the model. As a result, the model incorrectly assigns learned bicycle labels to these pixels and therefore needs to be updated. Our proposed method can adapt to the novel motorcycle class with an annotation of a single motorcycle, which can be efficiently annotated; whereas previous work (Cermelli et al., 2020; 2021) require time-consuming annotation of all instances of motorcycles or even the entire image. Best view in color.

Input: Figure 3: Long-term continual learning performance of PIFS(L2) and PIFS(L2)+GAPS on the COCO-20-1 split under 5-shot setting. Best view in color.

Input: Figure 7: Multi-step qualitative results of our method (GAPS) on the PASCAL-5-1 split under 1- shot setting. (Best view in color.)

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input: Figure 5: Qualitative results on the PASCAL-5-1 split under 1-shot setting. Sequential adaptation to five classes are performed in the order indicated by the blue arrow on the left. Visualization is done after all adaptation steps are completed. Observe, particularly in rows 1, 3, and 4, that our method mitigates catastrophic forgetting and improves performance on both base and novel classes compared to PIFS and PIFS+NPS. (Best viewed in color)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the extensive evidence provided, including detailed methodology descriptions, experimental results, and qualitative examples, I find the claim about GAPS to be well-stated and accurately reflects the evidence without exaggeration for these reasons:

1. The paper clearly demonstrates how GAPS improves incremental segmentation through synthesis of fully-annotated images from partially-annotated examples, supported by quantitative results across multiple datasets and settings.

2. The claim is appropriately scoped - it states what GAPS does without overstating its capabilities or making unsupported generalizations.

3. The evidence thoroughly validates the claim through ablation studies, comparative experiments, and qualitative results showing GAPS' effectiveness.

The claim directly aligns with the demonstrated capabilities and results, making it well-stated rather than overstated.

<score>1</score>

Justification:

1. Evidence shows consistent performance improvements when using GAPS, as demonstrated in Tables and Figures (e.g., Table 9, Figures 3-5)

2. The paper thoroughly explains and validates each component of GAPS with empirical results

3. The limitations and scope are appropriately acknowledged (e.g., "In future work, we are interested in further application...")

4. Implementation details and experimental settings are clearly specified

5. Multiple datasets and experimental settings validate the claims (PASCAL, COCO, ADE20k)

The evidence strongly supports the claim without any signs of overstatement or exaggeration.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address the aforementioned challenges, we propose GAPS (Guided copy-And-Paste Synthesis), which improves the training of incremental segmentation models by synthesizing fully-annotated images from partially-annotated examples.

The review comment to be evaluated is:

Strengths:
1. The paper investigates a novel and understudied scenario that is more realistic than earlier benchmarks.
2. The study gives a comprehensive summary of similar works as well as comparisons and contrasts between them.
3. The study expands the present benchmark with new experiments involving only one labeled instance of the new classes, demonstrating that the GAPS can address the issue of missing annotations.

Weaknesses:
1. The ablation study may be extended to other settings and may provide more insights about the paper choices. In particular, it may start from a baseline that uses exemplars without pasting the new classes on them, to show that pasting is important to improve performance and that the gain is not only given by the regularization effect of using exemplars (as proposed by GDumb [a]). Moreover, the paper should clarify if, while disabling the diversity guidance, the exemplars are sampled uniformly for the old classes or if they are randomly sampled on the full dataset.
2. From the ablation study, context sampling appears to marginally improve the results. Moreover, it is not clear how the performance will change by using a pre-training dataset different from Places-365, such as ImageNet or ADE20K. Furthermore, can it be trained directly on the base dataset of the considered setting or can the method use the segmentation network itself (for example, using an auxiliary classifier on the encoder features)?
3. The paper baselines appear to be very naive. In particular, the paper reports the results for Fine-Tuning, MiB, and PIFS without considering exemplars, introducing an unfair comparison with respect to GAPS. The baselines should be reported both without exemplars (as in the original paper) and with them (indeed, without the proposed copy-and-paste strategy).

[a] Prabhu, Ameya, Philip HS Torr, and Puneet K. Dokania. "Gdumb: A simple approach that questions our progress in continual learning." European conference on computer vision. Springer, Cham, 2020.

The evidence to be evaluated is:Noticeably, a few attempts have been made by recent work to address the semantic background shift problem proposed by Cermelli et al. (2020) via either pseudo-labeling (Cha et al., 2021; Douillard et al., 2021; Yan et al., 2021) or modified loss (Cermelli et al., 2020; Zhang et al., 2022) to train on partially-annotated images of novel classes.

In contrast, our work enables incremental segmentation learning with _few data_ via a guided copy-paste process, which demonstrates promising performance under the few-shot and more impoverished single-instance setting. Furthermore, GAPS is a _model-agnostic data pre-processor_, which is orthogonal to incremental learning techniques such as regularization (Li and Hoiem, 2017).

While there are many works in few-shot incremental image classification (Tao et al., 2020; Cheraghian et al., 2021), relatively fewer works have been done to investigate few-shot incremental segmentation (Tian et al., 2020; Cermelli et al., 2021; Ganea et al., 2021).

However, PIFS (Cermelli et al., 2021) is fine-tuned on only a small number of samples, which leads to sub-optimal performance due to overfitting. In addition, PIFS requires fully-annotated images as input, which hinders its potential for practical applications.

Copy-and-paste is an augmentation technique that copies a subset of objects from one image and pastes onto the other image using their segmentation masks. Many works (Dwibedi et al., 2017; Ghiasi et al., 2021; Dvornik et al., 2018) have been done to investigate how copy-and-paste augmentation can help with various visual tasks.

We extend the augmentation strategy from Ghiasi et al. (2021) and construct an intuitive baseline called Naive copy-Paste Synthesis (NPS) to adapt it to our online task setting. However, as we will demonstrate, such naive adaptation gives unsatisfactory performance in our task setting because of _gaps_ between the static offline learning and continual online learning. In our work, we propose a series of techniques to guide the copy-paste synthesizer to address these gaps, whose effectiveness is evident from the significant improvement from NPS.

Different from standard semantic segmentation, in few-shot incremental segmentation, \(\mathcal{C}\) is expanded over time through two stages. During the _base learning stage_, the model is provided with a base dataset \(\mathcal{D}_{0}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{ \mathcal{C}_{0}}\}\), where \(\mathcal{C}_{0}\) is a set of classes in the base dataset.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\). In few-shot learning, the size of the training sets for the novel tasks is small, _i.e_., \(|D_{j}|\ll|\mathcal{D}_{0}|\).

The goal of incremental learning is to optimize the model performance jointly on both previous tasks and the current task. To enforce the partially-annotated image setting, we follow Cermelli _et al_. and assume that only novel classes are annotated, _i.e_., \(\mathcal{C}_{i}\cap C_{j}=\varnothing\) for all \(i\neq j\).

Fig. 2 illustrates our proposed Guided copy-Paste Synthesis (GAPS) framework for few-shot incremental segmentation. It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

During the incremental learning stage, fully-labeled samples are synthesized by copying from the masked novel objects in \(D_{1},\dots,D_{j}\) and pasting onto base exemplars from the replay buffer \(\hat{D}_{0}\).

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance.

For methods with memory-replaying (e.g., SSUL (Cha et al., 2021)), GAPS can work directly on top of their constructed buffers with minimal modification. For other methods such as PIFS (Cermelli et al., 2021), we propose a diversity-guided exemplar selection process that builds a small yet diverse memory-replay buffer \(\hat{D_{0}}\) from \(\mathcal{D}_{0}\) to mitigate catastrophic forgetting. Selecting diverse examples that are representative of the base dataset helps mitigate catastrophic forgetting, as suggested by Rebuffi et al. (2017). Inspired by Bang et al. (2021), we select samples distributed uniformly along a spectrum from easy to hard for diversity. Here, we present an algorithm (Algorithm 1) to construct \(\hat{D_{0}}\) by exploiting the Masked Average Pooling (Map) function from Cermelli et al. (2021). Intuitively, we approximate the difficulty of every sample by their similarity between the estimated prototype with learned prototypes. Estimated prototypes that are close to the learned prototype are considered easy samples and vice versa. After building a list of base samples sorted by difficulties, we select samples from equally-spaced intervals to ensure samples of all difficulties are selected for diversity.

We hypothesize that synthesizing novel objects onto contextually consistent base images would result in an improved learning process.

We design a context-guided sampling algorithm to select images from \(\hat{D_{0}}\) that are contextually similar to the provided partially-labeled images. GAPS uses a scene embedding network \(h:\mathcal{X}\rightarrow\mathbb{R}^{m}\) to estimate pairwise contextual similarity between any two given images. The embedding \(h\) maps an input image to a scene embedding vector in a metric space where pairwise similarity comparison between two embeddings are possible. In our system, we use an off-the-shelf VGGNet (Simonyan and Zisserman, 2015) and replace its last fully connected layer with a cosine-similarity-based classifier. The network \(h\) is trained using the Places365 (Zhou et al., 2017) dataset, which contains \(365\) scene categories and roughly 1.6 million images in its training split. During training, the scene prediction is done by comparing the predicted embedding with existing prototypes.

To find contextually similar base images to each novel image, we evaluate the cosine similarity of the novel image to each of the examples in \(\hat{D}_{0}\), and constuct a contextually similar subset \(\mathbb{S}\) with \(|\hat{D}_{0}|/10\) most contextually similar examples. When there are multiple novel images, we take a union of selected examples. To allow other base scenes to be sampled to mitigate catastrophic forgetting, we sample from \(\mathbb{S}\) with a probability of \(\alpha\), and sample from \(\hat{D}_{0}\) with a probability of \(1-\alpha\), where \(\alpha\) is a hyperparameter set to \(0.9\) in our implementation.

Now the final question is, given a fully-annotated image \(x_{B}\) and an image of a novel object \(x_{N}\), how frequent should we apply copy-paste? There is a trade-off between oversampling and undersampling. As one extreme, one can follow Ghiasi et al. (2021) and always apply copy-paste augmentation to paste novel objects onto every base image. However, this will lead to oversampling of novel categories in the current task, which we found to hurt the performance of existing classes.

Therefore, to guide copy-paste in the online setting, we design a synthesis strategy called vRFS based on RFS (Repeat Factor Sampling) described by Gupta et al. (2019) to perform synthesis. To apply vRFS, we first need to compute category-wise sampling factor \(r_{c}\) for every \(c\) as in RFS. If \(c\in\mathcal{C}_{0}\), we set \(r_{c}=1\) as since during the construction of \(\hat{D}_{0}\) we already consider class balance by class-wise uniformly sampling. If \(c\in\mathcal{C}_{j}\) with \(j\geq 1\), we first compute its class frequency by \(f_{c}=nShot/|\hat{D}_{0}|\), where \(nShot\) denotes the number of images in \(D_{j}\) with at least one pixel of \(c\). Then, the category-wise sampling factor for \(c\) is given by \(r_{c}=\textsc{Max}(1,\sqrt{t/f_{c}})\). Note that in (Gupta et al., 2019), \(t\) is chosen as a hyperparameter to be tuned. However, we empirically found that setting \(t\) to be the multiplicative inverse of total number of classes, or \(t=1/|\cup_{1,...,j}Y_{j}|\), is enough to yield stable results across different datasets and under different few-shot settings. This eliminates the need to search a hyperparameter for different settings and make our proposed method more robust towards different task settings. During the synthesis process, we first randomly select a novel class \(c_{N}\) from \(\mathcal{C}_{j}\), and another class \(c_{o}\) from \(\cup_{1,...,j}C_{j}\setminus\{c_{N}\}\). We first decide if \(c_{o}\) should be pasted onto \(x_{B}\). To apply vRFS resampling, we hallucinate two _virtual_ samples: in the first sample where copy-paste would not be applied, the image-level sampling factor is given by 1. In the second sample where copy-paste synthesis were to be performed, we would obtain a sample with image-level sampling factor of \(r_{i}=\textsc{Max}_{c\in i}r_{c}=r_{c_{o}}\). Thus, the probability to synthesize class \(c_{o}\) onto \(x_{B}\) is given by \(r_{c_{o}}/(1+r_{c_{o}})\). We then repeat the process for the novel class \(c_{N}\). Note that vRFS synthesis is applied twice for every class, resulting in up to two pasted instances of \(c_{N}\) in the final image.

**GAPS consistently increases performance under few-shot settings.** Methods combined with our proposed data source, GAPS, consistently outperform their un-augmented counterpart on both the base and novel categories performance. It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

**Context-guided sampling steadily improves performance on novel classes**, which is consistent with findings in previous work (Dvornik et al., 2018) that background context is an important factor to consider in copy-paste synthesis. **Frequency-guided probabilistic synthesis boosts results on novel classes**.

Though the semantic background shift proposed by Cermelli et al. (2020) relaxes the requirement to provide full segmentation annotations, it still requires _all novel instances_ in images to be annotated, which can be time-consuming to obtain in cluttered scenes and hinder potential applications. Here we consider a more challenging task setting, which we term _single-instance incremental learning_. Namely, for training images provided in incremental learning stages, if there are multiple instances of a novel class in the image, we assume that only _one instance_ will be annotated. To simulate this setting, we use the instance-level segmentation annotation provided by the COCO dataset to enforce only annotation of one novel instance in every image is available to the model.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images. Our proposed GAPS technique selects representative exemplars in the memory-replay buffer and addresses the problems of class imbalance and contextual mismatch in synthesis. In future work, we are interested in further application of copy-paste as a synthesis technique to cope with the background shifting problem for incremental segmentation. We believe that copy-paste can serve as a promising alternative to pseudo-labeling and modified loss to enable learning on partially-annotated images.

In our implementation, the probability to synthesize novel objects is simply given by \(1/M\), where \(M\) is the sum of total number of base classes and seen novel classes. For every selected base image, the _Aways Paste_ method randomly picks a novel category and always synthesizes an instance of the selected novel category onto the given image. Finally, the _50-50-Paste_ method is an extension of the naive copy-paste method to leverage past novel samples like vRFS. Essentially, it replays samples from learned novel classes with 50% probability, and uses novel class from the current incremental learning task also with 50% probability.

We perform an additional experiment on COCO-20-1 where a long few-shot incremental learning sequence (60 base; 20 novel classes with 20 incremental steps; 5-shot) is used.

PIFS(\(\mathcal{L}_{2}\)), augmented with GAPS, demonstrates more robustness to forgetting with many incremental learning steps than standard PIFS. To validate our approach on the ADE20k dataset (Zhou et al., 2017, 2019), we ran a suite of experiments where we follow the ADE100-5 (11 steps) setup discussed in (Cha et al., 2021) and modify it as 5-shot learning. We tested PIFS(\(\mathcal{L}_{2}\)) and PIFS(\(\mathcal{L}_{2}\))+GAPS on this setting. The results are given in Table 9. We observe a steady increase in both the base IoU and novel IoU in this challenging task setting on methods augmented with GAPS. This further validates the effectiveness of our approach.

Our PIFS(\(\mathcal{L}_{2}\))+GAPS significantly outperforms both baselines.

As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method.

Fig. 5 shows qualitative results of PIFS(\(\mathcal{L}_{2}\)), PIFS(\(\mathcal{L}_{2}\))+NPS (naive copy-paste), and PIFS(\(\mathcal{L}_{2}\))+GAPS on the PASCAL-5-1 split dataset under the 1-shot setting using images on the left column. We can see that PIFS and PIFS+NPS gives reasonably good predictions on classes that are adapted recently (e.g., the cow class in the last row). However, they tend to exhibit catastrophic forgetting for classes learned earlier (e.g., the bus example in PIFS). On the other hand, the model trained using GAPS not only yields good result on recently learned classes, but also maintains promising performance on classes adapted earlier.

We can see that given an image of traffic lights on a railway, our scene embedding model is able to select contextually-similar scenes such as railways and street scenes for subsequent synthesis. Fig. 7 and Fig. 8 demonstrate how PIFS(\(\mathcal{L}_{2}\))+GAPS compare to the baseline PIFS(\(\mathcal{L}_{2}\)) across multiple step.

On the other hand, PIFS(\(\mathcal{L}_{2}\))+GAPS is able to maintain good performance on learned classes as it adapts to new classes.

Input: Table 9: Performance of PIFS(L2) and PIFS(L2)+GAPS on the ADE100-5 disjoint 5-shot setting. Mean and standard deviation over 5 runs are reported.

Input: Figure 4: Performance of PIFS(L2)+GAPS on the COCO-20-1 split under 5-shot setting with varying memory-replay buffer size. Best view in color.

Input: Figure 1: Our proposed method utilizes guided copy-paste augmentation to synthesize diverse training data, using as few as one single novel instance for training. For example, the model encounters an image of many motorcycles, which is novel to the model. As a result, the model incorrectly assigns learned bicycle labels to these pixels and therefore needs to be updated. Our proposed method can adapt to the novel motorcycle class with an annotation of a single motorcycle, which can be efficiently annotated; whereas previous work (Cermelli et al., 2020; 2021) require time-consuming annotation of all instances of motorcycles or even the entire image. Best view in color.

Input: Figure 3: Long-term continual learning performance of PIFS(L2) and PIFS(L2)+GAPS on the COCO-20-1 split under 5-shot setting. Best view in color.

Input: Figure 7: Multi-step qualitative results of our method (GAPS) on the PASCAL-5-1 split under 1- shot setting. (Best view in color.)

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input: Figure 5: Qualitative results on the PASCAL-5-1 split under 1-shot setting. Sequential adaptation to five classes are performed in the order indicated by the blue arrow on the left. Visualization is done after all adaptation steps are completed. Observe, particularly in rows 1, 3, and 4, that our method mitigates catastrophic forgetting and improves performance on both base and novel classes compared to PIFS and PIFS+NPS. (Best viewed in color)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim appears to be well-stated (1) for several reasons:

1. The evidence thoroughly demonstrates that GAPS effectively synthesizes fully-annotated images from partially-annotated examples through guided copy-and-paste methods. The extensive experimental results, ablation studies, and qualitative examples support this capability.

2. The technical details and algorithmic components (context-guided sampling, diversity-guided exemplar selection, vRFS synthesis) are clearly explained and justified.

3. The review comments support the novelty and effectiveness while raising reasonable questions about ablation studies and baselines that don't invalidate the core claim.

4. Multiple experimental settings validate that GAPS consistently improves performance across different datasets, shot settings, and base models, supporting its effectiveness for improving incremental segmentation model training.

<score>1</score>