Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

This paper proposes a data synthesizer, Guided copy-And-Paste Synthesis (GAPS), that improves the performance of few-shot incremental segmentation in a _model-agnostic_ fashion.

The review comment to be evaluated is:

Strengths:
- The paper introduces the copy-paste data augmentation strategy to the few-shot incremental semantic segmentation, which seems to be a novel combination.
- The proposed guided sample selection method for memory construction and data synthesis improves the performance of several baseline methods.
- The paper is most clearly written and easy to follow.

Weaknesses:
- While the proposed data augmentation strategy helps few-shot learning, its technical novelty in the incremental semantic segmentation (ISS) is rather limited, as it is an add-on module and has to rely on the base ISS methods.
- The construction of the memory-replay buffer has several limitations and will affect the quality of the data augmentation in ISS:
1) Only the base stage has fully-annotated images that can be used for the target images in the copy-paste operation, and all the images from the incremental stages are partially labeled, which are unusable for that purpose. As a result, when the incremental learning evolves, the memory has to add samples of new classes and remove some base-stage images due to the capacity limit. Eventually, the memory buffer would be less useful.
2) The diversity-guided selection procedure in Sec 3.2 considers each class separately in sampling. However, for semantic segmentation, it is common that each image has regions of multiple classes. It is unclear how such a method can balance different classes. This is also a problem for the class-frequency-guided selection, as each image can have multiple classes. Another issue is that this method did not take into account the region sizes as it only considers the image-level sampling factor.
- There are several concerns on the experimental evaluation, which make the conclusion less convincing:
1) Both benchmarks are mainly used in the few-shot segmentation literature, which are a bit small in terms of data set size and number of classes. Similar to SSUL and other ISS works, the method should be evaluated on more diverse ISS task settings and larger dataset, such as ADE.
2) For the few-shot side, while this work adopts a different evaluation protocol (as described in Sec. 4.2), it would be more informative if it also uses the protocol in Cermelli et al 2021 for a fair comparison. Moreover, unlike the standard ISS, the training process of the few-shot setting typically has a large variance due to different training samples annotated. Thus it should report both the mean and standard deviation in the final results.
3) The gaps in the ablation study are relatively small. It would more convincing if the standard deviation can be provided. It is unclear how significant such improvements are in the few-shot learning setting.

The evidence to be evaluated is:In contrast, our work enables incremental segmentation learning with _few data_ via a guided copy-paste process, which demonstrates promising performance under the few-shot and more impoverished single-instance setting. Furthermore, GAPS is a _model-agnostic data pre-processor_, which is orthogonal to incremental learning techniques such as regularization (Li and Hoiem, 2017).

However, few-shot semantic segmentation methods produce novel-class-only binary foreground-background segmentation. In comparison, our proposed method works in a more challenging and realistic setting where both base classes and novel classes need to be segmented.

While there are many works in few-shot incremental image classification (Tao et al., 2020; Cheraghian et al., 2021), relatively fewer works have been done to investigate few-shot incremental segmentation (Tian et al., 2020; Cermelli et al., 2021; Ganea et al., 2021). Tian et al. (2020a) designs a meta-learning-based classifier that adjusts learned prototypes by modeling interaction between base classes and incoming novel class. Unlike Tian et al. (2020a), which only performs a single update of weights in the classifier, PIFS (Cermelli et al., 2021) apply regularization techniques to allow fine-tuning of the entire network, achieving state-of-the-art result in few-shot incremental semantic segmentation. However, PIFS (Cermelli et al., 2021) is fine-tuned on only a small number of samples, which leads to sub-optimal performance due to overfitting. In addition, PIFS requires fully-annotated images as input, which hinders its potential for practical applications.

We extend the augmentation strategy from Ghiasi et al. (2021) and construct an intuitive baseline called Naive copy-Paste Synthesis (NPS) to adapt it to our online task setting. However, as we will demonstrate, such naive adaptation gives unsatisfactory performance in our task setting because of _gaps_ between the static offline learning and continual online learning. In our work, we propose a series of techniques to guide the copy-paste synthesizer to address these gaps, whose effectiveness is evident from the significant improvement from NPS.

Different from standard semantic segmentation, in few-shot incremental segmentation, \(\mathcal{C}\) is expanded over time through two stages.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\). In few-shot learning, the size of the training sets for the novel tasks is small, _i.e_., \(|D_{j}|\ll|\mathcal{D}_{0}|\).

The goal of incremental learning is to optimize the model performance jointly on both previous tasks and the current task.

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance.

For methods with memory-replaying (e.g., SSUL (Cha et al., 2021)), GAPS can work directly on top of their constructed buffers with minimal modification. For other methods such as PIFS (Cermelli et al., 2021), we propose a diversity-guided exemplar selection process that builds a small yet diverse memory-replay buffer \(\hat{D_{0}}\) from \(\mathcal{D}_{0}\) to mitigate catastrophic forgetting. Selecting diverse examples that are representative of the base dataset helps mitigate catastrophic forgetting, as suggested by Rebuffi et al. (2017). Inspired by Bang et al. (2021), we select samples distributed uniformly along a spectrum from easy to hard for diversity.

We hypothesize that synthesizing novel objects onto contextually consistent base images would result in an improved learning process. For example, a TV in an apartment should more likely be pasted onto an image of another apartment rather than an outdoor landscape. We design a context-guided sampling algorithm to select images from \(\hat{D_{0}}\) that are contextually similar to the provided partially-labeled images. GAPS uses a scene embedding network \(h:\mathcal{X}\rightarrow\mathbb{R}^{m}\) to estimate pairwise contextual similarity between any two given images. The embedding \(h\) maps an input image to a scene embedding vector in a metric space where pairwise similarity comparison between two embeddings are possible. In our system, we use an off-the-shelf VGGNet (Simonyan and Zisserman, 2015) and replace its last fully connected layer with a cosine-similarity-based classifier. The network \(h\) is trained using the Places365 (Zhou et al., 2017) dataset, which contains \(365\) scene categories and roughly 1.6 million images in its training split. During training, the scene prediction is done by comparing the predicted embedding with existing prototypes.

We first decide if \(c_{o}\) should be pasted onto \(x_{B}\). To apply vRFS resampling, we hallucinate two _virtual_ samples: in the first sample where copy-paste would not be applied, the image-level sampling factor is given by 1. In the second sample where copy-paste synthesis were to be performed, we would obtain a sample with image-level sampling factor of \(r_{i}=\textsc{Max}_{c\in i}r_{c}=r_{c_{o}}\). Thus, the probability to synthesize class \(c_{o}\) onto \(x_{B}\) is given by \(r_{c_{o}}/(1+r_{c_{o}})\). We then repeat the process for the novel class \(c_{N}\). Note that vRFS synthesis is applied twice for every class, resulting in up to two pasted instances of \(c_{N}\) in the final image. We follow literature in few-shot segmentation and few-shot incremental segmentation (Shaban et al., 2017; Nguyen and Todorovic, 2019; Tian et al., 2020; Cermelli et al., 2021) and evaluate our model on the PASCAL-5\({}^{\text{i}}\) dataset (Shaban et al., 2017) and the COCO-20\({}^{\text{i}}\) dataset (Nguyen and Todorovic, 2019).

**GAPS consistently increases performance under few-shot settings.** Methods combined with our proposed data source, GAPS, consistently outperform their un-augmented counterpart on both the base and novel categories performance. It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images. Our proposed GAPS technique selects representative exemplars in the memory-replay buffer and addresses the problems of class imbalance and contextual mismatch in synthesis.

Here, _vRFS_ (virtual RFS) is our proposed method.

In our implementation, the probability to synthesize novel objects is simply given by \(1/M\), where \(M\) is the sum of total number of base classes and seen novel classes. For every selected base image, the _Aways Paste_ method randomly picks a novel category and always synthesizes an instance of the selected novel category onto the given image. Finally, the _50-50-Paste_ method is an extension of the naive copy-paste method to leverage past novel samples like vRFS. Essentially, it replays samples from learned novel classes with 50% probability, and uses novel class from the current incremental learning task also with 50% probability.

In the result, we can observe that our method (vRFS) achieves the best result in terms of the novel IoU and harmonic mean of base and novel IoU.

Beyond confirming conclusion from previous work, such improvement alongside stronger augmentation demonstrates the effectiveness of augmentation technique even when there is few data presented, highlighting a possibility to further improve the performance by using even stronger augmentation technique. We perform an additional experiment on COCO-20-1 where a long few-shot incremental learning sequence (60 base; 20 novel classes with 20 incremental steps; 5-shot) is used.

PIFS(\(\mathcal{L}_{2}\)), augmented with GAPS, demonstrates more robustness to forgetting with many incremental learning steps than standard PIFS. To validate our approach on the ADE20k dataset (Zhou et al., 2017, 2019), we ran a suite of experiments where we follow the ADE100-5 (11 steps) setup discussed in (Cha et al., 2021) and modify it as 5-shot learning. We tested PIFS(\(\mathcal{L}_{2}\)) and PIFS(\(\mathcal{L}_{2}\))+GAPS on this setting. The results are given in Table 9. We observe a steady increase in both the base IoU and novel IoU in this challenging task setting on methods augmented with GAPS.

Our PIFS(\(\mathcal{L}_{2}\))+GAPS significantly outperforms both baselines.

As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method. To allow fair comparisons with existing methods (Cermelli et al., 2021; Cha et al., 2021; Cermelli et al., 2020), we use the same Deeplab-V3 (Chen et al., 2017) architecture with ResNet-101 (He et al., 2016) backbone.

Fig. 5 shows qualitative results of PIFS(\(\mathcal{L}_{2}\)), PIFS(\(\mathcal{L}_{2}\))+NPS (naive copy-paste), and PIFS(\(\mathcal{L}_{2}\))+GAPS on the PASCAL-5-1 split dataset under the 1-shot setting using images on the left column.

On the other hand, the model trained using GAPS not only yields good result on recently learned classes, but also maintains promising performance on classes adapted earlier.

Fig. 7 and Fig. 8 demonstrate how PIFS(\(\mathcal{L}_{2}\))+GAPS compare to the baseline PIFS(\(\mathcal{L}_{2}\)) across multiple step.

These visualizations demonstrate how PIFS(\(\mathcal{L}_{2}\)) forgets learned classes earlier as more incremental learning steps occur. On the other hand, PIFS(\(\mathcal{L}_{2}\))+GAPS is able to maintain good performance on learned classes as it adapts to new classes.

Input: Table 9: Performance of PIFS(L2) and PIFS(L2)+GAPS on the ADE100-5 disjoint 5-shot setting. Mean and standard deviation over 5 runs are reported.

Input: Figure 1: Our proposed method utilizes guided copy-paste augmentation to synthesize diverse training data, using as few as one single novel instance for training. For example, the model encounters an image of many motorcycles, which is novel to the model. As a result, the model incorrectly assigns learned bicycle labels to these pixels and therefore needs to be updated. Our proposed method can adapt to the novel motorcycle class with an annotation of a single motorcycle, which can be efficiently annotated; whereas previous work (Cermelli et al., 2020; 2021) require time-consuming annotation of all instances of motorcycles or even the entire image. Best view in color.

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input: Figure 5: Qualitative results on the PASCAL-5-1 split under 1-shot setting. Sequential adaptation to five classes are performed in the order indicated by the blue arrow on the left. Visualization is done after all adaptation steps are completed. Observe, particularly in rows 1, 3, and 4, that our method mitigates catastrophic forgetting and improves performance on both base and novel classes compared to PIFS and PIFS+NPS. (Best viewed in color)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review, the claim that GAPS is a model-agnostic data synthesizer that improves few-shot incremental segmentation is generally well-supported by the evidence, but has some minor overstatement. The evidence demonstrates:

- Model-agnostic nature through successful integration with multiple models (PIFS, SSUL)
- Consistent performance improvements across different settings and datasets
- Works with partially annotated images and few-shot scenarios

However, the reviewer points out some limitations:
- Limited technical novelty as an add-on module
- Memory buffer limitations over time
- Need for more diverse evaluation settings and statistical significance validation

While the core claim is supported, it could better acknowledge these limitations.

<score>2</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

It is _model-agnostic_, and can be inserted as a plug-and-play module into different incremental learning algorithms, e.g., standard fine-tuning or PIFS (Cermelli et al., 2021).

The evidence to be evaluated is:Furthermore, GAPS is a _model-agnostic data pre-processor_, which is orthogonal to incremental learning techniques such as regularization (Li and Hoiem, 2017).

While there are many works in few-shot incremental image classification (Tao et al., 2020; Cheraghian et al., 2021), relatively fewer works have been done to investigate few-shot incremental segmentation (Tian et al., 2020; Cermelli et al., 2021; Ganea et al., 2021).

Unlike Tian et al. (2020a), which only performs a single update of weights in the classifier, PIFS (Cermelli et al., 2021) apply regularization techniques to allow fine-tuning of the entire network, achieving state-of-the-art result in few-shot incremental semantic segmentation.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\).

After adapting to task \(D_{j}\), the model is updated as \(\phi_{j}:\mathcal{X}\rightarrow\mathcal{Y}^{\mathcal{C}_{j}|_{i=0,\dots,j} \mathcal{C}_{i}}\). The goal of incremental learning is to optimize the model performance jointly on both previous tasks and the current task.

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance. Here we adopt PIFS (Cermelli et al., 2021) as the main baseline underlying segmentation model for its state-of-the-art performance on few-shot incremental semantic segmentation and support for end-to-end training.

For methods with memory-replaying (e.g., SSUL (Cha et al., 2021)), GAPS can work directly on top of their constructed buffers with minimal modification. For other methods such as PIFS (Cermelli et al., 2021), we propose a diversity-guided exemplar selection process that builds a small yet diverse memory-replay buffer \(\hat{D_{0}}\) from \(\mathcal{D}_{0}\) to mitigate catastrophic forgetting.

In addition, we also evaluate simple fine-tuning and MiB (Cermelli et al., 2020).

It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

Since state-of-the-art incremental segmentation approaches use pseudo-labeling (Cha et al., 2021), we design a method PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\), which simulates combining PIFS(\(\mathcal{L}_{2}\)) with pseudo-labeling to cope with partially-annotated sample. Here we allow PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\) to have access to additional information - the annotation of other non-novel background pixels

We can observe that the pseudo-labeling baseline, \(\text{PIFS}(\mathcal{L}_{2})^{\dagger}\), yields substantially worse performance when the model receives single-instance annotations.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images. Our proposed GAPS technique selects representative exemplars in the memory-replay buffer and addresses the problems of class imbalance and contextual mismatch in synthesis.

PIFS(\(\mathcal{L}_{2}\)), augmented with GAPS, demonstrates more robustness to forgetting with many incremental learning steps than standard PIFS.

We tested PIFS(\(\mathcal{L}_{2}\)) and PIFS(\(\mathcal{L}_{2}\))+GAPS on this setting.

Here, 'PIFS(\(\mathcal{L}_{2}\))+MEM+PL' means that we follow SSUL and perform pseudo-labeling on partially annotated samples before adding them to memory.

Our PIFS(\(\mathcal{L}_{2}\))+GAPS significantly outperforms both baselines. We experiment with several other pretraining models as the contextual guiding model.

As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method. To allow fair comparisons with existing methods (Cermelli et al., 2021; Cha et al., 2021; Cermelli et al., 2020), we use the same Deeplab-V3 (Chen et al., 2017) architecture with ResNet-101 (He et al., 2016) backbone.

For fine-tuning and PIFS(\(\mathcal{L}_{2}\)), during the base training stage, we use an initial learning rate of 0.007 and polynomial learning rate schedule with batch size of 32 and train for 20 epochs on both the PASCAL-5\({}^{\text{i}}\) dataset and the COCO-20\({}^{\text{j}}\) dataset. SGD optimizer is used with 0.9 momentum coefficient and 0.0001 weight decay.

Fig. 7 and Fig. 8 demonstrate how PIFS(\(\mathcal{L}_{2}\))+GAPS compare to the baseline PIFS(\(\mathcal{L}_{2}\)) across multiple step.

On the other hand, PIFS(\(\mathcal{L}_{2}\))+GAPS is able to maintain good performance on learned classes as it adapts to new classes.

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the evidence, the claim about GAPS being "model-agnostic" and usable as a "plug-and-play module" is well-supported by multiple pieces of evidence. The paper demonstrates GAPS working with different incremental learning methods including PIFS, fine-tuning, and SSUL. The evidence explicitly states "GAPS is model-agnostic" multiple times and shows through experiments how it can enhance various baseline methods. Figure 2 illustrates GAPS as a preprocessor that can work with different underlying models. The evidence thoroughly validates both aspects of the claim without overstatement.

<score>1</score>

Justification:
1. Multiple direct statements support model-agnostic nature:
- "GAPS is a model-agnostic data pre-processor"
- "In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models"

2. Evidence demonstrates compatibility with multiple methods:
- Works with PIFS (Cermelli et al., 2021)
- Compatible with memory-replaying methods like SSUL
- Tested with simple fine-tuning and MiB
- Shows performance improvements across different baseline methods

3. The "plug-and-play" aspect is supported by:
- Functions as a preprocessor to underlying models
- Can work "directly on top of" existing methods
- Figure 2 clearly shows GAPS as a modular preprocessor component

4. No exaggeration in claim's scope - matches demonstrated capabilities in evidence.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

It is _model-agnostic_, and can be inserted as a plug-and-play module into different incremental learning algorithms, e.g., standard fine-tuning or PIFS (Cermelli et al., 2021).

The review comment to be evaluated is:

Strengths:
1. The paper investigates a novel and understudied scenario that is more realistic than earlier benchmarks.
2. The study gives a comprehensive summary of similar works as well as comparisons and contrasts between them.
3. The study expands the present benchmark with new experiments involving only one labeled instance of the new classes, demonstrating that the GAPS can address the issue of missing annotations.

Weaknesses:
1. The ablation study may be extended to other settings and may provide more insights about the paper choices. In particular, it may start from a baseline that uses exemplars without pasting the new classes on them, to show that pasting is important to improve performance and that the gain is not only given by the regularization effect of using exemplars (as proposed by GDumb [a]). Moreover, the paper should clarify if, while disabling the diversity guidance, the exemplars are sampled uniformly for the old classes or if they are randomly sampled on the full dataset.
2. From the ablation study, context sampling appears to marginally improve the results. Moreover, it is not clear how the performance will change by using a pre-training dataset different from Places-365, such as ImageNet or ADE20K. Furthermore, can it be trained directly on the base dataset of the considered setting or can the method use the segmentation network itself (for example, using an auxiliary classifier on the encoder features)?
3. The paper baselines appear to be very naive. In particular, the paper reports the results for Fine-Tuning, MiB, and PIFS without considering exemplars, introducing an unfair comparison with respect to GAPS. The baselines should be reported both without exemplars (as in the original paper) and with them (indeed, without the proposed copy-and-paste strategy).

[a] Prabhu, Ameya, Philip HS Torr, and Puneet K. Dokania. "Gdumb: A simple approach that questions our progress in continual learning." European conference on computer vision. Springer, Cham, 2020.

The evidence to be evaluated is:Furthermore, GAPS is a _model-agnostic data pre-processor_, which is orthogonal to incremental learning techniques such as regularization (Li and Hoiem, 2017).

While there are many works in few-shot incremental image classification (Tao et al., 2020; Cheraghian et al., 2021), relatively fewer works have been done to investigate few-shot incremental segmentation (Tian et al., 2020; Cermelli et al., 2021; Ganea et al., 2021).

Unlike Tian et al. (2020a), which only performs a single update of weights in the classifier, PIFS (Cermelli et al., 2021) apply regularization techniques to allow fine-tuning of the entire network, achieving state-of-the-art result in few-shot incremental semantic segmentation.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\).

After adapting to task \(D_{j}\), the model is updated as \(\phi_{j}:\mathcal{X}\rightarrow\mathcal{Y}^{\mathcal{C}_{j}|_{i=0,\dots,j} \mathcal{C}_{i}}\). The goal of incremental learning is to optimize the model performance jointly on both previous tasks and the current task.

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance. Here we adopt PIFS (Cermelli et al., 2021) as the main baseline underlying segmentation model for its state-of-the-art performance on few-shot incremental semantic segmentation and support for end-to-end training.

For methods with memory-replaying (e.g., SSUL (Cha et al., 2021)), GAPS can work directly on top of their constructed buffers with minimal modification. For other methods such as PIFS (Cermelli et al., 2021), we propose a diversity-guided exemplar selection process that builds a small yet diverse memory-replay buffer \(\hat{D_{0}}\) from \(\mathcal{D}_{0}\) to mitigate catastrophic forgetting.

In addition, we also evaluate simple fine-tuning and MiB (Cermelli et al., 2020).

It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

Since state-of-the-art incremental segmentation approaches use pseudo-labeling (Cha et al., 2021), we design a method PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\), which simulates combining PIFS(\(\mathcal{L}_{2}\)) with pseudo-labeling to cope with partially-annotated sample. Here we allow PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\) to have access to additional information - the annotation of other non-novel background pixels

We can observe that the pseudo-labeling baseline, \(\text{PIFS}(\mathcal{L}_{2})^{\dagger}\), yields substantially worse performance when the model receives single-instance annotations.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images. Our proposed GAPS technique selects representative exemplars in the memory-replay buffer and addresses the problems of class imbalance and contextual mismatch in synthesis.

PIFS(\(\mathcal{L}_{2}\)), augmented with GAPS, demonstrates more robustness to forgetting with many incremental learning steps than standard PIFS.

We tested PIFS(\(\mathcal{L}_{2}\)) and PIFS(\(\mathcal{L}_{2}\))+GAPS on this setting.

Here, 'PIFS(\(\mathcal{L}_{2}\))+MEM+PL' means that we follow SSUL and perform pseudo-labeling on partially annotated samples before adding them to memory.

Our PIFS(\(\mathcal{L}_{2}\))+GAPS significantly outperforms both baselines. We experiment with several other pretraining models as the contextual guiding model.

As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method. To allow fair comparisons with existing methods (Cermelli et al., 2021; Cha et al., 2021; Cermelli et al., 2020), we use the same Deeplab-V3 (Chen et al., 2017) architecture with ResNet-101 (He et al., 2016) backbone.

For fine-tuning and PIFS(\(\mathcal{L}_{2}\)), during the base training stage, we use an initial learning rate of 0.007 and polynomial learning rate schedule with batch size of 32 and train for 20 epochs on both the PASCAL-5\({}^{\text{i}}\) dataset and the COCO-20\({}^{\text{j}}\) dataset. SGD optimizer is used with 0.9 momentum coefficient and 0.0001 weight decay.

Fig. 7 and Fig. 8 demonstrate how PIFS(\(\mathcal{L}_{2}\))+GAPS compare to the baseline PIFS(\(\mathcal{L}_{2}\)) across multiple step.

On the other hand, PIFS(\(\mathcal{L}_{2}\))+GAPS is able to maintain good performance on learned classes as it adapts to new classes.

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence provided, the claim about GAPS being "model-agnostic" and usable as a "plug-and-play module" is well-supported. The paper demonstrates GAPS working with multiple methods (PIFS, SSUL, fine-tuning, MiB) and explicitly states it is a "generic and model-agnostic data synthesis framework." Figure 2 shows GAPS as a preprocessor that can work with different underlying models. The evidence shows actual implementations with various methods, though primarily focusing on PIFS. While the claim is fundamentally accurate, it could benefit from noting that the degree of performance improvement may vary across different methods.

<score>1</score>

Justification:
1. Multiple pieces of evidence directly support the model-agnostic nature:
- "It is a generic and model-agnostic data synthesis framework"
- "For methods with memory-replaying (e.g., SSUL), GAPS can work directly on top of their constructed buffers"
- Evidence shows implementation with multiple methods (PIFS, fine-tuning, MiB, SSUL)

2. The plug-and-play capability is demonstrated through:
- Figure 2 showing GAPS as a preprocessor module
- Implementation examples with different baseline methods
- Experimental results showing GAPS boosting performance across different methods

3. The claim is carefully stated without overstating capabilities or results.

The review comments do not contradict the model-agnostic nature of GAPS, though they suggest additional baseline comparisons could strengthen the empirical validation.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

It is _model-agnostic_, and can be inserted as a plug-and-play module into different incremental learning algorithms, e.g., standard fine-tuning or PIFS (Cermelli et al., 2021).

The review comment to be evaluated is:

Strength:
 (1) This paper introduces copy-paste as a synthesis technique to few-shot incremental segmentation.
 (2) Experiments show clear improvement on performance with the proposed method.

Weaknesses:
(1)  My main concern is the novelty of the paper, copy-paste technique is more about a data augmentation methods, which has already been shown effective in many segmentation task, it is not novel enough to be accepted as an ICLR paper.
(2) The paper is not clear at least for me, especially figure 1 and figure, it's really hard for readers to understand the task, although the idea is clear.

The evidence to be evaluated is:Furthermore, GAPS is a _model-agnostic data pre-processor_, which is orthogonal to incremental learning techniques such as regularization (Li and Hoiem, 2017).

While there are many works in few-shot incremental image classification (Tao et al., 2020; Cheraghian et al., 2021), relatively fewer works have been done to investigate few-shot incremental segmentation (Tian et al., 2020; Cermelli et al., 2021; Ganea et al., 2021).

Unlike Tian et al. (2020a), which only performs a single update of weights in the classifier, PIFS (Cermelli et al., 2021) apply regularization techniques to allow fine-tuning of the entire network, achieving state-of-the-art result in few-shot incremental semantic segmentation.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\).

After adapting to task \(D_{j}\), the model is updated as \(\phi_{j}:\mathcal{X}\rightarrow\mathcal{Y}^{\mathcal{C}_{j}|_{i=0,\dots,j} \mathcal{C}_{i}}\). The goal of incremental learning is to optimize the model performance jointly on both previous tasks and the current task.

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance. Here we adopt PIFS (Cermelli et al., 2021) as the main baseline underlying segmentation model for its state-of-the-art performance on few-shot incremental semantic segmentation and support for end-to-end training.

For methods with memory-replaying (e.g., SSUL (Cha et al., 2021)), GAPS can work directly on top of their constructed buffers with minimal modification. For other methods such as PIFS (Cermelli et al., 2021), we propose a diversity-guided exemplar selection process that builds a small yet diverse memory-replay buffer \(\hat{D_{0}}\) from \(\mathcal{D}_{0}\) to mitigate catastrophic forgetting.

In addition, we also evaluate simple fine-tuning and MiB (Cermelli et al., 2020).

It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

Since state-of-the-art incremental segmentation approaches use pseudo-labeling (Cha et al., 2021), we design a method PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\), which simulates combining PIFS(\(\mathcal{L}_{2}\)) with pseudo-labeling to cope with partially-annotated sample. Here we allow PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\) to have access to additional information - the annotation of other non-novel background pixels

We can observe that the pseudo-labeling baseline, \(\text{PIFS}(\mathcal{L}_{2})^{\dagger}\), yields substantially worse performance when the model receives single-instance annotations.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images. Our proposed GAPS technique selects representative exemplars in the memory-replay buffer and addresses the problems of class imbalance and contextual mismatch in synthesis.

PIFS(\(\mathcal{L}_{2}\)), augmented with GAPS, demonstrates more robustness to forgetting with many incremental learning steps than standard PIFS.

We tested PIFS(\(\mathcal{L}_{2}\)) and PIFS(\(\mathcal{L}_{2}\))+GAPS on this setting.

Here, 'PIFS(\(\mathcal{L}_{2}\))+MEM+PL' means that we follow SSUL and perform pseudo-labeling on partially annotated samples before adding them to memory.

Our PIFS(\(\mathcal{L}_{2}\))+GAPS significantly outperforms both baselines. We experiment with several other pretraining models as the contextual guiding model.

As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method. To allow fair comparisons with existing methods (Cermelli et al., 2021; Cha et al., 2021; Cermelli et al., 2020), we use the same Deeplab-V3 (Chen et al., 2017) architecture with ResNet-101 (He et al., 2016) backbone.

For fine-tuning and PIFS(\(\mathcal{L}_{2}\)), during the base training stage, we use an initial learning rate of 0.007 and polynomial learning rate schedule with batch size of 32 and train for 20 epochs on both the PASCAL-5\({}^{\text{i}}\) dataset and the COCO-20\({}^{\text{j}}\) dataset. SGD optimizer is used with 0.9 momentum coefficient and 0.0001 weight decay.

Fig. 7 and Fig. 8 demonstrate how PIFS(\(\mathcal{L}_{2}\))+GAPS compare to the baseline PIFS(\(\mathcal{L}_{2}\)) across multiple step.

On the other hand, PIFS(\(\mathcal{L}_{2}\))+GAPS is able to maintain good performance on learned classes as it adapts to new classes.

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim that the proposed method is "model-agnostic and can be inserted as a plug-and-play module into different incremental learning algorithms."

The evidence strongly supports this claim through multiple demonstrations:

1. The paper explicitly states GAPS is a "model-agnostic data pre-processor" and demonstrates this by testing it with multiple methods:
- Standard fine-tuning
- PIFS
- Methods with memory-replaying (SSUL)
- MiB

2. Figure 2 illustrates GAPS as a separate pre-processing module that can work with any underlying model.

3. Results show successful integration with different approaches, including those requiring fully-annotated images and those that only update classifiers.

The claim is well-stated and accurately reflects the evidence without exaggeration, as it's been thoroughly demonstrated across multiple model architectures.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

It is _model-agnostic_, and can be inserted as a plug-and-play module into different incremental learning algorithms, e.g., standard fine-tuning or PIFS (Cermelli et al., 2021).

The review comment to be evaluated is:

Strengths:
- The paper introduces the copy-paste data augmentation strategy to the few-shot incremental semantic segmentation, which seems to be a novel combination.
- The proposed guided sample selection method for memory construction and data synthesis improves the performance of several baseline methods.
- The paper is most clearly written and easy to follow.

Weaknesses:
- While the proposed data augmentation strategy helps few-shot learning, its technical novelty in the incremental semantic segmentation (ISS) is rather limited, as it is an add-on module and has to rely on the base ISS methods.
- The construction of the memory-replay buffer has several limitations and will affect the quality of the data augmentation in ISS:
1) Only the base stage has fully-annotated images that can be used for the target images in the copy-paste operation, and all the images from the incremental stages are partially labeled, which are unusable for that purpose. As a result, when the incremental learning evolves, the memory has to add samples of new classes and remove some base-stage images due to the capacity limit. Eventually, the memory buffer would be less useful.
2) The diversity-guided selection procedure in Sec 3.2 considers each class separately in sampling. However, for semantic segmentation, it is common that each image has regions of multiple classes. It is unclear how such a method can balance different classes. This is also a problem for the class-frequency-guided selection, as each image can have multiple classes. Another issue is that this method did not take into account the region sizes as it only considers the image-level sampling factor.
- There are several concerns on the experimental evaluation, which make the conclusion less convincing:
1) Both benchmarks are mainly used in the few-shot segmentation literature, which are a bit small in terms of data set size and number of classes. Similar to SSUL and other ISS works, the method should be evaluated on more diverse ISS task settings and larger dataset, such as ADE.
2) For the few-shot side, while this work adopts a different evaluation protocol (as described in Sec. 4.2), it would be more informative if it also uses the protocol in Cermelli et al 2021 for a fair comparison. Moreover, unlike the standard ISS, the training process of the few-shot setting typically has a large variance due to different training samples annotated. Thus it should report both the mean and standard deviation in the final results.
3) The gaps in the ablation study are relatively small. It would more convincing if the standard deviation can be provided. It is unclear how significant such improvements are in the few-shot learning setting.

The evidence to be evaluated is:Furthermore, GAPS is a _model-agnostic data pre-processor_, which is orthogonal to incremental learning techniques such as regularization (Li and Hoiem, 2017).

While there are many works in few-shot incremental image classification (Tao et al., 2020; Cheraghian et al., 2021), relatively fewer works have been done to investigate few-shot incremental segmentation (Tian et al., 2020; Cermelli et al., 2021; Ganea et al., 2021).

Unlike Tian et al. (2020a), which only performs a single update of weights in the classifier, PIFS (Cermelli et al., 2021) apply regularization techniques to allow fine-tuning of the entire network, achieving state-of-the-art result in few-shot incremental semantic segmentation.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\).

After adapting to task \(D_{j}\), the model is updated as \(\phi_{j}:\mathcal{X}\rightarrow\mathcal{Y}^{\mathcal{C}_{j}|_{i=0,\dots,j} \mathcal{C}_{i}}\). The goal of incremental learning is to optimize the model performance jointly on both previous tasks and the current task.

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance. Here we adopt PIFS (Cermelli et al., 2021) as the main baseline underlying segmentation model for its state-of-the-art performance on few-shot incremental semantic segmentation and support for end-to-end training.

For methods with memory-replaying (e.g., SSUL (Cha et al., 2021)), GAPS can work directly on top of their constructed buffers with minimal modification. For other methods such as PIFS (Cermelli et al., 2021), we propose a diversity-guided exemplar selection process that builds a small yet diverse memory-replay buffer \(\hat{D_{0}}\) from \(\mathcal{D}_{0}\) to mitigate catastrophic forgetting.

In addition, we also evaluate simple fine-tuning and MiB (Cermelli et al., 2020).

It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

Since state-of-the-art incremental segmentation approaches use pseudo-labeling (Cha et al., 2021), we design a method PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\), which simulates combining PIFS(\(\mathcal{L}_{2}\)) with pseudo-labeling to cope with partially-annotated sample. Here we allow PIFS(\(\mathcal{L}_{2}\))\({}^{\dagger}\) to have access to additional information - the annotation of other non-novel background pixels

We can observe that the pseudo-labeling baseline, \(\text{PIFS}(\mathcal{L}_{2})^{\dagger}\), yields substantially worse performance when the model receives single-instance annotations.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images. Our proposed GAPS technique selects representative exemplars in the memory-replay buffer and addresses the problems of class imbalance and contextual mismatch in synthesis.

PIFS(\(\mathcal{L}_{2}\)), augmented with GAPS, demonstrates more robustness to forgetting with many incremental learning steps than standard PIFS.

We tested PIFS(\(\mathcal{L}_{2}\)) and PIFS(\(\mathcal{L}_{2}\))+GAPS on this setting.

Here, 'PIFS(\(\mathcal{L}_{2}\))+MEM+PL' means that we follow SSUL and perform pseudo-labeling on partially annotated samples before adding them to memory.

Our PIFS(\(\mathcal{L}_{2}\))+GAPS significantly outperforms both baselines. We experiment with several other pretraining models as the contextual guiding model.

As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method. To allow fair comparisons with existing methods (Cermelli et al., 2021; Cha et al., 2021; Cermelli et al., 2020), we use the same Deeplab-V3 (Chen et al., 2017) architecture with ResNet-101 (He et al., 2016) backbone.

For fine-tuning and PIFS(\(\mathcal{L}_{2}\)), during the base training stage, we use an initial learning rate of 0.007 and polynomial learning rate schedule with batch size of 32 and train for 20 epochs on both the PASCAL-5\({}^{\text{i}}\) dataset and the COCO-20\({}^{\text{j}}\) dataset. SGD optimizer is used with 0.9 momentum coefficient and 0.0001 weight decay.

Fig. 7 and Fig. 8 demonstrate how PIFS(\(\mathcal{L}_{2}\))+GAPS compare to the baseline PIFS(\(\mathcal{L}_{2}\)) across multiple step.

On the other hand, PIFS(\(\mathcal{L}_{2}\))+GAPS is able to maintain good performance on learned classes as it adapts to new classes.

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that GAPS is "model-agnostic" and can be inserted as a plug-and-play module appears generally well-supported, but with some minor overstatement. The evidence shows GAPS works with multiple models (PIFS, SSUL, fine-tuning, MiB) as a data preprocessor. However, the reviewer points out it still relies on base ISS methods and has limitations, particularly regarding memory buffer construction and usability across incremental stages. While technically model-agnostic, characterizing it as simple "plug-and-play" slightly overstates its seamless integration given these implementation constraints.

<score>2</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Copy-paste generates diverse training data to boost performance under few-shot settings, enables the model to learn with partially-annotated images with as few as one annotated novel instance out of many novel instances in an image (e.g., as illustrated at the lower left part of Fig. 1), which is a _stricter_ setting than semantic background shift (Cermelli et al., 2020).

The evidence to be evaluated is:Noticeably, a few attempts have been made by recent work to address the semantic background shift problem proposed by Cermelli et al. (2020) via either pseudo-labeling (Cha et al., 2021; Douillard et al., 2021; Yan et al., 2021) or modified loss (Cermelli et al., 2020; Zhang et al., 2022) to train on partially-annotated images of novel classes.

In contrast, our work enables incremental segmentation learning with _few data_ via a guided copy-paste process, which demonstrates promising performance under the few-shot and more impoverished single-instance setting.

Copy-and-paste is an augmentation technique that copies a subset of objects from one image and pastes onto the other image using their segmentation masks. Many works (Dwibedi et al., 2017; Ghiasi et al., 2021; Dvornik et al., 2018) have been done to investigate how copy-and-paste augmentation can help with various visual tasks.

We extend the augmentation strategy from Ghiasi et al. (2021) and construct an intuitive baseline called Naive copy-Paste Synthesis (NPS) to adapt it to our online task setting.

Different from standard semantic segmentation, in few-shot incremental segmentation, \(\mathcal{C}\) is expanded over time through two stages.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\). In few-shot learning, the size of the training sets for the novel tasks is small, _i.e_., \(|D_{j}|\ll|\mathcal{D}_{0}|\).

To enforce the partially-annotated image setting, we follow Cermelli _et al_. and assume that only novel classes are annotated, _i.e_., \(\mathcal{C}_{i}\cap C_{j}=\varnothing\) for all \(i\neq j\).

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

During the incremental learning stage, fully-labeled samples are synthesized by copying from the masked novel objects in \(D_{1},\dots,D_{j}\) and pasting onto base exemplars from the replay buffer \(\hat{D}_{0}\).

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance.

We hypothesize that synthesizing novel objects onto contextually consistent base images would result in an improved learning process. For example, a TV in an apartment should more likely be pasted onto an image of another apartment rather than an outdoor landscape. We design a context-guided sampling algorithm to select images from \(\hat{D_{0}}\) that are contextually similar to the provided partially-labeled images. GAPS uses a scene embedding network \(h:\mathcal{X}\rightarrow\mathbb{R}^{m}\) to estimate pairwise contextual similarity between any two given images. The embedding \(h\) maps an input image to a scene embedding vector in a metric space where pairwise similarity comparison between two embeddings are possible. In our system, we use an off-the-shelf VGGNet (Simonyan and Zisserman, 2015) and replace its last fully connected layer with a cosine-similarity-based classifier. The network \(h\) is trained using the Places365 (Zhou et al., 2017) dataset, which contains \(365\) scene categories and roughly 1.6 million images in its training split. During training, the scene prediction is done by comparing the predicted embedding with existing prototypes.

Note that we only need to compute scene embedding once for every image in \(\hat{D}_{0}\) and incoming partially-annotated images.

Now the final question is, given a fully-annotated image \(x_{B}\) and an image of a novel object \(x_{N}\), how frequent should we apply copy-paste? There is a trade-off between oversampling and undersampling. As one extreme, one can follow Ghiasi et al. (2021) and always apply copy-paste augmentation to paste novel objects onto every base image. However, this will lead to oversampling of novel categories in the current task, which we found to hurt the performance of existing classes.

**GAPS consistently increases performance under few-shot settings.** Methods combined with our proposed data source, GAPS, consistently outperform their un-augmented counterpart on both the base and novel categories performance. It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

Though the semantic background shift proposed by Cermelli et al. (2020) relaxes the requirement to provide full segmentation annotations, it still requires _all novel instances_ in images to be annotated, which can be time-consuming to obtain in cluttered scenes and hinder potential applications. Here we consider a more challenging task setting, which we term _single-instance incremental learning_. Namely, for training images provided in incremental learning stages, if there are multiple instances of a novel class in the image, we assume that only _one instance_ will be annotated. To simulate this setting, we use the instance-level segmentation annotation provided by the COCO dataset to enforce only annotation of one novel instance in every image is available to the model.

This highlights the potential of copy-paste synthesis as an alternative to the existing pseudo-labeling paradigm to cope with the more realistic single-instance setting.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images.

We believe that copy-paste can serve as a promising alternative to pseudo-labeling and modified loss to enable learning on partially-annotated images.

Finally, the ability to learn with as few as one annotated instance in an image raises several intriguing possibilities.

For every selected base image, the _Aways Paste_ method randomly picks a novel category and always synthesizes an instance of the selected novel category onto the given image. Finally, the _50-50-Paste_ method is an extension of the naive copy-paste method to leverage past novel samples like vRFS. Essentially, it replays samples from learned novel classes with 50% probability, and uses novel class from the current incremental learning task also with 50% probability.

Beyond confirming conclusion from previous work, such improvement alongside stronger augmentation demonstrates the effectiveness of augmentation technique even when there is few data presented, highlighting a possibility to further improve the performance by using even stronger augmentation technique. We perform an additional experiment on COCO-20-1 where a long few-shot incremental learning sequence (60 base; 20 novel classes with 20 incremental steps; 5-shot) is used.

To validate our approach on the ADE20k dataset (Zhou et al., 2017, 2019), we ran a suite of experiments where we follow the ADE100-5 (11 steps) setup discussed in (Cha et al., 2021) and modify it as 5-shot learning.

Since examples observed during the incremental learning stage are only partially annotated, they can not be directly added to the memory-replay buffer.

Here, 'PIFS(\(\mathcal{L}_{2}\))+MEM+PL' means that we follow SSUL and perform pseudo-labeling on partially annotated samples before adding them to memory.

We conducted an additional experiment on the COCO-20-1 dataset under the 5-shot setting, where we vary the number of fully-annotated images in the memory-replay buffer to investigate the per formance of our method. As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method.

Input: Figure 1: Our proposed method utilizes guided copy-paste augmentation to synthesize diverse training data, using as few as one single novel instance for training. For example, the model encounters an image of many motorcycles, which is novel to the model. As a result, the model incorrectly assigns learned bicycle labels to these pixels and therefore needs to be updated. Our proposed method can adapt to the novel motorcycle class with an annotation of a single motorcycle, which can be efficiently annotated; whereas previous work (Cermelli et al., 2020; 2021) require time-consuming annotation of all instances of motorcycles or even the entire image. Best view in color.

Input: Figure 7: Multi-step qualitative results of our method (GAPS) on the PASCAL-5-1 split under 1- shot setting. (Best view in color.)

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be well-stated and supported by the evidence. The paper demonstrates through figures, experimental details, and results that their copy-paste method can indeed work with partially-annotated images containing as few as one annotated novel instance. Figure 1 clearly illustrates this capability, showing how the system can learn from a single motorcycle annotation. The evidence provides detailed methodology, experimental validation across multiple datasets, and comparative results showing improved performance even under strict single-instance settings. The claim about it being "stricter" than semantic background shift is justified since previous methods required all instances to be annotated.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Copy-paste generates diverse training data to boost performance under few-shot settings, enables the model to learn with partially-annotated images with as few as one annotated novel instance out of many novel instances in an image (e.g., as illustrated at the lower left part of Fig. 1), which is a _stricter_ setting than semantic background shift (Cermelli et al., 2020).

The review comment to be evaluated is:

Strengths:
1. The paper investigates a novel and understudied scenario that is more realistic than earlier benchmarks.
2. The study gives a comprehensive summary of similar works as well as comparisons and contrasts between them.
3. The study expands the present benchmark with new experiments involving only one labeled instance of the new classes, demonstrating that the GAPS can address the issue of missing annotations.

Weaknesses:
1. The ablation study may be extended to other settings and may provide more insights about the paper choices. In particular, it may start from a baseline that uses exemplars without pasting the new classes on them, to show that pasting is important to improve performance and that the gain is not only given by the regularization effect of using exemplars (as proposed by GDumb [a]). Moreover, the paper should clarify if, while disabling the diversity guidance, the exemplars are sampled uniformly for the old classes or if they are randomly sampled on the full dataset.
2. From the ablation study, context sampling appears to marginally improve the results. Moreover, it is not clear how the performance will change by using a pre-training dataset different from Places-365, such as ImageNet or ADE20K. Furthermore, can it be trained directly on the base dataset of the considered setting or can the method use the segmentation network itself (for example, using an auxiliary classifier on the encoder features)?
3. The paper baselines appear to be very naive. In particular, the paper reports the results for Fine-Tuning, MiB, and PIFS without considering exemplars, introducing an unfair comparison with respect to GAPS. The baselines should be reported both without exemplars (as in the original paper) and with them (indeed, without the proposed copy-and-paste strategy).

[a] Prabhu, Ameya, Philip HS Torr, and Puneet K. Dokania. "Gdumb: A simple approach that questions our progress in continual learning." European conference on computer vision. Springer, Cham, 2020.

The evidence to be evaluated is:Noticeably, a few attempts have been made by recent work to address the semantic background shift problem proposed by Cermelli et al. (2020) via either pseudo-labeling (Cha et al., 2021; Douillard et al., 2021; Yan et al., 2021) or modified loss (Cermelli et al., 2020; Zhang et al., 2022) to train on partially-annotated images of novel classes.

In contrast, our work enables incremental segmentation learning with _few data_ via a guided copy-paste process, which demonstrates promising performance under the few-shot and more impoverished single-instance setting.

Copy-and-paste is an augmentation technique that copies a subset of objects from one image and pastes onto the other image using their segmentation masks. Many works (Dwibedi et al., 2017; Ghiasi et al., 2021; Dvornik et al., 2018) have been done to investigate how copy-and-paste augmentation can help with various visual tasks.

We extend the augmentation strategy from Ghiasi et al. (2021) and construct an intuitive baseline called Naive copy-Paste Synthesis (NPS) to adapt it to our online task setting.

Different from standard semantic segmentation, in few-shot incremental segmentation, \(\mathcal{C}\) is expanded over time through two stages.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\). In few-shot learning, the size of the training sets for the novel tasks is small, _i.e_., \(|D_{j}|\ll|\mathcal{D}_{0}|\).

To enforce the partially-annotated image setting, we follow Cermelli _et al_. and assume that only novel classes are annotated, _i.e_., \(\mathcal{C}_{i}\cap C_{j}=\varnothing\) for all \(i\neq j\).

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

During the incremental learning stage, fully-labeled samples are synthesized by copying from the masked novel objects in \(D_{1},\dots,D_{j}\) and pasting onto base exemplars from the replay buffer \(\hat{D}_{0}\).

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance.

We hypothesize that synthesizing novel objects onto contextually consistent base images would result in an improved learning process. For example, a TV in an apartment should more likely be pasted onto an image of another apartment rather than an outdoor landscape. We design a context-guided sampling algorithm to select images from \(\hat{D_{0}}\) that are contextually similar to the provided partially-labeled images. GAPS uses a scene embedding network \(h:\mathcal{X}\rightarrow\mathbb{R}^{m}\) to estimate pairwise contextual similarity between any two given images. The embedding \(h\) maps an input image to a scene embedding vector in a metric space where pairwise similarity comparison between two embeddings are possible. In our system, we use an off-the-shelf VGGNet (Simonyan and Zisserman, 2015) and replace its last fully connected layer with a cosine-similarity-based classifier. The network \(h\) is trained using the Places365 (Zhou et al., 2017) dataset, which contains \(365\) scene categories and roughly 1.6 million images in its training split. During training, the scene prediction is done by comparing the predicted embedding with existing prototypes.

Note that we only need to compute scene embedding once for every image in \(\hat{D}_{0}\) and incoming partially-annotated images.

Now the final question is, given a fully-annotated image \(x_{B}\) and an image of a novel object \(x_{N}\), how frequent should we apply copy-paste? There is a trade-off between oversampling and undersampling. As one extreme, one can follow Ghiasi et al. (2021) and always apply copy-paste augmentation to paste novel objects onto every base image. However, this will lead to oversampling of novel categories in the current task, which we found to hurt the performance of existing classes.

**GAPS consistently increases performance under few-shot settings.** Methods combined with our proposed data source, GAPS, consistently outperform their un-augmented counterpart on both the base and novel categories performance. It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

Though the semantic background shift proposed by Cermelli et al. (2020) relaxes the requirement to provide full segmentation annotations, it still requires _all novel instances_ in images to be annotated, which can be time-consuming to obtain in cluttered scenes and hinder potential applications. Here we consider a more challenging task setting, which we term _single-instance incremental learning_. Namely, for training images provided in incremental learning stages, if there are multiple instances of a novel class in the image, we assume that only _one instance_ will be annotated. To simulate this setting, we use the instance-level segmentation annotation provided by the COCO dataset to enforce only annotation of one novel instance in every image is available to the model.

This highlights the potential of copy-paste synthesis as an alternative to the existing pseudo-labeling paradigm to cope with the more realistic single-instance setting.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images.

We believe that copy-paste can serve as a promising alternative to pseudo-labeling and modified loss to enable learning on partially-annotated images.

Finally, the ability to learn with as few as one annotated instance in an image raises several intriguing possibilities.

For every selected base image, the _Aways Paste_ method randomly picks a novel category and always synthesizes an instance of the selected novel category onto the given image. Finally, the _50-50-Paste_ method is an extension of the naive copy-paste method to leverage past novel samples like vRFS. Essentially, it replays samples from learned novel classes with 50% probability, and uses novel class from the current incremental learning task also with 50% probability.

Beyond confirming conclusion from previous work, such improvement alongside stronger augmentation demonstrates the effectiveness of augmentation technique even when there is few data presented, highlighting a possibility to further improve the performance by using even stronger augmentation technique. We perform an additional experiment on COCO-20-1 where a long few-shot incremental learning sequence (60 base; 20 novel classes with 20 incremental steps; 5-shot) is used.

To validate our approach on the ADE20k dataset (Zhou et al., 2017, 2019), we ran a suite of experiments where we follow the ADE100-5 (11 steps) setup discussed in (Cha et al., 2021) and modify it as 5-shot learning.

Since examples observed during the incremental learning stage are only partially annotated, they can not be directly added to the memory-replay buffer.

Here, 'PIFS(\(\mathcal{L}_{2}\))+MEM+PL' means that we follow SSUL and perform pseudo-labeling on partially annotated samples before adding them to memory.

We conducted an additional experiment on the COCO-20-1 dataset under the 5-shot setting, where we vary the number of fully-annotated images in the memory-replay buffer to investigate the per formance of our method. As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method.

Input: Figure 1: Our proposed method utilizes guided copy-paste augmentation to synthesize diverse training data, using as few as one single novel instance for training. For example, the model encounters an image of many motorcycles, which is novel to the model. As a result, the model incorrectly assigns learned bicycle labels to these pixels and therefore needs to be updated. Our proposed method can adapt to the novel motorcycle class with an annotation of a single motorcycle, which can be efficiently annotated; whereas previous work (Cermelli et al., 2020; 2021) require time-consuming annotation of all instances of motorcycles or even the entire image. Best view in color.

Input: Figure 7: Multi-step qualitative results of our method (GAPS) on the PASCAL-5-1 split under 1- shot setting. (Best view in color.)

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that it is fairly well-stated but has some minor overstatement issues. The evidence demonstrates that GAPS can indeed work with single-instance annotation (shown in Fig. 1 and described in the text) and is tested under stricter conditions than semantic background shift. However, calling this a "stricter" setting than semantic background shift may be slightly overstated since they present different, not necessarily hierarchical, challenges. The paper does validate the claim through experiments but could be more precise in its comparative language.

<score>2</score>

The claim is supported by:
1. Implementation and testing under single-instance annotation conditions
2. Experimental validation showing GAPS works with minimal annotation
3. Practical demonstration through figures and results
4. Clear explanation of how it differs from semantic background shift

Slight overstatement comes from:
1. Using comparative term "stricter" without fully justifying the hierarchy
2. Could be more precise in describing the relationship between the approaches

The reviewer comments acknowledge the novel scenario but don't specifically address this comparative claim, so they weren't factored heavily into the scoring.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Copy-paste generates diverse training data to boost performance under few-shot settings, enables the model to learn with partially-annotated images with as few as one annotated novel instance out of many novel instances in an image (e.g., as illustrated at the lower left part of Fig. 1), which is a _stricter_ setting than semantic background shift (Cermelli et al., 2020).

The review comment to be evaluated is:

Strength:
 (1) This paper introduces copy-paste as a synthesis technique to few-shot incremental segmentation.
 (2) Experiments show clear improvement on performance with the proposed method.

Weaknesses:
(1)  My main concern is the novelty of the paper, copy-paste technique is more about a data augmentation methods, which has already been shown effective in many segmentation task, it is not novel enough to be accepted as an ICLR paper.
(2) The paper is not clear at least for me, especially figure 1 and figure, it's really hard for readers to understand the task, although the idea is clear.

The evidence to be evaluated is:Noticeably, a few attempts have been made by recent work to address the semantic background shift problem proposed by Cermelli et al. (2020) via either pseudo-labeling (Cha et al., 2021; Douillard et al., 2021; Yan et al., 2021) or modified loss (Cermelli et al., 2020; Zhang et al., 2022) to train on partially-annotated images of novel classes.

In contrast, our work enables incremental segmentation learning with _few data_ via a guided copy-paste process, which demonstrates promising performance under the few-shot and more impoverished single-instance setting.

Copy-and-paste is an augmentation technique that copies a subset of objects from one image and pastes onto the other image using their segmentation masks. Many works (Dwibedi et al., 2017; Ghiasi et al., 2021; Dvornik et al., 2018) have been done to investigate how copy-and-paste augmentation can help with various visual tasks.

We extend the augmentation strategy from Ghiasi et al. (2021) and construct an intuitive baseline called Naive copy-Paste Synthesis (NPS) to adapt it to our online task setting.

Different from standard semantic segmentation, in few-shot incremental segmentation, \(\mathcal{C}\) is expanded over time through two stages.

During the _incremental learning stage_, a sequence of tasks \(\{D_{1},D_{2},\dots\}\) with novel categories is presented to the model, where \(D_{j}=\{(x_{i},y_{i})|x_{i}\in\mathcal{X},y_{i}\in\mathcal{Y}^{\mathcal{C}_{j}}\}\) and \(\mathcal{C}_{j}\) is a set of classes for task \(D_{j}\). In few-shot learning, the size of the training sets for the novel tasks is small, _i.e_., \(|D_{j}|\ll|\mathcal{D}_{0}|\).

To enforce the partially-annotated image setting, we follow Cermelli _et al_. and assume that only novel classes are annotated, _i.e_., \(\mathcal{C}_{i}\cap C_{j}=\varnothing\) for all \(i\neq j\).

It is a _generic and model-agnostic_ data synthesis framework that generates fully-labeled scenes from partially-annotated images of novel objects as a preprocessor to the underlying segmentation model.

During the incremental learning stage, fully-labeled samples are synthesized by copying from the masked novel objects in \(D_{1},\dots,D_{j}\) and pasting onto base exemplars from the replay buffer \(\hat{D}_{0}\).

In principle, GAPS is model-agnostic, which means that it can work with many incremental segmentation models as a diverse data source to improve their performance.

We hypothesize that synthesizing novel objects onto contextually consistent base images would result in an improved learning process. For example, a TV in an apartment should more likely be pasted onto an image of another apartment rather than an outdoor landscape. We design a context-guided sampling algorithm to select images from \(\hat{D_{0}}\) that are contextually similar to the provided partially-labeled images. GAPS uses a scene embedding network \(h:\mathcal{X}\rightarrow\mathbb{R}^{m}\) to estimate pairwise contextual similarity between any two given images. The embedding \(h\) maps an input image to a scene embedding vector in a metric space where pairwise similarity comparison between two embeddings are possible. In our system, we use an off-the-shelf VGGNet (Simonyan and Zisserman, 2015) and replace its last fully connected layer with a cosine-similarity-based classifier. The network \(h\) is trained using the Places365 (Zhou et al., 2017) dataset, which contains \(365\) scene categories and roughly 1.6 million images in its training split. During training, the scene prediction is done by comparing the predicted embedding with existing prototypes.

Note that we only need to compute scene embedding once for every image in \(\hat{D}_{0}\) and incoming partially-annotated images.

Now the final question is, given a fully-annotated image \(x_{B}\) and an image of a novel object \(x_{N}\), how frequent should we apply copy-paste? There is a trade-off between oversampling and undersampling. As one extreme, one can follow Ghiasi et al. (2021) and always apply copy-paste augmentation to paste novel objects onto every base image. However, this will lead to oversampling of novel categories in the current task, which we found to hurt the performance of existing classes.

**GAPS consistently increases performance under few-shot settings.** Methods combined with our proposed data source, GAPS, consistently outperform their un-augmented counterpart on both the base and novel categories performance. It is worth noting that GAPS substantially boosts the performance of methods that originally require fully-annotated training images (i.e., fine-tuning and PIFS), despite using only partially-annotated images now. Even for methods that do not carry out end-to-end training and update only the classifier (i.e., SSUL), GAPS still steadily increases performance on novel categories.

Though the semantic background shift proposed by Cermelli et al. (2020) relaxes the requirement to provide full segmentation annotations, it still requires _all novel instances_ in images to be annotated, which can be time-consuming to obtain in cluttered scenes and hinder potential applications. Here we consider a more challenging task setting, which we term _single-instance incremental learning_. Namely, for training images provided in incremental learning stages, if there are multiple instances of a novel class in the image, we assume that only _one instance_ will be annotated. To simulate this setting, we use the instance-level segmentation annotation provided by the COCO dataset to enforce only annotation of one novel instance in every image is available to the model.

This highlights the potential of copy-paste synthesis as an alternative to the existing pseudo-labeling paradigm to cope with the more realistic single-instance setting.

In this paper, we demonstrate how judicious use of copy-paste dramatically boosts the performance of incremental segmentation methods under the few-shot setting and enables learning with partially-annotated images.

We believe that copy-paste can serve as a promising alternative to pseudo-labeling and modified loss to enable learning on partially-annotated images.

Finally, the ability to learn with as few as one annotated instance in an image raises several intriguing possibilities.

For every selected base image, the _Aways Paste_ method randomly picks a novel category and always synthesizes an instance of the selected novel category onto the given image. Finally, the _50-50-Paste_ method is an extension of the naive copy-paste method to leverage past novel samples like vRFS. Essentially, it replays samples from learned novel classes with 50% probability, and uses novel class from the current incremental learning task also with 50% probability.

Beyond confirming conclusion from previous work, such improvement alongside stronger augmentation demonstrates the effectiveness of augmentation technique even when there is few data presented, highlighting a possibility to further improve the performance by using even stronger augmentation technique. We perform an additional experiment on COCO-20-1 where a long few-shot incremental learning sequence (60 base; 20 novel classes with 20 incremental steps; 5-shot) is used.

To validate our approach on the ADE20k dataset (Zhou et al., 2017, 2019), we ran a suite of experiments where we follow the ADE100-5 (11 steps) setup discussed in (Cha et al., 2021) and modify it as 5-shot learning.

Since examples observed during the incremental learning stage are only partially annotated, they can not be directly added to the memory-replay buffer.

Here, 'PIFS(\(\mathcal{L}_{2}\))+MEM+PL' means that we follow SSUL and perform pseudo-labeling on partially annotated samples before adding them to memory.

We conducted an additional experiment on the COCO-20-1 dataset under the 5-shot setting, where we vary the number of fully-annotated images in the memory-replay buffer to investigate the per formance of our method. As can be seen from Fig. 4, even with as few as 100 fully-annotated base images, GAPS still consistently boosts the performance of the underlying PIFS(\(\mathcal{L}_{2}\)) method.

Input: Figure 1: Our proposed method utilizes guided copy-paste augmentation to synthesize diverse training data, using as few as one single novel instance for training. For example, the model encounters an image of many motorcycles, which is novel to the model. As a result, the model incorrectly assigns learned bicycle labels to these pixels and therefore needs to be updated. Our proposed method can adapt to the novel motorcycle class with an annotation of a single motorcycle, which can be efficiently annotated; whereas previous work (Cermelli et al., 2020; 2021) require time-consuming annotation of all instances of motorcycles or even the entire image. Best view in color.

Input: Figure 7: Multi-step qualitative results of our method (GAPS) on the PASCAL-5-1 split under 1- shot setting. (Best view in color.)

Input: Figure 2: Overview of GAPS. During the incremental learning stage, GAPS takes in as few as one annotated instance of a single image. It is more probable for GAPS to select a scene contextually similar to the provided image from memory-replay buffer D̂0. The image is then probabilistically pasted to generate synthetic fully-labeled scenes. Note that GAPS is model-agnostic, and here we use PIFS (Cermelli et al., 2021) as an example for the underlying segmentation model to illustrate how GAPS is applied as a pre-processor. Best seen in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, the assertion that copy-paste enables learning with partially-annotated images containing as few as one novel instance appears supported by the evidence, but some aspects are overstated. While the evidence demonstrates that GAPS works with single-instance annotation and shows improvements over baselines, calling it a "stricter" setting than semantic background shift is debatable since they address different challenges. The figures and experimental results validate the method's effectiveness but don't fully justify the comparative strength claim.

The reviewer comments highlight that while the method works well, copy-paste is not entirely novel as a technique. The evidence shows good performance but doesn't definitively prove this is a "stricter" setting - it's more accurate to say it's a different type of challenging scenario.

<score>3</score>

Justification:
1. The method does work with single-instance annotation, as demonstrated in the figures and results
2. Experimental evidence shows performance improvements across datasets
3. The term "stricter" is problematic as it implies direct comparison between different types of challenges
4. The evidence focuses more on showing the method works rather than proving it's a stricter setting
5. The reviewer's point about copy-paste being a known technique suggests some overstatement of novelty