Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We propose an end-to-end framework to learn feature compensation for transfer learning with soft gating to decide whether and how much feature compensation is needed, accounting for the mismatch between source domain and target domain.

The review comment to be evaluated is:

Strengths:
- The problem addressed by the paper is relevant.
- The methodology is clear and well-described.
- The experiments are fair and show that the method is competitive with SOTA in some cases.

Weaknesses:
1. In the introduction, the authors remark that "due to the overlap between the source domain and the target domain, not every data example needs transferring, and how much transferring is needed for a data example is subject to its position in the overall data distribution". Nonetheless, in their algorithm, clustering is performed using the target data. Thus, the computed similarities will not tell how far a given target example is from the source distribution. Therefore, I don't see how they are useful to assess "how much transferring is needed" for that example. When the source distribution is very different from the target, the method seems inappropriate. The poor results on MIT Indoor-67 reinforce this suspicion.
2. Are the displacements $\Delta$ normalized? Otherwise, the role of the predicted magnitude $W$ is redundant.
3. The experimental results are not particularly convincing. In particular, it is unclear which of the training strategies proposed in section 3.3 is the best one. In addition, it would be interesting to visualize what kind of transformations the model is learning, i.e. if it's learning $\Delta$s that pull representations towards the cluster centroids (or something else).
4. A theoretical analysis of the proposed methodology is missing. It is presented as a mere heuristic without any theoretical guarantees.

The evidence to be evaluated is:Network-based deep transfer learning reuses the model pre-trained on source domain, including architecture and parameters, to form a part of the solution of target domain, since filters in convolutional layers act as feature extractors in fact and the feature embedding turned out from them are regarded generic to some extent. An early work for speech recognition (Huang et al., 2013) partitions the neural networks of a solution into two parts: Language-irrelevant and language relevant, and reuse the language-irrelevant part for other recognition tasks. Oquab et al. (Oquab et al., 2014) reuses the Convolutional Neural Network (CNN) pre-trained on ImageNet with additional fully connected layers to adapt to target-domain classification. Another classical method, namely fine tune (Yosinski et al., 2014), reuses the original network without any change on the structure and just modifies the parameter values with the data from target domain, where fine tune can be performed on the whole network, or a part of them after fixing the other part.

Fine tune is limited in that the difference between source domain and target domain is not taken into account. In view of that, \(L^{2}\)-SP (Xuhong et al., 2018) and DELTA (Li et al., 2019) apply regularization to loss function in order to compensate for such difference. GTN (Zhu et al., 2018) holds in terms of feature selection to weight features with gating neural network such that the common features shared by source domain and target domain can receive more attention.

From another point of view, parameter modification on the original network (Xuhong et al., 2018; Li et al., 2019; Zhu et al., 2018; Zhong & Maki, 2020) is not able to tackle all mismatches between source and target domain.

In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network. Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

For a given example, whether and how much the corresponding embedding should be changed to approach a new task is subject to its position in the data distribution. If its position is closer to an existing cluster center, in general, a higher probability of correct classification can be expected due to the explicit association with known data representatives. In such a case, the embedding should be changed little. On the contrary, the embedding should be remedied remarkably if its association to any cluster center is not so deterministic. Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

The transfer network to produce \(\mathbf{W}.\Delta=MLP(\mathbf{S}).MLP(\mathbf{X}\)()S) is trained end to end.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework. The proposed transfer learning framework can work in 4 manners to form the following solutions:* (I) At first, fine tune the backbone network, and then fix the backbone and train the transfer network only.

* (II) Train the backbone and transfer network as a whole end to end.

* GTN (Zhu et al., 2018): Gated Transfer Network is used to weight features from pre-trained model, which works in the sense of feature selection.

* PtR (Zhong & Maki, 2020): Pseudo-regression Task Loss is combined with Cross Entropy Loss as a regularization term once the training approaches a sound solution, by whichfeature learning takes into account not only the classification perspective but also the transferability of features.

* BSS (Chen et al., 2019): Batch Spectral Shrinkage (BSS) inhibits negative transfer by suppressing the spectral components with small singular values that correspond to detrimental pre-trained knowledge.

* StochNorm (Kou et al., 2020): Stochastic Normalization (StochNorm) uses a two-branch normalization architecture to avoid model over-depending on biased statistics.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region. This coincides with the principle that transfer learning is in general focused on such data examples that are incompatible with the target domain. Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring. The experiments demonstrate that the proposed feature transferring scheme is technically sound, and a big performance improvement is achieved using a tiny network.

Delta: Deep learning transfer using feature map with attention for convolutional networks.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and reviewer comments, I find this claim needs careful assessment:

The paper proposes an end-to-end framework using soft gating for feature compensation in transfer learning. While the evidence shows a clear implementation with clustering and similarity-based gating mechanisms (Figure 1), and experimental validation with visualizations (Figure 2), the reviewer raises valid concerns about: 1) The clustering being performed only on target data, potentially limiting its ability to assess true domain differences, 2) Lack of theoretical guarantees, and 3) Unclear comparative performance of different training strategies.

The claim is somewhat overstated in suggesting a comprehensive solution without acknowledging these limitations.

<score>3</score>

Justification:
- The framework is indeed implemented end-to-end as claimed, with clear architectural details
- The soft gating mechanism is demonstrated through visualizations
- However, the claim overstates the framework's ability to "account for mismatch between domains" since clustering only uses target domain data
- The claim would be better stated with appropriate caveats about limitations and theoretical foundations
- The review highlights empirical weaknesses that suggest more measured claims would be appropriate


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We propose an end-to-end framework to learn feature compensation for transfer learning with soft gating to decide whether and how much feature compensation is needed, accounting for the mismatch between source domain and target domain.

The review comment to be evaluated is:

Strengths:
- The paper tackles an important problem of identifying the mismatch between a source domain and a target domain before transfer.
- The paper is, in general, well-aligned with literature in the area.

Weaknesses:
- The paper does not provide explanations / intuitions to most technical decisions. For instance, the proposed method is very quickly explained in roughly half a page (sections 3.1 - 3.2). For example, why is it reasonable to use DEC? Why is compensation necessary - what are the underlying factors that make it necessary once the closest cluster centers have been identified? Why is cosine similarity, and no other measure, applicable in this case?
- It is very unclear to me how the proposed method is actually helping to identify the mismatch between a source domain and a target domain. In several parts of the paper, it is mentioned that the cluster centers are calculated on the full training set, and according to my understanding it is the whole training set of the target domain for which cluster centers are being calculated? I think it would greatly benefit the explanation of the method if it would clearly distinguish which parts of it come from the source domain and which from the target domain.
- In the experiments, I do not see any reasoning as to what the proposed method works in some cases but not in others. In which cases doing this kind of clustering + compensation approach would work better? This kind of description is very important to really see the contribution of the proposed method to the problem of transfer learning.

The evidence to be evaluated is:Network-based deep transfer learning reuses the model pre-trained on source domain, including architecture and parameters, to form a part of the solution of target domain, since filters in convolutional layers act as feature extractors in fact and the feature embedding turned out from them are regarded generic to some extent. An early work for speech recognition (Huang et al., 2013) partitions the neural networks of a solution into two parts: Language-irrelevant and language relevant, and reuse the language-irrelevant part for other recognition tasks. Oquab et al. (Oquab et al., 2014) reuses the Convolutional Neural Network (CNN) pre-trained on ImageNet with additional fully connected layers to adapt to target-domain classification. Another classical method, namely fine tune (Yosinski et al., 2014), reuses the original network without any change on the structure and just modifies the parameter values with the data from target domain, where fine tune can be performed on the whole network, or a part of them after fixing the other part.

Fine tune is limited in that the difference between source domain and target domain is not taken into account. In view of that, \(L^{2}\)-SP (Xuhong et al., 2018) and DELTA (Li et al., 2019) apply regularization to loss function in order to compensate for such difference. GTN (Zhu et al., 2018) holds in terms of feature selection to weight features with gating neural network such that the common features shared by source domain and target domain can receive more attention.

From another point of view, parameter modification on the original network (Xuhong et al., 2018; Li et al., 2019; Zhu et al., 2018; Zhong & Maki, 2020) is not able to tackle all mismatches between source and target domain.

In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network. Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

For a given example, whether and how much the corresponding embedding should be changed to approach a new task is subject to its position in the data distribution. If its position is closer to an existing cluster center, in general, a higher probability of correct classification can be expected due to the explicit association with known data representatives. In such a case, the embedding should be changed little. On the contrary, the embedding should be remedied remarkably if its association to any cluster center is not so deterministic. Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

The transfer network to produce \(\mathbf{W}.\Delta=MLP(\mathbf{S}).MLP(\mathbf{X}\)()S) is trained end to end.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework. The proposed transfer learning framework can work in 4 manners to form the following solutions:* (I) At first, fine tune the backbone network, and then fix the backbone and train the transfer network only.

* (II) Train the backbone and transfer network as a whole end to end.

* GTN (Zhu et al., 2018): Gated Transfer Network is used to weight features from pre-trained model, which works in the sense of feature selection.

* PtR (Zhong & Maki, 2020): Pseudo-regression Task Loss is combined with Cross Entropy Loss as a regularization term once the training approaches a sound solution, by whichfeature learning takes into account not only the classification perspective but also the transferability of features.

* BSS (Chen et al., 2019): Batch Spectral Shrinkage (BSS) inhibits negative transfer by suppressing the spectral components with small singular values that correspond to detrimental pre-trained knowledge.

* StochNorm (Kou et al., 2020): Stochastic Normalization (StochNorm) uses a two-branch normalization architecture to avoid model over-depending on biased statistics.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region. This coincides with the principle that transfer learning is in general focused on such data examples that are incompatible with the target domain. Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring. The experiments demonstrate that the proposed feature transferring scheme is technically sound, and a big performance improvement is achieved using a tiny network.

Delta: Deep learning transfer using feature map with attention for convolutional networks.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be moderately overstated. While the paper does present an end-to-end framework with soft gating for feature compensation, several key concerns emerge:

1. The reviewer points out lack of explanation for technical decisions (why DEC? why cosine similarity?)
2. The distinction between source and target domain processing is unclear
3. The evidence shows the method works in some cases but lacks clear explanation of when/why
4. While the framework is demonstrated, the claim's certainty about "accounting for mismatch" isn't fully supported by the experimental results and explanations provided

The framework exists but its effectiveness and theoretical foundations aren't as definitively established as the claim suggests.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We propose an end-to-end framework to learn feature compensation for transfer learning with soft gating to decide whether and how much feature compensation is needed, accounting for the mismatch between source domain and target domain.

The review comment to be evaluated is:

### Strength

- The method is interesting. Studying the contextual relation between source and target domains is novel.
- The related work section is sufficient. So much literature is surveyed.
- Experimental datasets and comparison methods are representative.

### Weakness

- Although the idea of learning contextual relation is novel, this approach is not. It remains unclear whether clustering can model the context information, or if the results of clusters and original inputs can solve the mismatch between source and target domains in transfer learning.
- The claim that "this method only adds two MLPs to the backbone network" is not correct. As can be seen from Figure 1, there are more than two MLPs: it still has an autoencoder (DEC), which is much more complicated than MLPs.
- The method part is extremely confusing and not clear, which should be rewritten. I find figure 1 hard to understand, specifically how to compute $\mathbf{X} \textcopyright \mathbf{S}$ and $\mathbf{W} \cdot \Delta$. (I even think that should be $\cdot$ instead of $.$, since you claim it's a product.)
- The success of "compensation" heavily relies on DEC (deep embedding clustering). If DEC does not give the correct clustering results, the method will fail. How to justify this? Additionally, DEC brings extra computations, which should be highlighted in the paper.
- This method will consume a lot of training time than comparison methods, according to Sec. 3.3. We see that there is more than one step of fine-tuning, which is more complicated than existing work. Authors should compare your training and inference time in the experiments.
- In experiments, you should also list the number of training parameters of each method for a more fair study since this method introduces extra neural network modules.
- More importantly, the performance on general fine-tuning tasks of this method is NOT good. According to table 1, This approach only succeeds using Resnet-101, while it does not generate competitive results in Resnet-50 and VGG16. Hence, it is difficult to say that this approach actually works in different backbones.
- No experiments to explain why this method works. Lack of detailed analysis.
- With regards to related work, I think this paper misses two very important references on the same topic:

[1] Jang Y, Lee H, Hwang S J, et al. Learning what and where to transfer[C]//International Conference on Machine Learning. PMLR, 2019: 3030-3039.

[2] Murugesan K, Sadashivaiah V, Luss R, et al. Auto-Transfer: Learning to Route Transferrable Representations[J]. arXiv preprint arXiv:2202.01011, 2022.

The evidence to be evaluated is:Network-based deep transfer learning reuses the model pre-trained on source domain, including architecture and parameters, to form a part of the solution of target domain, since filters in convolutional layers act as feature extractors in fact and the feature embedding turned out from them are regarded generic to some extent. An early work for speech recognition (Huang et al., 2013) partitions the neural networks of a solution into two parts: Language-irrelevant and language relevant, and reuse the language-irrelevant part for other recognition tasks. Oquab et al. (Oquab et al., 2014) reuses the Convolutional Neural Network (CNN) pre-trained on ImageNet with additional fully connected layers to adapt to target-domain classification. Another classical method, namely fine tune (Yosinski et al., 2014), reuses the original network without any change on the structure and just modifies the parameter values with the data from target domain, where fine tune can be performed on the whole network, or a part of them after fixing the other part.

Fine tune is limited in that the difference between source domain and target domain is not taken into account. In view of that, \(L^{2}\)-SP (Xuhong et al., 2018) and DELTA (Li et al., 2019) apply regularization to loss function in order to compensate for such difference. GTN (Zhu et al., 2018) holds in terms of feature selection to weight features with gating neural network such that the common features shared by source domain and target domain can receive more attention.

From another point of view, parameter modification on the original network (Xuhong et al., 2018; Li et al., 2019; Zhu et al., 2018; Zhong & Maki, 2020) is not able to tackle all mismatches between source and target domain.

In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network. Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

For a given example, whether and how much the corresponding embedding should be changed to approach a new task is subject to its position in the data distribution. If its position is closer to an existing cluster center, in general, a higher probability of correct classification can be expected due to the explicit association with known data representatives. In such a case, the embedding should be changed little. On the contrary, the embedding should be remedied remarkably if its association to any cluster center is not so deterministic. Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

The transfer network to produce \(\mathbf{W}.\Delta=MLP(\mathbf{S}).MLP(\mathbf{X}\)()S) is trained end to end.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework. The proposed transfer learning framework can work in 4 manners to form the following solutions:* (I) At first, fine tune the backbone network, and then fix the backbone and train the transfer network only.

* (II) Train the backbone and transfer network as a whole end to end.

* GTN (Zhu et al., 2018): Gated Transfer Network is used to weight features from pre-trained model, which works in the sense of feature selection.

* PtR (Zhong & Maki, 2020): Pseudo-regression Task Loss is combined with Cross Entropy Loss as a regularization term once the training approaches a sound solution, by whichfeature learning takes into account not only the classification perspective but also the transferability of features.

* BSS (Chen et al., 2019): Batch Spectral Shrinkage (BSS) inhibits negative transfer by suppressing the spectral components with small singular values that correspond to detrimental pre-trained knowledge.

* StochNorm (Kou et al., 2020): Stochastic Normalization (StochNorm) uses a two-branch normalization architecture to avoid model over-depending on biased statistics.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region. This coincides with the principle that transfer learning is in general focused on such data examples that are incompatible with the target domain. Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring. The experiments demonstrate that the proposed feature transferring scheme is technically sound, and a big performance improvement is achieved using a tiny network.

Delta: Deep learning transfer using feature map with attention for convolutional networks.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that while the framework does implement feature compensation with soft gating as claimed, there are several concerns that make the claim somewhat overstated:

1. The review points out that the method's performance is inconsistent across different backbones, working well only with ResNet-101.
2. The framework relies heavily on DEC clustering, which introduces significant complexity not mentioned in the claim.
3. The "end-to-end" claim is questionable since there are multiple training steps in some implementations.
4. The evidence shows the framework is more complex than suggested, with an autoencoder and multiple MLPs.

However, the visualization and experimental results do support the basic concept of context-aware feature compensation and soft gating working as described.

<score>3</score>

Justification:
- The claim describes the core functionality accurately but understates the complexity and limitations
- The method does implement feature compensation with soft gating as claimed
- Evidence shows the framework is more complex and has more dependencies than suggested
- Performance inconsistency across architectures indicates limitations not acknowledged in the claim
- The term "end-to-end" is potentially misleading given the multi-step training options


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We propose an end-to-end framework to learn feature compensation for transfer learning with soft gating to decide whether and how much feature compensation is needed, accounting for the mismatch between source domain and target domain.

The review comment to be evaluated is:

Strengths:
the paper is well written, methodology is well-described, and the problem addressed is relevant.

Weaknesses.
The author is describing how to use data centers to do data compensation, however, how this is related to data transferring is poorly described. For examples, is data centers computed on target data set? how is its feature vector computed, using the pretrained network? using distance to get compensation factor seems only to be working under a strong assumption that this is used for classification, and it only works for image classification tasks? why using distance to centers work? all these questions are not answered properly.

Based on the experimental setup, it is just finetuning the pretrained network using some downstream tasks. It just barely competes with SOTA methods, seems that with or without such 2 layers of compensation networks does not change the result at all.

The evidence to be evaluated is:Network-based deep transfer learning reuses the model pre-trained on source domain, including architecture and parameters, to form a part of the solution of target domain, since filters in convolutional layers act as feature extractors in fact and the feature embedding turned out from them are regarded generic to some extent. An early work for speech recognition (Huang et al., 2013) partitions the neural networks of a solution into two parts: Language-irrelevant and language relevant, and reuse the language-irrelevant part for other recognition tasks. Oquab et al. (Oquab et al., 2014) reuses the Convolutional Neural Network (CNN) pre-trained on ImageNet with additional fully connected layers to adapt to target-domain classification. Another classical method, namely fine tune (Yosinski et al., 2014), reuses the original network without any change on the structure and just modifies the parameter values with the data from target domain, where fine tune can be performed on the whole network, or a part of them after fixing the other part.

Fine tune is limited in that the difference between source domain and target domain is not taken into account. In view of that, \(L^{2}\)-SP (Xuhong et al., 2018) and DELTA (Li et al., 2019) apply regularization to loss function in order to compensate for such difference. GTN (Zhu et al., 2018) holds in terms of feature selection to weight features with gating neural network such that the common features shared by source domain and target domain can receive more attention.

From another point of view, parameter modification on the original network (Xuhong et al., 2018; Li et al., 2019; Zhu et al., 2018; Zhong & Maki, 2020) is not able to tackle all mismatches between source and target domain.

In contrast to (Guo et al., 2020; Liu et al., 2021), our method can approach this goal since we just add a tiny network to obtain enhanced feature to compensate for the feature from backbone network. Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

For a given example, whether and how much the corresponding embedding should be changed to approach a new task is subject to its position in the data distribution. If its position is closer to an existing cluster center, in general, a higher probability of correct classification can be expected due to the explicit association with known data representatives. In such a case, the embedding should be changed little. On the contrary, the embedding should be remedied remarkably if its association to any cluster center is not so deterministic. Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed. As shown in Figure 1, the compensation to the original embedding is subject to two branches: The first branch appends the context vector \(\mathbf{S}\) to the original embedding \(\mathbf{X}\) via concatenation, and then a multilayer perceptron (MLP) is used to transform \(\mathbf{X}\)()S into \(\Delta=MLP(\mathbf{X}\)()S), where MLP is designed to force \(\Delta\) has the same dimension as \(\mathbf{X}\), and () denotes concatenation. The second branch computes the soft gating factor \(\mathbf{W}=MLP(\mathbf{S})\) so as to obtain the weighted compensation subject to the context, that is, \(\mathbf{W}.\Delta\), where \(\mathbf{W}\) is a scalar.

The transfer network to produce \(\mathbf{W}.\Delta=MLP(\mathbf{S}).MLP(\mathbf{X}\)()S) is trained end to end.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework. The proposed transfer learning framework can work in 4 manners to form the following solutions:* (I) At first, fine tune the backbone network, and then fix the backbone and train the transfer network only.

* (II) Train the backbone and transfer network as a whole end to end.

* GTN (Zhu et al., 2018): Gated Transfer Network is used to weight features from pre-trained model, which works in the sense of feature selection.

* PtR (Zhong & Maki, 2020): Pseudo-regression Task Loss is combined with Cross Entropy Loss as a regularization term once the training approaches a sound solution, by whichfeature learning takes into account not only the classification perspective but also the transferability of features.

* BSS (Chen et al., 2019): Batch Spectral Shrinkage (BSS) inhibits negative transfer by suppressing the spectral components with small singular values that correspond to detrimental pre-trained knowledge.

* StochNorm (Kou et al., 2020): Stochastic Normalization (StochNorm) uses a two-branch normalization architecture to avoid model over-depending on biased statistics.

Here, the color of each pixel corresponds with the corresponding soft gating factor \(\mathbf{W}\), and a higher heat value of \(\mathbf{W}\) means a bigger compensation applied to transfer the feature embedding. We can see that bigger compensations are mostly applied to the data points locating in a self-contained region. This coincides with the principle that transfer learning is in general focused on such data examples that are incompatible with the target domain. Figure 2(b) shows the 2-dimensional histogram of maximum cosine similarity against soft gating factor \(\mathbf{W}\), which shows such a trend that lower similarity corresponds with in general higher compensation level, say, higher soft gating. Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring. The experiments demonstrate that the proposed feature transferring scheme is technically sound, and a big performance improvement is achieved using a tiny network.

Delta: Deep learning transfer using feature map with attention for convolutional networks.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be moderately overstated. While the paper does present an end-to-end framework with soft gating for feature compensation, several key concerns emerge:

1. The reviewer points out valid limitations about unclear data center computation and assumptions about classification tasks.

2. The evidence shows the framework uses clustering centers and similarity measures, but doesn't fully justify why this approach is optimal for determining feature compensation needs.

3. The experimental results, according to the reviewer, show the method "barely competes with SOTA methods" and the compensation networks don't significantly change results.

4. While the framework does consider domain mismatch through similarity measures and soft gating, the effectiveness of this approach isn't conclusively demonstrated.

<score>4</score>

The claim overstates the framework's capabilities and effectiveness without acknowledging key limitations and assumptions, though it does accurately describe the basic technical approach used.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We apply clustering at first to represent the data distribution in a compact form of cluster centers, and then compute the similarities between the input and the cluster centers to define its location in the overall data distribution, which acts as a clue to lead subsequent computation on how much compensation is needed to enhance the feature representation from backbone network.

The evidence to be evaluated is:Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

For our solution, the number of cluster centers is set to the true class number of the corresponding data set, which is 256, 200, 120, and 67 for Caltech 256, CUB-200, Stanford Dogs, and MIT Indoor-67, respectively.

This degrades our method much because we use clustering to select representatives but the random selection prevents our method from finding the truly representative examples due to the too sparse random sampling of 30 data examples per class in contrast to the 30,607 images in total.

For our solution (I+), however, its performance improvement compared with that on Caltech 256-30 is minor as the full training data based clustering has lead to the best extent to figure out the data distribution, and the minor improvement comes from the backbone trained with more training data.

Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation. This fully exhibits the motivation of this study, that is, why we make use of clustering centers as the anchors to figure out the contextual clue in directing feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, the statement appears to be well-supported by the detailed technical descriptions, mathematical formulations, and architectural diagrams provided. The evidence thoroughly explains:
- The clustering process to obtain centers representing data distribution
- The computation of similarities between input and cluster centers
- How these similarities inform feature compensation
- The practical implementation details including specific numbers of clusters for different datasets
- Visual evidence through Figure 1 showing the complete architecture
- Experimental validation shown in Figure 2

The claim accurately reflects what is demonstrated in the evidence without exaggeration, making it well-stated.

<score>1</score>

The claim is very precise and measured in describing exactly what the method does - using clustering to represent data distribution and computing similarities to determine compensation needed. The evidence provides comprehensive support through technical details, equations, architecture diagrams, and experimental results that directly validate each component mentioned in the claim. There is no overstatement or exaggeration beyond what is clearly demonstrated.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We apply clustering at first to represent the data distribution in a compact form of cluster centers, and then compute the similarities between the input and the cluster centers to define its location in the overall data distribution, which acts as a clue to lead subsequent computation on how much compensation is needed to enhance the feature representation from backbone network.

The review comment to be evaluated is:

Strengths:
- The problem addressed by the paper is relevant.
- The methodology is clear and well-described.
- The experiments are fair and show that the method is competitive with SOTA in some cases.

Weaknesses:
1. In the introduction, the authors remark that "due to the overlap between the source domain and the target domain, not every data example needs transferring, and how much transferring is needed for a data example is subject to its position in the overall data distribution". Nonetheless, in their algorithm, clustering is performed using the target data. Thus, the computed similarities will not tell how far a given target example is from the source distribution. Therefore, I don't see how they are useful to assess "how much transferring is needed" for that example. When the source distribution is very different from the target, the method seems inappropriate. The poor results on MIT Indoor-67 reinforce this suspicion.
2. Are the displacements $\Delta$ normalized? Otherwise, the role of the predicted magnitude $W$ is redundant.
3. The experimental results are not particularly convincing. In particular, it is unclear which of the training strategies proposed in section 3.3 is the best one. In addition, it would be interesting to visualize what kind of transformations the model is learning, i.e. if it's learning $\Delta$s that pull representations towards the cluster centroids (or something else).
4. A theoretical analysis of the proposed methodology is missing. It is presented as a mere heuristic without any theoretical guarantees.

The evidence to be evaluated is:Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

For our solution, the number of cluster centers is set to the true class number of the corresponding data set, which is 256, 200, 120, and 67 for Caltech 256, CUB-200, Stanford Dogs, and MIT Indoor-67, respectively.

This degrades our method much because we use clustering to select representatives but the random selection prevents our method from finding the truly representative examples due to the too sparse random sampling of 30 data examples per class in contrast to the 30,607 images in total.

For our solution (I+), however, its performance improvement compared with that on Caltech 256-30 is minor as the full training data based clustering has lead to the best extent to figure out the data distribution, and the minor improvement comes from the backbone trained with more training data.

Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation. This fully exhibits the motivation of this study, that is, why we make use of clustering centers as the anchors to figure out the contextual clue in directing feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, the primary issue lies in the claim's implication about using clustering to determine "how much compensation is needed." While the evidence shows that clustering and similarity calculations are indeed used, the reviewer correctly points out that clustering is performed only on target data, making it impossible to truly assess transfer needs relative to the source domain. The evidence demonstrates the method works for computing similarities and contextual positioning, but oversells its ability to determine transfer requirements. The poor performance on MIT Indoor-67 dataset supports this limitation.

Key evidence limitations:
- Clustering only uses target data (not source domain)
- No theoretical guarantees provided
- Limited experimental validation of transfer effectiveness
- Mixed results across datasets

The claim somewhat overstates the method's capability to guide transfer compensation, though the basic technical description is accurate.

<score>3</score>

Justification:
1. The claim accurately describes the technical approach of using clustering and similarity computations
2. However, it overstates the method's ability to determine "how much compensation is needed" since:
   - The clustering doesn't consider source domain information
   - No theoretical proof is provided for transfer effectiveness
   - Evidence shows inconsistent results across datasets
3. The reviewer's critique about source-target domain relationship is valid and not addressed in the evidence
4. The evidence shows the method works as a heuristic but doesn't fully support claims about transfer guidance


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We apply clustering at first to represent the data distribution in a compact form of cluster centers, and then compute the similarities between the input and the cluster centers to define its location in the overall data distribution, which acts as a clue to lead subsequent computation on how much compensation is needed to enhance the feature representation from backbone network.

The review comment to be evaluated is:

Strengths:
- The paper tackles an important problem of identifying the mismatch between a source domain and a target domain before transfer.
- The paper is, in general, well-aligned with literature in the area.

Weaknesses:
- The paper does not provide explanations / intuitions to most technical decisions. For instance, the proposed method is very quickly explained in roughly half a page (sections 3.1 - 3.2). For example, why is it reasonable to use DEC? Why is compensation necessary - what are the underlying factors that make it necessary once the closest cluster centers have been identified? Why is cosine similarity, and no other measure, applicable in this case?
- It is very unclear to me how the proposed method is actually helping to identify the mismatch between a source domain and a target domain. In several parts of the paper, it is mentioned that the cluster centers are calculated on the full training set, and according to my understanding it is the whole training set of the target domain for which cluster centers are being calculated? I think it would greatly benefit the explanation of the method if it would clearly distinguish which parts of it come from the source domain and which from the target domain.
- In the experiments, I do not see any reasoning as to what the proposed method works in some cases but not in others. In which cases doing this kind of clustering + compensation approach would work better? This kind of description is very important to really see the contribution of the proposed method to the problem of transfer learning.

The evidence to be evaluated is:Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

For our solution, the number of cluster centers is set to the true class number of the corresponding data set, which is 256, 200, 120, and 67 for Caltech 256, CUB-200, Stanford Dogs, and MIT Indoor-67, respectively.

This degrades our method much because we use clustering to select representatives but the random selection prevents our method from finding the truly representative examples due to the too sparse random sampling of 30 data examples per class in contrast to the 30,607 images in total.

For our solution (I+), however, its performance improvement compared with that on Caltech 256-30 is minor as the full training data based clustering has lead to the best extent to figure out the data distribution, and the minor improvement comes from the backbone trained with more training data.

Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation. This fully exhibits the motivation of this study, that is, why we make use of clustering centers as the anchors to figure out the contextual clue in directing feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll analyze the claim based on the evidence and review comments:

Summary: The claim describes using clustering to represent data distribution and compute similarities between inputs and cluster centers to guide feature compensation. While the evidence shows a detailed implementation with clustering (DEC), cosine similarity calculations, and compensation mechanisms through MLPs, several key weaknesses exist:

1. The paper lacks explanation of why DEC specifically was chosen and why compensation is necessary
2. The relationship between source and target domains is unclear
3. Limited justification for technical decisions like using cosine similarity
4. Missing analysis of when/why the method works better in some cases

The claim itself reasonably reflects what is implemented, but lacks important context and justification for design choices. The evidence shows the technical implementation matches the claim, but deeper theoretical foundations are missing.

<score>2</score>

I give it a score of 2 - slightly overstated but generally well-supported by the evidence. While the basic mechanism described in the claim is demonstrated in the evidence through figures and mathematical formulations, the claim would be better stated with acknowledgment of limitations and clearer theoretical justification for the approach.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We apply clustering at first to represent the data distribution in a compact form of cluster centers, and then compute the similarities between the input and the cluster centers to define its location in the overall data distribution, which acts as a clue to lead subsequent computation on how much compensation is needed to enhance the feature representation from backbone network.

The review comment to be evaluated is:

### Strength

- The method is interesting. Studying the contextual relation between source and target domains is novel.
- The related work section is sufficient. So much literature is surveyed.
- Experimental datasets and comparison methods are representative.

### Weakness

- Although the idea of learning contextual relation is novel, this approach is not. It remains unclear whether clustering can model the context information, or if the results of clusters and original inputs can solve the mismatch between source and target domains in transfer learning.
- The claim that "this method only adds two MLPs to the backbone network" is not correct. As can be seen from Figure 1, there are more than two MLPs: it still has an autoencoder (DEC), which is much more complicated than MLPs.
- The method part is extremely confusing and not clear, which should be rewritten. I find figure 1 hard to understand, specifically how to compute $\mathbf{X} \textcopyright \mathbf{S}$ and $\mathbf{W} \cdot \Delta$. (I even think that should be $\cdot$ instead of $.$, since you claim it's a product.)
- The success of "compensation" heavily relies on DEC (deep embedding clustering). If DEC does not give the correct clustering results, the method will fail. How to justify this? Additionally, DEC brings extra computations, which should be highlighted in the paper.
- This method will consume a lot of training time than comparison methods, according to Sec. 3.3. We see that there is more than one step of fine-tuning, which is more complicated than existing work. Authors should compare your training and inference time in the experiments.
- In experiments, you should also list the number of training parameters of each method for a more fair study since this method introduces extra neural network modules.
- More importantly, the performance on general fine-tuning tasks of this method is NOT good. According to table 1, This approach only succeeds using Resnet-101, while it does not generate competitive results in Resnet-50 and VGG16. Hence, it is difficult to say that this approach actually works in different backbones.
- No experiments to explain why this method works. Lack of detailed analysis.
- With regards to related work, I think this paper misses two very important references on the same topic:

[1] Jang Y, Lee H, Hwang S J, et al. Learning what and where to transfer[C]//International Conference on Machine Learning. PMLR, 2019: 3030-3039.

[2] Murugesan K, Sadashivaiah V, Luss R, et al. Auto-Transfer: Learning to Route Transferrable Representations[J]. arXiv preprint arXiv:2202.01011, 2022.

The evidence to be evaluated is:Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

For our solution, the number of cluster centers is set to the true class number of the corresponding data set, which is 256, 200, 120, and 67 for Caltech 256, CUB-200, Stanford Dogs, and MIT Indoor-67, respectively.

This degrades our method much because we use clustering to select representatives but the random selection prevents our method from finding the truly representative examples due to the too sparse random sampling of 30 data examples per class in contrast to the 30,607 images in total.

For our solution (I+), however, its performance improvement compared with that on Caltech 256-30 is minor as the full training data based clustering has lead to the best extent to figure out the data distribution, and the minor improvement comes from the backbone trained with more training data.

Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation. This fully exhibits the motivation of this study, that is, why we make use of clustering centers as the anchors to figure out the contextual clue in directing feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, including the reviewer comments, I assess that the claim is moderately overstated. While the evidence shows they do apply clustering and compute similarities between inputs and cluster centers, several issues emerge:

1. The reviewer points out that it's unclear whether clustering effectively models context information
2. The evidence reveals more complexity than suggested (additional DEC autoencoder component)
3. The method heavily relies on DEC's clustering performance
4. The experimental evidence shows inconsistent performance across different backbones
5. The visual evidence (Figure 1) shows additional components not mentioned in the claim

The claim oversimplifies the actual complexity of the method and doesn't acknowledge important limitations and dependencies.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We apply clustering at first to represent the data distribution in a compact form of cluster centers, and then compute the similarities between the input and the cluster centers to define its location in the overall data distribution, which acts as a clue to lead subsequent computation on how much compensation is needed to enhance the feature representation from backbone network.

The review comment to be evaluated is:

Strengths:
the paper is well written, methodology is well-described, and the problem addressed is relevant.

Weaknesses.
The author is describing how to use data centers to do data compensation, however, how this is related to data transferring is poorly described. For examples, is data centers computed on target data set? how is its feature vector computed, using the pretrained network? using distance to get compensation factor seems only to be working under a strong assumption that this is used for classification, and it only works for image classification tasks? why using distance to centers work? all these questions are not answered properly.

Based on the experimental setup, it is just finetuning the pretrained network using some downstream tasks. It just barely competes with SOTA methods, seems that with or without such 2 layers of compensation networks does not change the result at all.

The evidence to be evaluated is:Compared with the Pseudo label used in (Zhong & Maki, 2020), we use the similarity between the data example of interest to all the centers from clustering, which figures out the contextual situation regarding feature discriminant more comprehensively than a simple Pseudo label.

Therefore, we first perform clustering on training data to obtain the centers representing data distribution. Then, we compute the similarity between the input data example and each of the centroids resulting from clustering so as to figure out the position of the input in reference to the cluster centers, where the cluster centers act as anchors to figure out data distribution as well as how far from the input to each anchor.

Then, we compute the cosine similarity between \(\mathbf{X}\) and \(\mathbf{C}_{1},\mathbf{C}_{2},\ldots,\mathbf{C}_{K}\) respectively, and the corresponding similarity is denoted as \(\mathbf{S}=(\mathbf{S}_{1},\mathbf{S}_{2},\ldots,\mathbf{S}_{K})\).

These similarities are referred to as context since they figure out the relative position of \(\mathbf{X}\) in regard to the cluster centers, based on which the compensation to the original feature embedding is computed.

In Eq. (1), we concatenate \(\mathbf{X}\) and \(\mathbf{S}\) to produce the compensation \(MLP(\mathbf{X}(\mathbf{\bar{\odot}}\mathbf{S})\) for transferring feature embedding because \(\mathbf{S}\) indicates how ambiguous \(\mathbf{X}\) is in terms of being associate with a deterministic cluster, say, context. Moreover, the soft weighting \(MLP(\mathbf{S})\) aims to control the overall compensation adaptive to the context \(\mathbf{S}\). It is apparent that anchors resulting from clustering determines the performance of the proposed solution since they are the base to figure out data distribution and aware of the contextual clue to produce compensation adaptively. Here, we make use of the algorithm referred to as Deep Embedded Clustering (DEC) (Xie et al., 2016) to obtain such anchor points, and the experiments confirm that DEC performs very well in the proposed framework.

For our solution, the number of cluster centers is set to the true class number of the corresponding data set, which is 256, 200, 120, and 67 for Caltech 256, CUB-200, Stanford Dogs, and MIT Indoor-67, respectively.

This degrades our method much because we use clustering to select representatives but the random selection prevents our method from finding the truly representative examples due to the too sparse random sampling of 30 data examples per class in contrast to the 30,607 images in total.

For our solution (I+), however, its performance improvement compared with that on Caltech 256-30 is minor as the full training data based clustering has lead to the best extent to figure out the data distribution, and the minor improvement comes from the backbone trained with more training data.

Since the maximum similarity to cluster centers indicates the membership degree of the data example in association with a deterministic cluster, and a smaller value means a higher ambiguous case in terms of membership association, it is thus natural to result in higher-level transferring for feature compensation. This fully exhibits the motivation of this study, that is, why we make use of clustering centers as the anchors to figure out the contextual clue in directing feature compensation.

We propose a new transfer learning method with context-aware compensation to feature embedding, where a contextual descriptor figuring out how close from the input to each cluster center is applied, and soft gating is used to make the feature transfer focused on the data samples that truly need transferring.

Input: Figure 2: (a) Visualization of the data distribution of Stanford Dogs data via PCA. The heat value of each pixel corresponds with the value of the soft gating factor W in association with this pixel. (b) 2-dimensional histogram of maximum cosine similarity against soft gating factor (W ).

Input: Figure 1: Context-directed Transfer Learning

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find this claim to be moderately overstated. Here's why:

While the evidence does show that the method uses clustering centers and similarity computations for feature enhancement, several key issues arise:

1. The review points out valid concerns about unclear relationships between data centers and transfer learning, assumptions about classification tasks, and minimal performance improvements.

2. The evidence shows the technical implementation but doesn't fully validate that this approach "leads subsequent computation" effectively, as claimed. The results suggest minor improvements at best.

3. While the methodology is described in detail, the experimental results don't strongly support the claimed effectiveness of using cluster centers as guiding mechanisms.

The paper provides technical implementation details but overstates the effectiveness and broad applicability of the approach.

<score>4</score>