Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* To resolve the key "chicken-and-egg" problem of extracting radiomic features without available pathology localization, we construct an innovative optimization loop with the image and radiomic branches deeply interacting via attention.

The review comment to be evaluated is:

Strengths
+ The paper is fairly well written and presents a well-studied literature review.
+ The paper proposes an end-to-end transformer-based disease classification and localization model incorporating radiomic feature knowledge with cross-attention. The idea of bounding box generation and radiomic feature extraction in a feedback loop module is new and interesting.
+ Experimental setting including the ablation study for the effect of thresholds on attention maps and the effect of contrastive learning is good.

Weaknesses
- Exposition of results is not good. CheXT is compared against some weak baselines. A more appropriate CNN baseline should be something that includes similar radiomic feature extraction and bounding box generation [1]. Also, [1] is not cited in Related Work.
- The paper claims significant performance improvement by the CheXT model although no evidence (statistical test) is provided to support the claim.
-  Classification performance drops for some of the disease classes (effusion, pneumonia, pneumothorax) after incorporating the radiomics branch. It looks like radiomics branch is hurting the performance for these classes. Is there any impact of label distribution?
- In Fig. 3, two images look different. Why so?
- The training of CheXT is not necessarily semi-supervised as every image has at least disease class label.
- It is not clear how to obtain f_i and f_r; y, y' are not defined Eqn. (1).
- It would be more viable to report both classification and localization performances on the same 880 images. Is there any correlation between e.g., classification AUC and localization IoU?
- It is not clear how the model could improve generalization. There is no generalization assessment of the model for out-of-distribution cases. For example, using other datasets such as, CheXpert and MIMIC-CXR.
- In 4.1, the paper mentions the train/test/val splits, but Table 1 reports results after 3-fold cross-validation. They don't look consistent.
- In 4.4.2: "when the IoU is set to 0.1" should be threshold of IoU. "For Cardiomegaly, Infiltration, and Pneumonia." incomplete sentence.
- It is not clear how higher values of \lambda implies lower weight to contrastive loss in Eqn. (3). How does the model perform without the contrastive loss at all (\lambda=0)?
- There is no mention of the execution speed, computing resource, and failure cases (if any) for the proposed model.
- There are certainly other Transformer-based chest X-ray models such as [2].

1. https://arxiv.org/abs/2011.12506
2. https://arxiv.org/abs/2104.07235

The evidence to be evaluated is:We believe that the joint utilization of radiomics and interpretable learning techniques in our framework can further advance accurate yet interpretable learning in the medical image domain.

Inspired by the promising performance of ViT, researchers have recently applied the idea to medical images. For example, (Oktay et al., 2018; Wang et al., 2019; Chen et al., 2021) used the attention mechanism to boost the performance of medical image segmentation. (Valanarasu et al., 2021) proposed a gated axial-attention model to introduce an additional control mechanism in the self-attention module. (Park et al., 2021) utilized a hybrid framework of CNN and Transformer for Covid-19 prediction.

(Han et al., 2021) applied pre-extracted radiomic features to guide pneumonia detection from chest X-ray images. However, they adopted a convolutional backbone for image encoder, while using a another specifically crafted radiomic features encoder. Therefore, the method involves no joint interaction between image and radiomic features, and need to use accurate bounding boxes during training in order to extract radiomic features.

In CheXT, we extend their idea by treating image itself as one "view" and the radiomic feature extracted from the same image as another "view" (Figure 2). The two views are the jointly learned by interacting through cross-attention. Transformer serves as the modality-agnostic backbone for both. Specifically, we introduce a dual-branch cross-attention Transformer where the first (primary) branch operates the image part, while the second (complementary) branch handles the radiomic features. To resolve the "chicken-and-egg" dilemma in extracting reliable radiomic features without bounding boxes, we have designed a novel _Bootstrap Your Own Attention_ (**BYOA**) module, using feedbacks to learn region localization for extracting radiomic features.

Handcrafted features usually cover a wide range of categories, such as first-order (basic intensity and shaped-based features), second-order (texture features extracted from various matrices), and more advanced features including those calculated from Fourier and wavelet transforms. Specifically, the radiomic features are composed of the following categories:* First-Order statistics features to measure the distribution of voxel intensities within the bounding boxes. The features include energy (the measurement of the magnitude of voxel values), entropy (the measurement of uncertainty in the image values), and max/mean/median gray level intensity within the region of interest.

* Shape-based features, such as Mesh Surface, Pixel Surface, and Perimeter.

* Gray-level features, such as Gray Level Co-occurrence Matrix (GLCM), Gray Level Size Zone (GLSZM), Gray Level Run Length Matrix (GLRLM), Neighboring Gray Tone Difference Matrix (NGTDM), and Gray Level Dependence Matrix (GLDM) features. In short, radiomic features are a set of quantitative features that can describe the characteristics of medical images. In our framework, we aim to make the hidden features of the CLS similar to the radiomic features to learn the localization of pathologies in the chest X-rays. For this branch, we use the vanilla Transformer (Liu et al., 2017) as the radiomic features encoder.

Our main _roadblock_ is how to generate robust radiomic features without pathology localization. On one hand, radiomic features are dependent on and highly sensitive to the local image regions of interest, for which we have no bounding box annotation. On the other hand, image features would benefit from the guidance from radiomic features that encode important domain knowledge. The learning of image and radiomic features are fully entangled, forming a challenging "chicken-and-egg" loop. To address this issue, we design **BYOA** to constitute an end-to-end feedback loop that can bootstrap accurate pathology localization from image data, without any bounding box used for training (Figure 3). BYOA contains two components: attention map generation and radiomic feature extraction.

Here, we have two CLS tokens from two branches, but the attention maps only come from the Image branch. Then we apply a threshold on the self-attention maps to generate bounding boxes for the extraction of radiomic features. The choice of the threshold will influence the quality of the radiomic features. Specifically, the threshold is designed as how much active pixels to keep in the generated attention map. The more active pixels we keep, the larger the generated bounding box is.

Given the original images and generated bounding boxes, we used the Pyradiomic tool to extract radiomic features (Van Griethuysen et al., 2017).

This fusion involves the CLS token of one branch and patch tokens of the other branch. As the CLS token is the aggregate representation of the branch, this interaction helps include information at different scale.

The difference is that we maximize agreement between two feature views extracted from different input formats, one the image and the other radiomics. Given an anchor chest X-ray in a minibatch, the positive sample will be its radiomic feature view, and the negative samples will be other chest X-rays (either image or radiomics).

And we add one more cross-attention layer to fuse these two format features.

Besides, compared to the Transformer-based models, the key difference is that we utilize the extracted radiomic features for disease prediction, which improves the accuracy and adds the model's interpretability due to the utilization of self-designed handcrafted radiomic features.

More importantly, we also add the ViT as our additional baseline here, the results show that the radiomics branch and BYOA module can help the model learn more important regions, which also improves the model's interpretability.

In this paper, we propose an end-to-end knowledge-guided cross-attention Transformer, named CheXT that can jointly model abnormality classification and localization without the supervision of localization annotation of chest X-rays. Our approach differs from previous studies in the choice of universal modeling transformer, the use of radiomic features as prior knowledge, and a feedback loop for image and radiomic features to mutually interact with each other.

Input: Figure 2: Overview of our proposed CheXT. It contains two branches, the Image branch and the Radiomics branch, to process the image and radiomic features (generated by the BYOA module shown in Figure 3), respectively. The output tokens are then fused by an efficient module via cross attention of the CLS tokens. Finally, the output of two CLS tokens (Icls andRcls) are used for disease classification. We minimize the classification errors via a focal loss. Since we train with both labeled and unlabeled images, we leverage a contrastive learning strategy. Specifically, we generate the image view zi = gi(Icls) by a projection head gi. Similarily, we generate the radiomic view zr = gr(Rcls) by another projection head gr. We maximize the agreement between zi and zr via a contrastive loss (NT-Xent). Of note, the contrastive loss is only active during the training.

Input: Figure 3: Overview of our BYOA module. For the input chest X-rays, we look at the self-attention of the CLS token of the Image branch on the heads of the final output of the cross attention module. Then we apply a threshold (0.1) (which means we only keep the top 10% active pixels in the whole attention map) to generate the bounding boxes. Then with the generated bounding boxes, we use the Pyradiomic tool as the radiomic extractor to extract the radiomic features.

Input: Figure 1: General overview of our CheXT framework for Cardiopulmonary Disease Classification and Localization from chest X-rays. CheXT takes the chest X-ray image as the input and outputs a heatmap for pathology localization, based on which the bounding box could be obtained. Radiomic features are further extracted from the bounded region and are fed to predict the disease.

Input: Figure 5: Examples of visualization of localization on the test images. The attention maps are generated from the self-attention maps of the CLS token. The ground-truth bounding boxes are shown in blue. The left image in each pair is the localization result of ViT (Dosovitskiy et al., 2020). The right one is our localization results. All examples are positive for corresponding disease labels. Best viewed in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim, evidence, and review comments, I will assess whether it is well-stated or overstated.

Key points influencing my evaluation:
1. The paper clearly demonstrates an optimization loop where image and radiomic branches interact via attention mechanisms, as shown in Figures 2 and 3. The BYOA module and cross-attention design establish this interaction.
2. Regarding the "chicken-and-egg" problem:
   - The approach does provide a solution by using attention maps to generate bounding boxes without requiring pre-existing pathology localization
   - The feedback loop bootstrap mechanism is well-documented in the evidence
   - The reviewer feedback acknowledges this as a "new and interesting" approach

However, some limitations exist:
- The reviewer notes lack of statistical evidence for performance improvements
- Some disease classes show decreased performance with the radiomic branch
- Limited generalization assessment and ablation studies

The claim is generally well-supported by the evidence, though slightly enthusiastic in calling it "innovative" given some limitations. A score of 2 reflects that it is mostly well-stated with minor overstatement.

<score>2</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* To resolve the key "chicken-and-egg" problem of extracting radiomic features without available pathology localization, we construct an innovative optimization loop with the image and radiomic branches deeply interacting via attention.

The review comment to be evaluated is:

1) Radiomics is a term that most people in the ICLR community are not familiar with. What is it exactily? Is it a set of radiological/image features or bio-markers in medical images? Or something else? The authors should consider explaining it in plain language or ML language.

2) Prior multi-modal feature fusion work in the chest x-ray domain should be discussed, examples below.

    - Moradi, Mehdi, Ali Madani, Yaniv Gur, Yufan Guo, and Tanveer Syeda-Mahmood. "Bimodal network architectures for automatic generation of image annotation from text." arXiv preprint arXiv:1809.01610 (2018).

    - Wang, Xiaosong, Yifan Peng, Le Lu, Zhiyong Lu, and Ronald M. Summers. "TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rays." arXiv e-prints (2018): arXiv-1801.

    - Xue, Yuan, and Xiaolei Huang. "Improved disease classification in chest x-rays with transferred features from report generation." In International Conference on Information Processing in Medical Imaging, pp. 125-138. Springer, Cham, 2019.

    - Liao, Ruizhi, Daniel Moyer, Miriam Cha, Keegan Quigley, Seth Berkowitz, Steven Horng, Polina Golland, and William M. Wells. "Multimodal Representation Learning via Maximization of Local Mutual Information." arXiv preprint arXiv:2103.04537 (2021).

3) I don't see "prior knowledge" and how it's utilized in this work. Isn't the so called prior knowledge in this paper essentially extracted from the data (chest x-ray images)?

4) What are the patch sizes? How are they determined?

5) There are 14 pathology labels in the NIH chest x-ray dataset. Why/how did the authors select the 8 classes out of the 14?

6) In Table 1, make the best results bold instead of in red.

7) Are the AUC values generated on the test set? If so, I'm concerned about overfitting by over tuning the hyper-parameters.

The evidence to be evaluated is:We believe that the joint utilization of radiomics and interpretable learning techniques in our framework can further advance accurate yet interpretable learning in the medical image domain.

Inspired by the promising performance of ViT, researchers have recently applied the idea to medical images. For example, (Oktay et al., 2018; Wang et al., 2019; Chen et al., 2021) used the attention mechanism to boost the performance of medical image segmentation. (Valanarasu et al., 2021) proposed a gated axial-attention model to introduce an additional control mechanism in the self-attention module. (Park et al., 2021) utilized a hybrid framework of CNN and Transformer for Covid-19 prediction.

(Han et al., 2021) applied pre-extracted radiomic features to guide pneumonia detection from chest X-ray images. However, they adopted a convolutional backbone for image encoder, while using a another specifically crafted radiomic features encoder. Therefore, the method involves no joint interaction between image and radiomic features, and need to use accurate bounding boxes during training in order to extract radiomic features.

In CheXT, we extend their idea by treating image itself as one "view" and the radiomic feature extracted from the same image as another "view" (Figure 2). The two views are the jointly learned by interacting through cross-attention. Transformer serves as the modality-agnostic backbone for both. Specifically, we introduce a dual-branch cross-attention Transformer where the first (primary) branch operates the image part, while the second (complementary) branch handles the radiomic features. To resolve the "chicken-and-egg" dilemma in extracting reliable radiomic features without bounding boxes, we have designed a novel _Bootstrap Your Own Attention_ (**BYOA**) module, using feedbacks to learn region localization for extracting radiomic features.

Handcrafted features usually cover a wide range of categories, such as first-order (basic intensity and shaped-based features), second-order (texture features extracted from various matrices), and more advanced features including those calculated from Fourier and wavelet transforms. Specifically, the radiomic features are composed of the following categories:* First-Order statistics features to measure the distribution of voxel intensities within the bounding boxes. The features include energy (the measurement of the magnitude of voxel values), entropy (the measurement of uncertainty in the image values), and max/mean/median gray level intensity within the region of interest.

* Shape-based features, such as Mesh Surface, Pixel Surface, and Perimeter.

* Gray-level features, such as Gray Level Co-occurrence Matrix (GLCM), Gray Level Size Zone (GLSZM), Gray Level Run Length Matrix (GLRLM), Neighboring Gray Tone Difference Matrix (NGTDM), and Gray Level Dependence Matrix (GLDM) features. In short, radiomic features are a set of quantitative features that can describe the characteristics of medical images. In our framework, we aim to make the hidden features of the CLS similar to the radiomic features to learn the localization of pathologies in the chest X-rays. For this branch, we use the vanilla Transformer (Liu et al., 2017) as the radiomic features encoder.

Our main _roadblock_ is how to generate robust radiomic features without pathology localization. On one hand, radiomic features are dependent on and highly sensitive to the local image regions of interest, for which we have no bounding box annotation. On the other hand, image features would benefit from the guidance from radiomic features that encode important domain knowledge. The learning of image and radiomic features are fully entangled, forming a challenging "chicken-and-egg" loop. To address this issue, we design **BYOA** to constitute an end-to-end feedback loop that can bootstrap accurate pathology localization from image data, without any bounding box used for training (Figure 3). BYOA contains two components: attention map generation and radiomic feature extraction.

Here, we have two CLS tokens from two branches, but the attention maps only come from the Image branch. Then we apply a threshold on the self-attention maps to generate bounding boxes for the extraction of radiomic features. The choice of the threshold will influence the quality of the radiomic features. Specifically, the threshold is designed as how much active pixels to keep in the generated attention map. The more active pixels we keep, the larger the generated bounding box is.

Given the original images and generated bounding boxes, we used the Pyradiomic tool to extract radiomic features (Van Griethuysen et al., 2017).

This fusion involves the CLS token of one branch and patch tokens of the other branch. As the CLS token is the aggregate representation of the branch, this interaction helps include information at different scale.

The difference is that we maximize agreement between two feature views extracted from different input formats, one the image and the other radiomics. Given an anchor chest X-ray in a minibatch, the positive sample will be its radiomic feature view, and the negative samples will be other chest X-rays (either image or radiomics).

And we add one more cross-attention layer to fuse these two format features.

Besides, compared to the Transformer-based models, the key difference is that we utilize the extracted radiomic features for disease prediction, which improves the accuracy and adds the model's interpretability due to the utilization of self-designed handcrafted radiomic features.

More importantly, we also add the ViT as our additional baseline here, the results show that the radiomics branch and BYOA module can help the model learn more important regions, which also improves the model's interpretability.

In this paper, we propose an end-to-end knowledge-guided cross-attention Transformer, named CheXT that can jointly model abnormality classification and localization without the supervision of localization annotation of chest X-rays. Our approach differs from previous studies in the choice of universal modeling transformer, the use of radiomic features as prior knowledge, and a feedback loop for image and radiomic features to mutually interact with each other.

Input: Figure 2: Overview of our proposed CheXT. It contains two branches, the Image branch and the Radiomics branch, to process the image and radiomic features (generated by the BYOA module shown in Figure 3), respectively. The output tokens are then fused by an efficient module via cross attention of the CLS tokens. Finally, the output of two CLS tokens (Icls andRcls) are used for disease classification. We minimize the classification errors via a focal loss. Since we train with both labeled and unlabeled images, we leverage a contrastive learning strategy. Specifically, we generate the image view zi = gi(Icls) by a projection head gi. Similarily, we generate the radiomic view zr = gr(Rcls) by another projection head gr. We maximize the agreement between zi and zr via a contrastive loss (NT-Xent). Of note, the contrastive loss is only active during the training.

Input: Figure 3: Overview of our BYOA module. For the input chest X-rays, we look at the self-attention of the CLS token of the Image branch on the heads of the final output of the cross attention module. Then we apply a threshold (0.1) (which means we only keep the top 10% active pixels in the whole attention map) to generate the bounding boxes. Then with the generated bounding boxes, we use the Pyradiomic tool as the radiomic extractor to extract the radiomic features.

Input: Figure 1: General overview of our CheXT framework for Cardiopulmonary Disease Classification and Localization from chest X-rays. CheXT takes the chest X-ray image as the input and outputs a heatmap for pathology localization, based on which the bounding box could be obtained. Radiomic features are further extracted from the bounded region and are fed to predict the disease.

Input: Figure 5: Examples of visualization of localization on the test images. The attention maps are generated from the self-attention maps of the CLS token. The ground-truth bounding boxes are shown in blue. The left image in each pair is the localization result of ViT (Dosovitskiy et al., 2020). The right one is our localization results. All examples are positive for corresponding disease labels. Best viewed in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence carefully, I find the claim to be moderately well-stated but contains some overstatement. While the evidence shows they do implement an innovative feedback loop between image and radiomic features via attention (BYOA module), framing it as resolving the "key chicken-and-egg problem" is somewhat overstated. The evidence demonstrates they propose one possible solution using attention maps and thresholding to generate bounding boxes, but this approach still has limitations and assumptions. The review comments also highlight gaps in explaining radiomics concepts and methodology details.

<score>3</score>

Justification:
1. The evidence supports they developed an attention-based interaction between image and radiomic branches through BYOA, as shown in Figures 2-3.

2. However, claiming to "resolve" the chicken-and-egg problem is too strong since:
- They use a fixed threshold (0.1) to generate bounding boxes, which is a simplifying assumption
- The review points out lack of clarity around patch sizes and methodology details
- No clear demonstration that this fully resolves the localization dependency issue

3. The solution appears to be one workable approach rather than a definitive resolution of this fundamental challenge. More measured language like "addresses" or "proposes an approach to" would better reflect the evidence.

4. The reviewer's comment about lacking clarity on radiomics definition suggests the claims could be more precise in explaining the technical concepts.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* To resolve the key "chicken-and-egg" problem of extracting radiomic features without available pathology localization, we construct an innovative optimization loop with the image and radiomic branches deeply interacting via attention.

The review comment to be evaluated is:

Strength:
- The proposed approach and considering radiomic features along with learnt features is valuable and very relevant to the community.

Main Concerns and comments to improve the paper:
- The paper does not consider evaluation of the method on more publicly available datasets and the study is only limited to NIH-14 Chest X-ray data.
- Baselines and state of the art results on the test data: One important baseline is to make the prediction using only radiomic features.  Also,  [Han’20]  has reported much higher performance on NIH-14 and for the same task which they have not been included in the paper.
Improvement over the CNN baseline on classification: We see 0.011 improvement in mean AUC over Rajpurkar’17 as one of the baseline which is not significant.
- Details of the comparison setups are missing. I am wondering if these numbers are collected from the reported values by authors or they ran every SOTA against their data division (less likely since there is no error bound reported for SOTAs). And if they are-implemented or re-evaluated each method what are the details and consideration for a fair comparison and reproducing the results.
- Main contribution of the paper is to activate usage or radiomic features within the blackbox  structure of deep networks, however, there is no discussion or insight around the meaning and importance of multiple radiomic features that have been used. There are also works that include prior knowledge for Chest X-ray classification including [Han’20]. These features either unlock a better performance (which seems not being really successful in) or demystify the learnt feature space in a way that is pleasant for human interpreters. The paper does not provide such results.

Suggestion to improve the work:
- It would be useful to add some more explanation around radiomics term, feature extraction procedure and meaning of it, since this is not a common term for the machine learning community.
- Consider boosting the results and observation using a few more test data such as MIMIC and CheXpert that have the similar label space as NIH-14 and publicly being used in most of the related papers.
- The above mentioned datasets also can be used for evaluation of the out-of-domain generalization of the method which has not been discussed in the paper.
- Experimental setup and evaluation toward the SOTA can improve significantly. For the comparison purpose I would consider producing a statistical error range with either k-fold cross validation or boot-strapping for both CheXT and SOTAs. I would personally see a proper statistical comparison between a single best SOTA with the proposed method more valuable than having multiple models without any statistical validation.
- Adding interpretation around radiomic feature meaning, how the proposed method unlocked one or more of the 3 listed limitations of CNN methods that has been discussed in the Introduction.
- It is worth also discussing failure cases and the reasoning behind lower performance for some of the conditions such as effusion and pneumonia.

[Han'20] Han, Yan, Chongyan Chen, Liyan Tang, Mingquan Lin, Ajay Jaiswal, Song Wang, Ahmed Tewfik, George Shih, Ying Ding, and Yifan Peng. "Using Radiomics as Prior Knowledge for Thorax Disease Classification and Localization in Chest X-rays." arXiv preprint arXiv:2011.12506 (2020).

The evidence to be evaluated is:We believe that the joint utilization of radiomics and interpretable learning techniques in our framework can further advance accurate yet interpretable learning in the medical image domain.

Inspired by the promising performance of ViT, researchers have recently applied the idea to medical images. For example, (Oktay et al., 2018; Wang et al., 2019; Chen et al., 2021) used the attention mechanism to boost the performance of medical image segmentation. (Valanarasu et al., 2021) proposed a gated axial-attention model to introduce an additional control mechanism in the self-attention module. (Park et al., 2021) utilized a hybrid framework of CNN and Transformer for Covid-19 prediction.

(Han et al., 2021) applied pre-extracted radiomic features to guide pneumonia detection from chest X-ray images. However, they adopted a convolutional backbone for image encoder, while using a another specifically crafted radiomic features encoder. Therefore, the method involves no joint interaction between image and radiomic features, and need to use accurate bounding boxes during training in order to extract radiomic features.

In CheXT, we extend their idea by treating image itself as one "view" and the radiomic feature extracted from the same image as another "view" (Figure 2). The two views are the jointly learned by interacting through cross-attention. Transformer serves as the modality-agnostic backbone for both. Specifically, we introduce a dual-branch cross-attention Transformer where the first (primary) branch operates the image part, while the second (complementary) branch handles the radiomic features. To resolve the "chicken-and-egg" dilemma in extracting reliable radiomic features without bounding boxes, we have designed a novel _Bootstrap Your Own Attention_ (**BYOA**) module, using feedbacks to learn region localization for extracting radiomic features.

Handcrafted features usually cover a wide range of categories, such as first-order (basic intensity and shaped-based features), second-order (texture features extracted from various matrices), and more advanced features including those calculated from Fourier and wavelet transforms. Specifically, the radiomic features are composed of the following categories:* First-Order statistics features to measure the distribution of voxel intensities within the bounding boxes. The features include energy (the measurement of the magnitude of voxel values), entropy (the measurement of uncertainty in the image values), and max/mean/median gray level intensity within the region of interest.

* Shape-based features, such as Mesh Surface, Pixel Surface, and Perimeter.

* Gray-level features, such as Gray Level Co-occurrence Matrix (GLCM), Gray Level Size Zone (GLSZM), Gray Level Run Length Matrix (GLRLM), Neighboring Gray Tone Difference Matrix (NGTDM), and Gray Level Dependence Matrix (GLDM) features. In short, radiomic features are a set of quantitative features that can describe the characteristics of medical images. In our framework, we aim to make the hidden features of the CLS similar to the radiomic features to learn the localization of pathologies in the chest X-rays. For this branch, we use the vanilla Transformer (Liu et al., 2017) as the radiomic features encoder.

Our main _roadblock_ is how to generate robust radiomic features without pathology localization. On one hand, radiomic features are dependent on and highly sensitive to the local image regions of interest, for which we have no bounding box annotation. On the other hand, image features would benefit from the guidance from radiomic features that encode important domain knowledge. The learning of image and radiomic features are fully entangled, forming a challenging "chicken-and-egg" loop. To address this issue, we design **BYOA** to constitute an end-to-end feedback loop that can bootstrap accurate pathology localization from image data, without any bounding box used for training (Figure 3). BYOA contains two components: attention map generation and radiomic feature extraction.

Here, we have two CLS tokens from two branches, but the attention maps only come from the Image branch. Then we apply a threshold on the self-attention maps to generate bounding boxes for the extraction of radiomic features. The choice of the threshold will influence the quality of the radiomic features. Specifically, the threshold is designed as how much active pixels to keep in the generated attention map. The more active pixels we keep, the larger the generated bounding box is.

Given the original images and generated bounding boxes, we used the Pyradiomic tool to extract radiomic features (Van Griethuysen et al., 2017).

This fusion involves the CLS token of one branch and patch tokens of the other branch. As the CLS token is the aggregate representation of the branch, this interaction helps include information at different scale.

The difference is that we maximize agreement between two feature views extracted from different input formats, one the image and the other radiomics. Given an anchor chest X-ray in a minibatch, the positive sample will be its radiomic feature view, and the negative samples will be other chest X-rays (either image or radiomics).

And we add one more cross-attention layer to fuse these two format features.

Besides, compared to the Transformer-based models, the key difference is that we utilize the extracted radiomic features for disease prediction, which improves the accuracy and adds the model's interpretability due to the utilization of self-designed handcrafted radiomic features.

More importantly, we also add the ViT as our additional baseline here, the results show that the radiomics branch and BYOA module can help the model learn more important regions, which also improves the model's interpretability.

In this paper, we propose an end-to-end knowledge-guided cross-attention Transformer, named CheXT that can jointly model abnormality classification and localization without the supervision of localization annotation of chest X-rays. Our approach differs from previous studies in the choice of universal modeling transformer, the use of radiomic features as prior knowledge, and a feedback loop for image and radiomic features to mutually interact with each other.

Input: Figure 2: Overview of our proposed CheXT. It contains two branches, the Image branch and the Radiomics branch, to process the image and radiomic features (generated by the BYOA module shown in Figure 3), respectively. The output tokens are then fused by an efficient module via cross attention of the CLS tokens. Finally, the output of two CLS tokens (Icls andRcls) are used for disease classification. We minimize the classification errors via a focal loss. Since we train with both labeled and unlabeled images, we leverage a contrastive learning strategy. Specifically, we generate the image view zi = gi(Icls) by a projection head gi. Similarily, we generate the radiomic view zr = gr(Rcls) by another projection head gr. We maximize the agreement between zi and zr via a contrastive loss (NT-Xent). Of note, the contrastive loss is only active during the training.

Input: Figure 3: Overview of our BYOA module. For the input chest X-rays, we look at the self-attention of the CLS token of the Image branch on the heads of the final output of the cross attention module. Then we apply a threshold (0.1) (which means we only keep the top 10% active pixels in the whole attention map) to generate the bounding boxes. Then with the generated bounding boxes, we use the Pyradiomic tool as the radiomic extractor to extract the radiomic features.

Input: Figure 1: General overview of our CheXT framework for Cardiopulmonary Disease Classification and Localization from chest X-rays. CheXT takes the chest X-ray image as the input and outputs a heatmap for pathology localization, based on which the bounding box could be obtained. Radiomic features are further extracted from the bounded region and are fed to predict the disease.

Input: Figure 5: Examples of visualization of localization on the test images. The attention maps are generated from the self-attention maps of the CLS token. The ground-truth bounding boxes are shown in blue. The left image in each pair is the localization result of ViT (Dosovitskiy et al., 2020). The right one is our localization results. All examples are positive for corresponding disease labels. Best viewed in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be moderately overstated. While the evidence shows they do develop an interactive optimization approach between image and radiomic branches using attention mechanisms (as shown in Figures 2 and 3), calling it "innovative" and claiming it "resolves" the chicken-and-egg problem may be too strong. The reviewer points out limitations including modest performance improvements and lack of validation across multiple datasets. The BYOA module provides a workaround rather than a complete resolution, using a simple threshold-based approach (keeping top 10% active pixels) to generate bounding boxes for radiomic feature extraction.

<score>4</score>

Justification:
1. The use of "innovative" is subjective and not well-supported by the modest performance gains noted in the review
2. "Resolves" implies a complete solution, but the evidence shows a basic thresholding approach
3. Limited validation on only one dataset (NIH-14) as highlighted by reviewer
4. The method provides a practical workaround rather than truly "resolving" the fundamental chicken-and-egg challenge
5. The interaction between branches is demonstrated but its effectiveness needs more robust validation


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes.

The evidence to be evaluated is:In CheXT, we extend their idea by treating image itself as one "view" and the radiomic feature extracted from the same image as another "view" (Figure 2). The two views are the jointly learned by interacting through cross-attention.

To resolve the "chicken-and-egg" dilemma in extracting reliable radiomic features without bounding boxes, we have designed a novel _Bootstrap Your Own Attention_ (**BYOA**) module, using feedbacks to learn region localization for extracting radiomic features.

The primary image branch uses a Progress-Sampling ViT (PS-ViT) (Yue et al., 2021) as its backbone. Unlike the vanilla ViT that splits images into fixed-size tokens, PS-ViT utilizes an iterative and progressive sampling strategy to locate discriminative regions and avoid over-partition object structures. We experimentally observed PS-ViT outperforms ViT and other variants in our framework because it generates higher-quality and more structure-aware attention maps, which are crucial for estimating the pathology localization during training.

In our framework, we aim to make the hidden features of the CLS similar to the radiomic features to learn the localization of pathologies in the chest X-rays.

Our main _roadblock_ is how to generate robust radiomic features without pathology localization. On one hand, radiomic features are dependent on and highly sensitive to the local image regions of interest, for which we have no bounding box annotation. On the other hand, image features would benefit from the guidance from radiomic features that encode important domain knowledge. The learning of image and radiomic features are fully entangled, forming a challenging "chicken-and-egg" loop. To address this issue, we design **BYOA** to constitute an end-to-end feedback loop that can bootstrap accurate pathology localization from image data, without any bounding box used for training (Figure 3).

Then we apply a threshold on the self-attention maps to generate bounding boxes for the extraction of radiomic features. The choice of the threshold will influence the quality of the radiomic features. Specifically, the threshold is designed as how much active pixels to keep in the generated attention map. The more active pixels we keep, the larger the generated bounding box is.

Given the original images and generated bounding boxes, we used the Pyradiomic tool to extract radiomic features (Van Griethuysen et al., 2017).

Our contrastive loss extends the normalized temperature scaled cross-entropy loss (NT-Xent). The difference is that we maximize agreement between two feature views extracted from different input formats, one the image and the other radiomics. Given an anchor chest X-ray in a minibatch, the positive sample will be its radiomic feature view, and the negative samples will be other chest X-rays (either image or radiomics). Since the CLS could be regarded as the representation of all other tokens, we only need to maximize the agreement between them.

Since for chest x-rays, there exists two labels, disease class labels and pathology bounding box annotations. In our case, we only have access to the disease labels, however, bounding box annotations are more important than the disease class labels for chest x-rays. Here, when we say "semi-supervised", we refer to the bounding box annotations, not disease class labels. For multi-task, one task is supervised disease classification. The other is unsupervised cross-view contrastive learning.

A significant difference between our method and existing baseline methods (Liu et al., 2019; Li et al., 2018) is that our method does not require any training data related to the bounding box while others use some percentage of these images for training. In our experiments, we followed the same protocol as in the studies of (Wang et al., 2017; Li et al., 2018), to shuffle our dataset (excluding images with Bounding Boxes annotations) into three subsets: 70% for training, 10% for validation, and 20% for testing.

During our experiments, we found that CheXT performs worse when very large bounding boxes are generated. This observation meets our expectations since radiomic features are very subtle to bounding boxes. Specifically, the extracted radiomic features are more robust if more focal and accurate bounding boxes are given.

Many prior works (Li et al., 2018; Liu et al., 2019) have used a fraction of ground truth (GT) bounding boxes for training and evaluated their system on the remaining. To ensure a robust evaluation, we do not use any GT for training.

For localization, we evaluated our detected regular rectangular regions against the annotated bounding boxes, using intersection over union ratio (IoU).

In this paper, we propose an end-to-end knowledge-guided cross-attention Transformer, named CheXT that can jointly model abnormality classification and localization without the supervision of localization annotation of chest X-rays. Our approach differs from previous studies in the choice of universal modeling transformer, the use of radiomic features as prior knowledge, and a feedback loop for image and radiomic features to mutually interact with each other.

Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, especially for the disease localization task, where our method can generate more accurate bounding boxes.

Input: Figure 2: Overview of our proposed CheXT. It contains two branches, the Image branch and the Radiomics branch, to process the image and radiomic features (generated by the BYOA module shown in Figure 3), respectively. The output tokens are then fused by an efficient module via cross attention of the CLS tokens. Finally, the output of two CLS tokens (Icls andRcls) are used for disease classification. We minimize the classification errors via a focal loss. Since we train with both labeled and unlabeled images, we leverage a contrastive learning strategy. Specifically, we generate the image view zi = gi(Icls) by a projection head gi. Similarily, we generate the radiomic view zr = gr(Rcls) by another projection head gr. We maximize the agreement between zi and zr via a contrastive loss (NT-Xent). Of note, the contrastive loss is only active during the training.

Input: Figure 3: Overview of our BYOA module. For the input chest X-rays, we look at the self-attention of the CLS token of the Image branch on the heads of the final output of the cross attention module. Then we apply a threshold (0.1) (which means we only keep the top 10% active pixels in the whole attention map) to generate the bounding boxes. Then with the generated bounding boxes, we use the Pyradiomic tool as the radiomic extractor to extract the radiomic features.

Input: Figure 1: General overview of our CheXT framework for Cardiopulmonary Disease Classification and Localization from chest X-rays. CheXT takes the chest X-ray image as the input and outputs a heatmap for pathology localization, based on which the bounding box could be obtained. Radiomic features are further extracted from the bounded region and are fed to predict the disease.

Input: Figure 5: Examples of visualization of localization on the test images. The attention maps are generated from the self-attention maps of the CLS token. The ground-truth bounding boxes are shown in blue. The left image in each pair is the localization result of ViT (Dosovitskiy et al., 2020). The right one is our localization results. All examples are positive for corresponding disease labels. Best viewed in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim "Such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes" appears well-supported by the presented evidence. The paper demonstrates:

1. A clear technical approach (BYOA module + feedback loop) for generating bounding boxes without annotations
2. Visual evidence in Figure 5 showing successful localization compared to baseline ViT
3. Quantitative evaluation using IoU against ground truth boxes
4. Clear differentiation from prior work that required partial bounding box annotations
5. Experimental validation with proper train/test splits excluding annotated data

While the results show promise, the claim remains measured by using "can bootstrap" rather than making absolute statements about performance.

<score>1</score>

Justification:
- The claim is well-stated and accurately reflects the demonstrated capabilities without exaggeration
- Evidence includes comprehensive technical details, ablation studies, and visual results
- Authors acknowledge limitations (e.g., "performs worse when very large bounding boxes are generated")
- The framework is clearly differentiated from previous approaches requiring annotations
- Results are validated through proper experimental protocols and metrics


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes.

The review comment to be evaluated is:

Strengths
+ The paper is fairly well written and presents a well-studied literature review.
+ The paper proposes an end-to-end transformer-based disease classification and localization model incorporating radiomic feature knowledge with cross-attention. The idea of bounding box generation and radiomic feature extraction in a feedback loop module is new and interesting.
+ Experimental setting including the ablation study for the effect of thresholds on attention maps and the effect of contrastive learning is good.

Weaknesses
- Exposition of results is not good. CheXT is compared against some weak baselines. A more appropriate CNN baseline should be something that includes similar radiomic feature extraction and bounding box generation [1]. Also, [1] is not cited in Related Work.
- The paper claims significant performance improvement by the CheXT model although no evidence (statistical test) is provided to support the claim.
-  Classification performance drops for some of the disease classes (effusion, pneumonia, pneumothorax) after incorporating the radiomics branch. It looks like radiomics branch is hurting the performance for these classes. Is there any impact of label distribution?
- In Fig. 3, two images look different. Why so?
- The training of CheXT is not necessarily semi-supervised as every image has at least disease class label.
- It is not clear how to obtain f_i and f_r; y, y' are not defined Eqn. (1).
- It would be more viable to report both classification and localization performances on the same 880 images. Is there any correlation between e.g., classification AUC and localization IoU?
- It is not clear how the model could improve generalization. There is no generalization assessment of the model for out-of-distribution cases. For example, using other datasets such as, CheXpert and MIMIC-CXR.
- In 4.1, the paper mentions the train/test/val splits, but Table 1 reports results after 3-fold cross-validation. They don't look consistent.
- In 4.4.2: "when the IoU is set to 0.1" should be threshold of IoU. "For Cardiomegaly, Infiltration, and Pneumonia." incomplete sentence.
- It is not clear how higher values of \lambda implies lower weight to contrastive loss in Eqn. (3). How does the model perform without the contrastive loss at all (\lambda=0)?
- There is no mention of the execution speed, computing resource, and failure cases (if any) for the proposed model.
- There are certainly other Transformer-based chest X-ray models such as [2].

1. https://arxiv.org/abs/2011.12506
2. https://arxiv.org/abs/2104.07235

The evidence to be evaluated is:In CheXT, we extend their idea by treating image itself as one "view" and the radiomic feature extracted from the same image as another "view" (Figure 2). The two views are the jointly learned by interacting through cross-attention.

To resolve the "chicken-and-egg" dilemma in extracting reliable radiomic features without bounding boxes, we have designed a novel _Bootstrap Your Own Attention_ (**BYOA**) module, using feedbacks to learn region localization for extracting radiomic features.

The primary image branch uses a Progress-Sampling ViT (PS-ViT) (Yue et al., 2021) as its backbone. Unlike the vanilla ViT that splits images into fixed-size tokens, PS-ViT utilizes an iterative and progressive sampling strategy to locate discriminative regions and avoid over-partition object structures. We experimentally observed PS-ViT outperforms ViT and other variants in our framework because it generates higher-quality and more structure-aware attention maps, which are crucial for estimating the pathology localization during training.

In our framework, we aim to make the hidden features of the CLS similar to the radiomic features to learn the localization of pathologies in the chest X-rays.

Our main _roadblock_ is how to generate robust radiomic features without pathology localization. On one hand, radiomic features are dependent on and highly sensitive to the local image regions of interest, for which we have no bounding box annotation. On the other hand, image features would benefit from the guidance from radiomic features that encode important domain knowledge. The learning of image and radiomic features are fully entangled, forming a challenging "chicken-and-egg" loop. To address this issue, we design **BYOA** to constitute an end-to-end feedback loop that can bootstrap accurate pathology localization from image data, without any bounding box used for training (Figure 3).

Then we apply a threshold on the self-attention maps to generate bounding boxes for the extraction of radiomic features. The choice of the threshold will influence the quality of the radiomic features. Specifically, the threshold is designed as how much active pixels to keep in the generated attention map. The more active pixels we keep, the larger the generated bounding box is.

Given the original images and generated bounding boxes, we used the Pyradiomic tool to extract radiomic features (Van Griethuysen et al., 2017).

Our contrastive loss extends the normalized temperature scaled cross-entropy loss (NT-Xent). The difference is that we maximize agreement between two feature views extracted from different input formats, one the image and the other radiomics. Given an anchor chest X-ray in a minibatch, the positive sample will be its radiomic feature view, and the negative samples will be other chest X-rays (either image or radiomics). Since the CLS could be regarded as the representation of all other tokens, we only need to maximize the agreement between them.

Since for chest x-rays, there exists two labels, disease class labels and pathology bounding box annotations. In our case, we only have access to the disease labels, however, bounding box annotations are more important than the disease class labels for chest x-rays. Here, when we say "semi-supervised", we refer to the bounding box annotations, not disease class labels. For multi-task, one task is supervised disease classification. The other is unsupervised cross-view contrastive learning.

A significant difference between our method and existing baseline methods (Liu et al., 2019; Li et al., 2018) is that our method does not require any training data related to the bounding box while others use some percentage of these images for training. In our experiments, we followed the same protocol as in the studies of (Wang et al., 2017; Li et al., 2018), to shuffle our dataset (excluding images with Bounding Boxes annotations) into three subsets: 70% for training, 10% for validation, and 20% for testing.

During our experiments, we found that CheXT performs worse when very large bounding boxes are generated. This observation meets our expectations since radiomic features are very subtle to bounding boxes. Specifically, the extracted radiomic features are more robust if more focal and accurate bounding boxes are given.

Many prior works (Li et al., 2018; Liu et al., 2019) have used a fraction of ground truth (GT) bounding boxes for training and evaluated their system on the remaining. To ensure a robust evaluation, we do not use any GT for training.

For localization, we evaluated our detected regular rectangular regions against the annotated bounding boxes, using intersection over union ratio (IoU).

In this paper, we propose an end-to-end knowledge-guided cross-attention Transformer, named CheXT that can jointly model abnormality classification and localization without the supervision of localization annotation of chest X-rays. Our approach differs from previous studies in the choice of universal modeling transformer, the use of radiomic features as prior knowledge, and a feedback loop for image and radiomic features to mutually interact with each other.

Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, especially for the disease localization task, where our method can generate more accurate bounding boxes.

Input: Figure 2: Overview of our proposed CheXT. It contains two branches, the Image branch and the Radiomics branch, to process the image and radiomic features (generated by the BYOA module shown in Figure 3), respectively. The output tokens are then fused by an efficient module via cross attention of the CLS tokens. Finally, the output of two CLS tokens (Icls andRcls) are used for disease classification. We minimize the classification errors via a focal loss. Since we train with both labeled and unlabeled images, we leverage a contrastive learning strategy. Specifically, we generate the image view zi = gi(Icls) by a projection head gi. Similarily, we generate the radiomic view zr = gr(Rcls) by another projection head gr. We maximize the agreement between zi and zr via a contrastive loss (NT-Xent). Of note, the contrastive loss is only active during the training.

Input: Figure 3: Overview of our BYOA module. For the input chest X-rays, we look at the self-attention of the CLS token of the Image branch on the heads of the final output of the cross attention module. Then we apply a threshold (0.1) (which means we only keep the top 10% active pixels in the whole attention map) to generate the bounding boxes. Then with the generated bounding boxes, we use the Pyradiomic tool as the radiomic extractor to extract the radiomic features.

Input: Figure 1: General overview of our CheXT framework for Cardiopulmonary Disease Classification and Localization from chest X-rays. CheXT takes the chest X-ray image as the input and outputs a heatmap for pathology localization, based on which the bounding box could be obtained. Radiomic features are further extracted from the bounded region and are fed to predict the disease.

Input: Figure 5: Examples of visualization of localization on the test images. The attention maps are generated from the self-attention maps of the CLS token. The ground-truth bounding boxes are shown in blue. The left image in each pair is the localization result of ViT (Dosovitskiy et al., 2020). The right one is our localization results. All examples are positive for corresponding disease labels. Best viewed in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim "Such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes."

Summary:
The claim appears somewhat overstated based on the evidence. While the paper demonstrates a novel approach using BYOA module and cross-attention between image and radiomic features to generate localization without bounding box annotations, several limitations exist:

1. Reviewer points out lack of statistical evidence for performance claims
2. No assessment of generalization to other datasets
3. Limited evaluation of failure cases
4. The evidence shows variable performance across different diseases
5. The visualizations in Figure 5 suggest reasonable but not necessarily "accurate" localization in all cases

The claim would be better stated with more measured language about the capabilities and limitations of the approach.

<score>4</score>

Justification:
1. "Accurate" is an overstatement - the paper doesn't quantitatively define or prove accuracy levels across all pathologies

2. The reviewer notes: "It is not clear how the model could improve generalization. There is no generalization assessment of the model for out-of-distribution cases."

3. The evidence shows limitations: "CheXT performs worse when very large bounding boxes are generated" and "Classification performance drops for some of the disease classes"

4. While novel, the approach lacks comprehensive validation - no statistical tests, limited comparison to strong baselines, and no failure analysis as noted in reviews

5. The visual results in Figure 5 show reasonable but variable localization quality across different conditions, not definitively proving "accurate" localization in all cases

A more precise claim would acknowledge both the capabilities and limitations of the bootstrapping approach.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes.

The review comment to be evaluated is:

1) Radiomics is a term that most people in the ICLR community are not familiar with. What is it exactily? Is it a set of radiological/image features or bio-markers in medical images? Or something else? The authors should consider explaining it in plain language or ML language.

2) Prior multi-modal feature fusion work in the chest x-ray domain should be discussed, examples below.

    - Moradi, Mehdi, Ali Madani, Yaniv Gur, Yufan Guo, and Tanveer Syeda-Mahmood. "Bimodal network architectures for automatic generation of image annotation from text." arXiv preprint arXiv:1809.01610 (2018).

    - Wang, Xiaosong, Yifan Peng, Le Lu, Zhiyong Lu, and Ronald M. Summers. "TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rays." arXiv e-prints (2018): arXiv-1801.

    - Xue, Yuan, and Xiaolei Huang. "Improved disease classification in chest x-rays with transferred features from report generation." In International Conference on Information Processing in Medical Imaging, pp. 125-138. Springer, Cham, 2019.

    - Liao, Ruizhi, Daniel Moyer, Miriam Cha, Keegan Quigley, Seth Berkowitz, Steven Horng, Polina Golland, and William M. Wells. "Multimodal Representation Learning via Maximization of Local Mutual Information." arXiv preprint arXiv:2103.04537 (2021).

3) I don't see "prior knowledge" and how it's utilized in this work. Isn't the so called prior knowledge in this paper essentially extracted from the data (chest x-ray images)?

4) What are the patch sizes? How are they determined?

5) There are 14 pathology labels in the NIH chest x-ray dataset. Why/how did the authors select the 8 classes out of the 14?

6) In Table 1, make the best results bold instead of in red.

7) Are the AUC values generated on the test set? If so, I'm concerned about overfitting by over tuning the hyper-parameters.

The evidence to be evaluated is:In CheXT, we extend their idea by treating image itself as one "view" and the radiomic feature extracted from the same image as another "view" (Figure 2). The two views are the jointly learned by interacting through cross-attention.

To resolve the "chicken-and-egg" dilemma in extracting reliable radiomic features without bounding boxes, we have designed a novel _Bootstrap Your Own Attention_ (**BYOA**) module, using feedbacks to learn region localization for extracting radiomic features.

The primary image branch uses a Progress-Sampling ViT (PS-ViT) (Yue et al., 2021) as its backbone. Unlike the vanilla ViT that splits images into fixed-size tokens, PS-ViT utilizes an iterative and progressive sampling strategy to locate discriminative regions and avoid over-partition object structures. We experimentally observed PS-ViT outperforms ViT and other variants in our framework because it generates higher-quality and more structure-aware attention maps, which are crucial for estimating the pathology localization during training.

In our framework, we aim to make the hidden features of the CLS similar to the radiomic features to learn the localization of pathologies in the chest X-rays.

Our main _roadblock_ is how to generate robust radiomic features without pathology localization. On one hand, radiomic features are dependent on and highly sensitive to the local image regions of interest, for which we have no bounding box annotation. On the other hand, image features would benefit from the guidance from radiomic features that encode important domain knowledge. The learning of image and radiomic features are fully entangled, forming a challenging "chicken-and-egg" loop. To address this issue, we design **BYOA** to constitute an end-to-end feedback loop that can bootstrap accurate pathology localization from image data, without any bounding box used for training (Figure 3).

Then we apply a threshold on the self-attention maps to generate bounding boxes for the extraction of radiomic features. The choice of the threshold will influence the quality of the radiomic features. Specifically, the threshold is designed as how much active pixels to keep in the generated attention map. The more active pixels we keep, the larger the generated bounding box is.

Given the original images and generated bounding boxes, we used the Pyradiomic tool to extract radiomic features (Van Griethuysen et al., 2017).

Our contrastive loss extends the normalized temperature scaled cross-entropy loss (NT-Xent). The difference is that we maximize agreement between two feature views extracted from different input formats, one the image and the other radiomics. Given an anchor chest X-ray in a minibatch, the positive sample will be its radiomic feature view, and the negative samples will be other chest X-rays (either image or radiomics). Since the CLS could be regarded as the representation of all other tokens, we only need to maximize the agreement between them.

Since for chest x-rays, there exists two labels, disease class labels and pathology bounding box annotations. In our case, we only have access to the disease labels, however, bounding box annotations are more important than the disease class labels for chest x-rays. Here, when we say "semi-supervised", we refer to the bounding box annotations, not disease class labels. For multi-task, one task is supervised disease classification. The other is unsupervised cross-view contrastive learning.

A significant difference between our method and existing baseline methods (Liu et al., 2019; Li et al., 2018) is that our method does not require any training data related to the bounding box while others use some percentage of these images for training. In our experiments, we followed the same protocol as in the studies of (Wang et al., 2017; Li et al., 2018), to shuffle our dataset (excluding images with Bounding Boxes annotations) into three subsets: 70% for training, 10% for validation, and 20% for testing.

During our experiments, we found that CheXT performs worse when very large bounding boxes are generated. This observation meets our expectations since radiomic features are very subtle to bounding boxes. Specifically, the extracted radiomic features are more robust if more focal and accurate bounding boxes are given.

Many prior works (Li et al., 2018; Liu et al., 2019) have used a fraction of ground truth (GT) bounding boxes for training and evaluated their system on the remaining. To ensure a robust evaluation, we do not use any GT for training.

For localization, we evaluated our detected regular rectangular regions against the annotated bounding boxes, using intersection over union ratio (IoU).

In this paper, we propose an end-to-end knowledge-guided cross-attention Transformer, named CheXT that can jointly model abnormality classification and localization without the supervision of localization annotation of chest X-rays. Our approach differs from previous studies in the choice of universal modeling transformer, the use of radiomic features as prior knowledge, and a feedback loop for image and radiomic features to mutually interact with each other.

Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, especially for the disease localization task, where our method can generate more accurate bounding boxes.

Input: Figure 2: Overview of our proposed CheXT. It contains two branches, the Image branch and the Radiomics branch, to process the image and radiomic features (generated by the BYOA module shown in Figure 3), respectively. The output tokens are then fused by an efficient module via cross attention of the CLS tokens. Finally, the output of two CLS tokens (Icls andRcls) are used for disease classification. We minimize the classification errors via a focal loss. Since we train with both labeled and unlabeled images, we leverage a contrastive learning strategy. Specifically, we generate the image view zi = gi(Icls) by a projection head gi. Similarily, we generate the radiomic view zr = gr(Rcls) by another projection head gr. We maximize the agreement between zi and zr via a contrastive loss (NT-Xent). Of note, the contrastive loss is only active during the training.

Input: Figure 3: Overview of our BYOA module. For the input chest X-rays, we look at the self-attention of the CLS token of the Image branch on the heads of the final output of the cross attention module. Then we apply a threshold (0.1) (which means we only keep the top 10% active pixels in the whole attention map) to generate the bounding boxes. Then with the generated bounding boxes, we use the Pyradiomic tool as the radiomic extractor to extract the radiomic features.

Input: Figure 1: General overview of our CheXT framework for Cardiopulmonary Disease Classification and Localization from chest X-rays. CheXT takes the chest X-ray image as the input and outputs a heatmap for pathology localization, based on which the bounding box could be obtained. Radiomic features are further extracted from the bounded region and are fed to predict the disease.

Input: Figure 5: Examples of visualization of localization on the test images. The attention maps are generated from the self-attention maps of the CLS token. The ground-truth bounding boxes are shown in blue. The left image in each pair is the localization result of ViT (Dosovitskiy et al., 2020). The right one is our localization results. All examples are positive for corresponding disease labels. Best viewed in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find that the claim "Such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes" is modestly overstated. While the evidence shows the method can generate localization without bounding box annotations, the term "accurate" needs qualification. The visualizations in Figure 5 show reasonable but not perfect localization, and the paper acknowledges performance sensitivity to bounding box size. The method shows promise but has limitations that aren't acknowledged in the claim's confident phrasing.

Here's why:
1. The paper notes "CheXT performs worse when very large bounding boxes are generated"
2. The approach relies on thresholding attention maps (only keeping top 10% active pixels)
3. While the method works without human annotations, accuracy levels compared to supervised approaches aren't clearly stated in the evidence

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes.

The review comment to be evaluated is:

Strength:
- The proposed approach and considering radiomic features along with learnt features is valuable and very relevant to the community.

Main Concerns and comments to improve the paper:
- The paper does not consider evaluation of the method on more publicly available datasets and the study is only limited to NIH-14 Chest X-ray data.
- Baselines and state of the art results on the test data: One important baseline is to make the prediction using only radiomic features.  Also,  [Han’20]  has reported much higher performance on NIH-14 and for the same task which they have not been included in the paper.
Improvement over the CNN baseline on classification: We see 0.011 improvement in mean AUC over Rajpurkar’17 as one of the baseline which is not significant.
- Details of the comparison setups are missing. I am wondering if these numbers are collected from the reported values by authors or they ran every SOTA against their data division (less likely since there is no error bound reported for SOTAs). And if they are-implemented or re-evaluated each method what are the details and consideration for a fair comparison and reproducing the results.
- Main contribution of the paper is to activate usage or radiomic features within the blackbox  structure of deep networks, however, there is no discussion or insight around the meaning and importance of multiple radiomic features that have been used. There are also works that include prior knowledge for Chest X-ray classification including [Han’20]. These features either unlock a better performance (which seems not being really successful in) or demystify the learnt feature space in a way that is pleasant for human interpreters. The paper does not provide such results.

Suggestion to improve the work:
- It would be useful to add some more explanation around radiomics term, feature extraction procedure and meaning of it, since this is not a common term for the machine learning community.
- Consider boosting the results and observation using a few more test data such as MIMIC and CheXpert that have the similar label space as NIH-14 and publicly being used in most of the related papers.
- The above mentioned datasets also can be used for evaluation of the out-of-domain generalization of the method which has not been discussed in the paper.
- Experimental setup and evaluation toward the SOTA can improve significantly. For the comparison purpose I would consider producing a statistical error range with either k-fold cross validation or boot-strapping for both CheXT and SOTAs. I would personally see a proper statistical comparison between a single best SOTA with the proposed method more valuable than having multiple models without any statistical validation.
- Adding interpretation around radiomic feature meaning, how the proposed method unlocked one or more of the 3 listed limitations of CNN methods that has been discussed in the Introduction.
- It is worth also discussing failure cases and the reasoning behind lower performance for some of the conditions such as effusion and pneumonia.

[Han'20] Han, Yan, Chongyan Chen, Liyan Tang, Mingquan Lin, Ajay Jaiswal, Song Wang, Ahmed Tewfik, George Shih, Ying Ding, and Yifan Peng. "Using Radiomics as Prior Knowledge for Thorax Disease Classification and Localization in Chest X-rays." arXiv preprint arXiv:2011.12506 (2020).

The evidence to be evaluated is:In CheXT, we extend their idea by treating image itself as one "view" and the radiomic feature extracted from the same image as another "view" (Figure 2). The two views are the jointly learned by interacting through cross-attention.

To resolve the "chicken-and-egg" dilemma in extracting reliable radiomic features without bounding boxes, we have designed a novel _Bootstrap Your Own Attention_ (**BYOA**) module, using feedbacks to learn region localization for extracting radiomic features.

The primary image branch uses a Progress-Sampling ViT (PS-ViT) (Yue et al., 2021) as its backbone. Unlike the vanilla ViT that splits images into fixed-size tokens, PS-ViT utilizes an iterative and progressive sampling strategy to locate discriminative regions and avoid over-partition object structures. We experimentally observed PS-ViT outperforms ViT and other variants in our framework because it generates higher-quality and more structure-aware attention maps, which are crucial for estimating the pathology localization during training.

In our framework, we aim to make the hidden features of the CLS similar to the radiomic features to learn the localization of pathologies in the chest X-rays.

Our main _roadblock_ is how to generate robust radiomic features without pathology localization. On one hand, radiomic features are dependent on and highly sensitive to the local image regions of interest, for which we have no bounding box annotation. On the other hand, image features would benefit from the guidance from radiomic features that encode important domain knowledge. The learning of image and radiomic features are fully entangled, forming a challenging "chicken-and-egg" loop. To address this issue, we design **BYOA** to constitute an end-to-end feedback loop that can bootstrap accurate pathology localization from image data, without any bounding box used for training (Figure 3).

Then we apply a threshold on the self-attention maps to generate bounding boxes for the extraction of radiomic features. The choice of the threshold will influence the quality of the radiomic features. Specifically, the threshold is designed as how much active pixels to keep in the generated attention map. The more active pixels we keep, the larger the generated bounding box is.

Given the original images and generated bounding boxes, we used the Pyradiomic tool to extract radiomic features (Van Griethuysen et al., 2017).

Our contrastive loss extends the normalized temperature scaled cross-entropy loss (NT-Xent). The difference is that we maximize agreement between two feature views extracted from different input formats, one the image and the other radiomics. Given an anchor chest X-ray in a minibatch, the positive sample will be its radiomic feature view, and the negative samples will be other chest X-rays (either image or radiomics). Since the CLS could be regarded as the representation of all other tokens, we only need to maximize the agreement between them.

Since for chest x-rays, there exists two labels, disease class labels and pathology bounding box annotations. In our case, we only have access to the disease labels, however, bounding box annotations are more important than the disease class labels for chest x-rays. Here, when we say "semi-supervised", we refer to the bounding box annotations, not disease class labels. For multi-task, one task is supervised disease classification. The other is unsupervised cross-view contrastive learning.

A significant difference between our method and existing baseline methods (Liu et al., 2019; Li et al., 2018) is that our method does not require any training data related to the bounding box while others use some percentage of these images for training. In our experiments, we followed the same protocol as in the studies of (Wang et al., 2017; Li et al., 2018), to shuffle our dataset (excluding images with Bounding Boxes annotations) into three subsets: 70% for training, 10% for validation, and 20% for testing.

During our experiments, we found that CheXT performs worse when very large bounding boxes are generated. This observation meets our expectations since radiomic features are very subtle to bounding boxes. Specifically, the extracted radiomic features are more robust if more focal and accurate bounding boxes are given.

Many prior works (Li et al., 2018; Liu et al., 2019) have used a fraction of ground truth (GT) bounding boxes for training and evaluated their system on the remaining. To ensure a robust evaluation, we do not use any GT for training.

For localization, we evaluated our detected regular rectangular regions against the annotated bounding boxes, using intersection over union ratio (IoU).

In this paper, we propose an end-to-end knowledge-guided cross-attention Transformer, named CheXT that can jointly model abnormality classification and localization without the supervision of localization annotation of chest X-rays. Our approach differs from previous studies in the choice of universal modeling transformer, the use of radiomic features as prior knowledge, and a feedback loop for image and radiomic features to mutually interact with each other.

Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, especially for the disease localization task, where our method can generate more accurate bounding boxes.

Input: Figure 2: Overview of our proposed CheXT. It contains two branches, the Image branch and the Radiomics branch, to process the image and radiomic features (generated by the BYOA module shown in Figure 3), respectively. The output tokens are then fused by an efficient module via cross attention of the CLS tokens. Finally, the output of two CLS tokens (Icls andRcls) are used for disease classification. We minimize the classification errors via a focal loss. Since we train with both labeled and unlabeled images, we leverage a contrastive learning strategy. Specifically, we generate the image view zi = gi(Icls) by a projection head gi. Similarily, we generate the radiomic view zr = gr(Rcls) by another projection head gr. We maximize the agreement between zi and zr via a contrastive loss (NT-Xent). Of note, the contrastive loss is only active during the training.

Input: Figure 3: Overview of our BYOA module. For the input chest X-rays, we look at the self-attention of the CLS token of the Image branch on the heads of the final output of the cross attention module. Then we apply a threshold (0.1) (which means we only keep the top 10% active pixels in the whole attention map) to generate the bounding boxes. Then with the generated bounding boxes, we use the Pyradiomic tool as the radiomic extractor to extract the radiomic features.

Input: Figure 1: General overview of our CheXT framework for Cardiopulmonary Disease Classification and Localization from chest X-rays. CheXT takes the chest X-ray image as the input and outputs a heatmap for pathology localization, based on which the bounding box could be obtained. Radiomic features are further extracted from the bounded region and are fed to predict the disease.

Input: Figure 5: Examples of visualization of localization on the test images. The attention maps are generated from the self-attention maps of the CLS token. The ground-truth bounding boxes are shown in blue. The left image in each pair is the localization result of ViT (Dosovitskiy et al., 2020). The right one is our localization results. All examples are positive for corresponding disease labels. Best viewed in color.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the provided claim, evidence, and reviewer comments, here is my assessment:

The claim that "such end-to-end loop can bootstrap accurate pathology localization from images without using human-annotated bounding boxes" is somewhat overstated. While the evidence demonstrates that CheXT can perform localization without bounding box annotations during training, several limitations exist:

1. The reviewer notes limited dataset testing (only NIH-14)
2. The evidence lacks comprehensive quantitative results comparing localization accuracy
3. The improvement over baseline methods is modest
4. The method's performance is sensitive to bounding box size generation
5. No statistical validation or error bounds are provided

The visual examples in Figure 5 show reasonable but not perfect localization. While the method works, claiming it achieves "accurate" localization without qualifying the limitations overstates its capabilities.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* Our approach achieve superior classification and localization results against several competitive baselines, on the NIH chest X-ray benchmark.

The evidence to be evaluated is:We conducted 3-fold cross-validation (Table 1). We compared CheXT with reference models, which have published state-of-the-art performance of disease classification on the NIH Chest X-ray dataset. We used Area under the Receiver Operating Characteristics (AUC) to estimate the performance of our model (FawcettTom, 2006). A higher AUC score implies a better classification model.

From the table, we can find that CheXT outperformed the baselines in the majority of diseases.

Compared to all baselines, CheXT achieves a mean AUC score of 0.839 using a dual-branch shallow Transformer across the 8 different diseases, which is 0.011 higher than the SOTA (uses DenseNet-121) (Rajpurkar et al., 2017) on disease classification. Specifically, our results significantly outperformed the best baseline models by 0.13, 0.09, and 0.02 in AUC for detecting Infiltration, Nodule, and Mass higher respectively.

The NIH Chest X-ray dataset has 880 images labeled by radiologists with the bounding box information. We have used this dataset to evaluate the performance of CheXT for abnormality localization.

Table 2 presents our evaluation results on all 880 images. We used (Wang et al., 2017) as our baseline to compare our localization results since it has the same experimental settings. For localization, we evaluated our detected regular rectangular regions against the annotated bounding boxes, using intersection over union ratio (IoU). Our localization results are only calculated for 880 images which have ground truth annotation for 8 diseases.

CheXT average performance across 8 diseases is significantly better than the baseline under all IoU thresholds. When the thresholds of IoU is set to 0.1, CheXT outperforms the baseline (Wang et al., 2017) in Cardiomegaly, infiltration, Mass, and Pneumonia. Even with higher thresholds, our model performs superior to baseline. For example, when evaluated at T(IoU) = 0.5, our "Cardiomegaly" accuracy is still 32%, while the reference model achieves only 18%.

More importantly, we also add the ViT as our additional baseline here, the results show that the radiomics branch and BYOA module can help the model learn more important regions, which also improves the model's interpretability.

Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, especially for the disease localization task, where our method can generate more accurate bounding boxes. \begin{table}
\begin{tabular}{c l c c c c c c c c c} \hline \hline
**T(IoU)** & Model & \multicolumn{2}{c}{Alelectasis} & \multicolumn{2}{c}{Cardiomegaly} & \multicolumn{2}{c}{Effusion} & \multicolumn{2}{c}{Infiltration} & \multicolumn{1}{c}{Mass} & \multicolumn{1}{c}{Nodule} & \multicolumn{1}{c}{Precmonia} & \multicolumn{1}{c}{Precumbotox} & \multicolumn{1}{c}{**Mean**} \\ \hline
0.1 & (Wang et al.,2017) & 0.69 & 0.94 & 0.66 & 0.71 & 0.40 & 0.14 & 0.63 & 0.38 & 0.569 \\  & ViT & 0.58 & 0.91 & 0.61 & 0.77 & 0.44 & 0.11 & 0.75 & 0.25 & 0.553 \\  & CheXT & 0.61 & 0.95 & 0.65 & 0.82 & 0.50 & 0.13 & 0.79 & 0.28 & **0.591** \\ \hline
0.2 & (Wang et al.,2017) & 0.47 & 0.68 & 0.45 & 0.48 & 0.26 & 0.05 & 0.35 & 0.23 & 0.371 \\  & ViT & 0.38 & 0.85 & 0.39 & 0.55 & 0.24 & 0.01 & 0.51 & 0.15 & 0.385 \\  & CheXT & 0.41 & 0.91 & 0.41 & 0.59 & 0.26 & 0.05 & 0.57 & 0.19 & **0.424** \\ \hline
0.3 & (Wang et al.,2017) & 0.24 & 0.46 & 0.30 & 0.28 & 0.15 & 0.04 & 0.17 & 0.13 & 0.221 \\  & ViT & 0.20 & 0.45 & 0.19 & 0.32 & 0.06 & 0.00 & 0.21 & 0.02 & 0.181 \\  & CheXT & 0.28 & 0.79 & 0.22 & 0.38 & 0.12 & 0.01 & 0.41 & 0.05 & **0.283** \\ \hline
0.4 & (Wang et al.,2017) & 0.09 & 0.28 & 0.20 & 0.12 & 0.07 & 0.01 & 0.08 & 0.07 & 0.115 \\  & ViT & 0.10 & 0.21 & 0.03 & 0.05 & 0.02 & 0.00 & 0.04 & 0.00 & 0.056 \\  & CheXT & 0.17 & 0.54 & 0.13 & 0.18 & 0.07 & 0.01 & 0.26 & 0.02 & **0.173** \\ \hline
0.5 & (Wang et al.,2017) & 0.05 & 0.18 & 0.11 & 0.07 & 0.01 & 0.01 & 0.03 & 0.03 & 0.061 \\  & ViT & 0.05 & 0.15 & 0.01 & 0.04 & 0.02 & 0.00 & 0.03 & 0.00 & 0.034 \\  & CheXT & 0.08 & 0.32 & 0.05 & 0.09 & 0.05 & 0.00 & 0.12 & 0.01 & **0.090** \\ \hline
0.6 & (Wang et al.,2017) & 0.02 & 0.08 & 0.05 & 0.02 & 0.00 & 0.01 & 0.02 & 0.03 & 0.029 \\  & ViT & 0.01 & 0.03 & 0.01 & 0.01 & 0.01 & 0.00 & 0.01 & 0.00 & 0.010 \\  & CheXT & 0.02 & 0.15 & 0.03 & 0.04 & 0.03 & 0.00 & 0.06 & 0.00 & **0.041** \\ \hline
0.7 & (Wang et al.,2017) & 0.01 & 0.03 & 0.02 & 0.00 & 0.00 & 0.00 & 0.01 & 0.02 & 0.011 \\  & ViT & 0.00 & 0.00 & 0.00 & 0.01 & 0.00 & 0.00 & 0.00 & 0.00 & 0.001 \\  & CheXT & 0.01 & 0.04 & 0.01 & 0.02 & 0.01 & 0.00 & 0.03 & 0.00 & **0.015** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Disease localization under varying IoU on the NIH Chest X-ray dataset. Please note that since our model doesn’t use any ground truth bounding box information, to fairly evaluate the performance of our model, we only consider the previous methods’ results under the same setting,

Input: Table 1: Comparison with the baseline models for AUC of each class and average AUC. For each column, bold values denote the best results.

Input: Table 2: Disease localization under varying IoU on the NIH Chest X-ray dataset. Please note that since our model doesn’t use any ground truth bounding box information, to fairly evaluate the performance of our model, we only consider the previous methods’ results under the same setting,

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim "Our approach achieve superior classification and localization results against several competitive baselines, on the NIH chest X-ray benchmark" appears to be well-stated based on the data presented:

For classification, CheXT achieved a mean AUC of 0.839, which is 0.011 higher than the previous SOTA. The improvements were particularly notable for specific conditions (Infiltration +0.13, Nodule +0.09).

For localization, Table 2 shows CheXT consistently outperformed the baseline (Wang et al. 2017) across all IoU thresholds, with higher mean scores (e.g., 0.591 vs 0.569 at IoU=0.1). The evidence directly supports both aspects of the claim without exaggeration.

<score>1</score>