Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Peaks2Image succesfully decodes 65 of them on thousands of brain images from the NeuroVault database, without using any supervision from NeuroVault samples during training.

The review comment to be evaluated is:

This paper contains several controversial claims.

The paper uses Nilearn to estimate peaks and then uses these peaks to learn a model from data. However, there is no evaluation of the error associated with the selection of these peaks. Further, what evidence shows that you are more likely to see neural responses in these peaks than examining the whole brain data?

Another concern is the proposed formulation. As an example, what is the definition of the function f() in Equation 1? To learn f*, what constraints are considered? The same question can be asked for D in Equation 2. Due to the fact that D without any constraint may increase the chance of overfitting.

Empirical studies are another concern. The proposed method only compares with some classic machine learning approaches, such as Kernel Ridge or Space Gaussian. There are several groups of state-of-the-art techniques that aim at similar types of analysis and should be compared with the proposed approach, e.g., HTFA (Manning 2018), BRSA (Cai 2016), and DRD (Wu 2019).

In addition to the aforementioned concerns, the proposed approach should clarify some related concepts for multi-subject and multi-resource analysis. It seems that this study uses multi-subject data. However, I could not find anywhere that discusses how the author(s) functionally align brain responses from different subjects. Alternatively, Neuro Vault is a multi-resource repository where data are provided from different machines and environments. I also did not find any information on how the proposed method deals with bach effects to learn a model from multi-site data. You may provide additional information in a preprocessing/preparation section.

The evidence to be evaluated is:Thanks to the rise of large-scale databases of fMRI brain images such as NeuroVault (Gorgolewski et al., 2015) or OpenNeuro (Gorgolewski et al., 2017), decoding can now be performed across multiple studies (Mensch et al., 2017; Walters et al., 2022). Nonetheless, annotations associated with those images are often of low quality. Automatic strategies to improve this labeling are necessary to benefit properly from the scale of the data (Poldrack and Yarkoni, 2016). Menuet et al. (2022) leverages the Cognitive Atlas (Poldrack et al., 2011) to improve the label quality of NeuroVault images, enabling the decoding of a large set of concepts.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted. From a given set of \(l\) peaks \(\mathbf{P_{y}}\in\mathbb{R}^{3\times l}\) extracted from an image \(\mathbf{y}\in\mathbb{R}^{m}\) with \(m\) voxels, we learn the transformation \(f:\mathbb{R}^{3\times l}\mapsto\mathbb{R}^{m}\) by minimization of the \(\mathbf{L_{2}}\) error over the original image: \[f^{*}=\operatorname*{arg\,min}_{f}\mathbf{E_{y}}\|f(\mathbf{P_{y}})-\mathbf{ y}\|_{2}^{2} \tag{1}\] To solve the minimization problem in eq 1, we leverage a set of statistical maps taken from NeuroVault, an online repository of fMRI maps, from which we extract peak coordinates. This leads to a training set of peaks and the corresponding brain images to be reconstructed. Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

To train this reconstruction process, we use around 6k images from the #4438 NeuroVault collection ([https://neurovault.org/collections/4438](https://neurovault.org/collections/4438)).

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting.

We evaluate the decoding performance on the NeuroVault database (#4438 excluded). We extract the term frequency-inverse document frequency (tf-idf) (Salton and Buckley, 1988) from the studies' text. The tf-idf reflects the relevance of a specific term to a study by giving higher power to terms that occurs repetitively in few studies. Based on it, we assign labels to studies, by setting a threshold at the 95th percentile of the tf-idf. This ensures that the few studies that are most relevant to each term get positive labels.

The model is trained from the sole neuroscientific studies, without supervision from any NeuroVault sample or its labels.

We leverage the NeuroVault database.

We take collection #4438, without labels (6k maps), to train the reconstruction task. We consider the remaining 43k brain maps from 2376 different collections to evaluate the decoding performances.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We validate the relevance of the reconstructions by decoding NeuroVault images using reconstructed neuroscientific studies as training set.

We exclude collection #4438 that has been used as training set in the reconstruction task, and evaluate the decoding of all remaining images from NeuroVault.

On terms that are shared with NeuroSynth (Yarkoni et al., 2011), we show that Peaks2Image outperforms the former, while largely expanding the number of cognitive processes decoded.

Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions. Most importantly, the decoding performance is reached without any labeling requirements on the images side, avoiding a time-consuming and error-prone task.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

In figure 8, we report the per-label decoding performance along with the variance of the results. Most terms are decoded above chance.

Input: Figure 8: Zero-shot decoding performance on NeuroVault: we use Peaks2Image to produce dense brain images for neuroscientific studies. We train a decoder from the generated data on a broad set of cognitive terms. We evaluate the decoding performance on NeuroVault data. Peaks2Image succesfully decodes a large part of those terms in a zero-shot setting. Peaks2Image could extend to any word from the studies’ vocabulary, but could be evaluated on the NeuroVault annotations only.

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 3: Zero-shot decoding performance on NeuroVault: evaluation is performed on all NeuroVault except collection #4438. We exclude categories with less than 50 positive examples for stability. Peaks2Image decodes better than chance 65 out of 81 cognitive processes over 43k images (AUC > 0.55).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim "Peaks2Image succesfully decodes 65 of them on thousands of brain images from the NeuroVault database, without using any supervision from NeuroVault samples during training."

Summary: Based on the evidence, this claim is moderately overstated. While Figure 3 shows that Peaks2Image does decode "65 out of 81 cognitive processes over 43k images (AUC > 0.55)," there are several concerns:

1. The threshold for "successful" decoding is quite low (AUC > 0.55)
2. The reviewer raises valid concerns about:
   - Lack of comparison with state-of-the-art techniques
   - Missing details about multi-subject alignment
   - Potential overfitting issues
3. The claim about "no supervision" is partially accurate but oversimplifies the use of NeuroVault collection #4438 for training.

<score>3</score>

Justification:

1. Evidence Supporting Claim:
- Figure 3 confirms "65 out of 81 cognitive processes" are decoded above chance
- The paper states they used collection #4438 for training without labels
- Testing was done on 43k remaining images

2. Evidence Against/Limitations:
- The success criterion (AUC > 0.55) is barely above chance (0.50)
- The reviewer points out missing comparisons with modern techniques like HTFA, BRSA, and DRD
- The reviewer highlights concerns about peak selection error and lack of functional alignment details
- Figure 8 shows many terms have relatively low decoding performance

3. Exaggeration Elements:
- The word "successfully" overstates the modest performance threshold
- The claim doesn't acknowledge the limitations in methodology identified by the reviewer
- The "without supervision" part doesn't clearly acknowledge using NeuroVault data (#4438) for training

The score of 3 reflects that while the basic facts are correct, the claim's wording overstates the strength and robustness of the results while understating important methodological limitations.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Peaks2Image succesfully decodes 65 of them on thousands of brain images from the NeuroVault database, without using any supervision from NeuroVault samples during training.

The review comment to be evaluated is:

Strength:
1.This paper is well-written and easy to read. The definition of tasks and problems are very clear.
2.For a decoding task, the idea that fully utilize the rich descriptions of the studied cognitive tasks in scientific report is novel.
Weaknesses
1.My major concern is the necessity of image reconstruction from peaks. As illustrated in the paper, the authors train a reconstruction network to convert the peak coordinates into images. In my opinion, this ill-posed problem is too hard to be reasonable. Reconstructing the fMRI from only a few peak coordinates (less than 100 according to the paper) seems impossible to recover the details of images. Thus, the reconstruction is unnecessary. If the peak coordinates contain enough information for the following decoding tasks, reconstruction would introduce inevitable noise. If the peak coordinates are not enough, reconstruction still can’t help the decoding since extra information comes from the ‘guessing’ of the trained network. Can the authors give more explanation about this?
2.Also, the experiments do not verify that image reconstruction can help in improving the decoding performance. The ablation study should compare with model using peak coordinates as input. So, I am wondering about the performance of model that uses peak coordinates as input.
3.Experimental results are all shown in figures, which is good for analysis. However, giving some extra tables with exact values can be better.

The evidence to be evaluated is:Thanks to the rise of large-scale databases of fMRI brain images such as NeuroVault (Gorgolewski et al., 2015) or OpenNeuro (Gorgolewski et al., 2017), decoding can now be performed across multiple studies (Mensch et al., 2017; Walters et al., 2022). Nonetheless, annotations associated with those images are often of low quality. Automatic strategies to improve this labeling are necessary to benefit properly from the scale of the data (Poldrack and Yarkoni, 2016). Menuet et al. (2022) leverages the Cognitive Atlas (Poldrack et al., 2011) to improve the label quality of NeuroVault images, enabling the decoding of a large set of concepts.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted. From a given set of \(l\) peaks \(\mathbf{P_{y}}\in\mathbb{R}^{3\times l}\) extracted from an image \(\mathbf{y}\in\mathbb{R}^{m}\) with \(m\) voxels, we learn the transformation \(f:\mathbb{R}^{3\times l}\mapsto\mathbb{R}^{m}\) by minimization of the \(\mathbf{L_{2}}\) error over the original image: \[f^{*}=\operatorname*{arg\,min}_{f}\mathbf{E_{y}}\|f(\mathbf{P_{y}})-\mathbf{ y}\|_{2}^{2} \tag{1}\] To solve the minimization problem in eq 1, we leverage a set of statistical maps taken from NeuroVault, an online repository of fMRI maps, from which we extract peak coordinates. This leads to a training set of peaks and the corresponding brain images to be reconstructed. Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

To train this reconstruction process, we use around 6k images from the #4438 NeuroVault collection ([https://neurovault.org/collections/4438](https://neurovault.org/collections/4438)).

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting.

We evaluate the decoding performance on the NeuroVault database (#4438 excluded). We extract the term frequency-inverse document frequency (tf-idf) (Salton and Buckley, 1988) from the studies' text. The tf-idf reflects the relevance of a specific term to a study by giving higher power to terms that occurs repetitively in few studies. Based on it, we assign labels to studies, by setting a threshold at the 95th percentile of the tf-idf. This ensures that the few studies that are most relevant to each term get positive labels.

The model is trained from the sole neuroscientific studies, without supervision from any NeuroVault sample or its labels.

We leverage the NeuroVault database.

We take collection #4438, without labels (6k maps), to train the reconstruction task. We consider the remaining 43k brain maps from 2376 different collections to evaluate the decoding performances.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We validate the relevance of the reconstructions by decoding NeuroVault images using reconstructed neuroscientific studies as training set.

We exclude collection #4438 that has been used as training set in the reconstruction task, and evaluate the decoding of all remaining images from NeuroVault.

On terms that are shared with NeuroSynth (Yarkoni et al., 2011), we show that Peaks2Image outperforms the former, while largely expanding the number of cognitive processes decoded.

Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions. Most importantly, the decoding performance is reached without any labeling requirements on the images side, avoiding a time-consuming and error-prone task.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

In figure 8, we report the per-label decoding performance along with the variance of the results. Most terms are decoded above chance.

Input: Figure 8: Zero-shot decoding performance on NeuroVault: we use Peaks2Image to produce dense brain images for neuroscientific studies. We train a decoder from the generated data on a broad set of cognitive terms. We evaluate the decoding performance on NeuroVault data. Peaks2Image succesfully decodes a large part of those terms in a zero-shot setting. Peaks2Image could extend to any word from the studies’ vocabulary, but could be evaluated on the NeuroVault annotations only.

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 3: Zero-shot decoding performance on NeuroVault: evaluation is performed on all NeuroVault except collection #4438. We exclude categories with less than 50 positive examples for stability. Peaks2Image decodes better than chance 65 out of 81 cognitive processes over 43k images (AUC > 0.55).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim "Peaks2Image successfully decodes 65 of them on thousands of brain images from the NeuroVault database, without using any supervision from NeuroVault samples during training" appears to be well-stated. The evidence shows:

- Figure 3 explicitly shows "Peaks2Image decodes better than chance 65 out of 81 cognitive processes over 43k images (AUC > 0.55)"
- The model was trained using only collection #4438 (6k maps) and evaluated on the remaining 43k brain maps
- The review comments don't dispute these specific performance claims

The claim accurately reflects the evidence without exaggeration, focusing on the core result that is directly supported by the data.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

While the evaluation has been performed on a limited set of terms, Peaks2Image can decode in a zero-shot setting any term from its vocabulary.

The evidence to be evaluated is:Ngo et al. (2021) leveraged language models (Beltagy et al., 2019) to capture term relationships semantically, leading to the encoding of any query.

Menuet et al. (2022) leverages the Cognitive Atlas (Poldrack et al., 2011) to improve the label quality of NeuroVault images, enabling the decoding of a large set of concepts.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

The figure is divided between the learning of the reconstruction process from peaks (A), the application of the reconstruction to neuroscientific studies (B), and the decoding of brain images with a model trained from those reconstructions in a zero-shot setting.

Peaks2Image outputs images from any set of positions.

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting.

In the following experiments, we predict only \(l{=}81\) terms from the 6308 in the tf-idf vocabulary. Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

It also transforms each study text into tf-idf features across a vocabulary of 6308 terms.

Among the 240 tags from the NeuroVault test collections, 189 are present in the vocabulary. We focus on those 189 tags without adding any additional mappings between the left-out tags and words from the vocabulary. Additionally, we filter out terms that contain less than 50 positive examples in the test set to prevent any non-significant effects, keeping 81 terms to decode.

The limitation of terms is induced by the evaluation against NeuroVault, but Peaks2Image could extend the prediction to any word included in the tf-idf vocabulary.

We noted an effect of NNoD being able to decode some terms unseen during training.

Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim that "Peaks2Image can decode in a zero-shot setting any term from its vocabulary" to be somewhat overstated. While the evidence shows Peaks2Image has zero-shot capabilities, there are important limitations:

1. The actual evaluation was limited to 81 terms out of 6308 vocabulary terms
2. The system was only tested on terms with >50 positive examples
3. Figure 5 shows variable performance across different cognitive terms
4. No comprehensive validation across the full vocabulary is demonstrated

While the system shows promise for broader application, claiming it can decode "any term" is not fully supported by the presented evidence.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

While the evaluation has been performed on a limited set of terms, Peaks2Image can decode in a zero-shot setting any term from its vocabulary.

The review comment to be evaluated is:

Strengths
1. The authors proposed an interesting workflow. If I understand it correctly, the authors are using the following priors: (1) fMRI brain maps is tightly correlated to the underlying cognitive states, and (2) distribution of peak locations in fMRI brain maps is highly correlated to the overall global pattern in fMRI brain maps. With these priors, the authors proposed to train a neural network model to predict the global fMRI pattern from sparse locations of peaks, and with that trained model they synthesized a paired dataset of (cognitive descriptions, global fMRI patterns) using the (cognitive descriptions, sparse locations of peaks) from the neuroscientific literatures. Finally, they trained a separate decoder with the (cognitive descriptions, global fMRI patterns) pairs to decode the former from the latter. The decoder is the ultimate piece of mastery that can describe the underlying cognitive states of an arbitrary fMRI image.

Weaknesses
1. In my humble opinion, the abstract may need to be better articulated. I was very disoriented and confused by the abstract until I finished reading the introduction section. Specifically, I was confused by (1) what the two complementary sources of data are, (2) why they are complementary, and (3) what is the proposed method trying to accomplish. For (1) and (2), even though the two sources of data are both explicitly mentioned, I did not have enough context to parse their relationship. In contrary, the introduction section has a very natural flow and is painless to follow.
2. It is a bit misleading that when the authors mentioned the word “image” they did not necessarily mean “image”, but rather the “1x1024 vector representation in the DeFuMo space of the image”. I believe it might be helpful to provide a clarifying statement somewhere early in the paper to avoid potential misunderstanding.
3. Figure 1 can be further polished. (1) Figure 1A did not clearly show what data format is provided and what are derived. It might be better if that information is illustrated more explicitly. (2) A lot of things are represented as “blue thin rectangle”, yet some of them are modules/operations while others are entities. It may be better to replace some of them. For example, “Reconstruction” and “decoder” can be replaced by sketches of the neural networks, and “Labels” can be replaced by some clipart object, etc. (3) The caption mentioned that in Figure 1B, the brain map is reconstructed using the trained model, so it would be better to show the sketch of model in an appropriate place and indicate that its weights are frozen.
4. The reconstruction performance (Section 5.1, Figure 2) is quite poor. a correlation of 0.3 does not seem too promising.

Minor Things
1. Is there some typo in the following sentence? Page 5, 3rd paragraph. “When the percentile corresponds to a To avoid selecting studies with a null tf-idf value, we replace the percentile threshold with a strictly positive tf-idf value criteria.”

The evidence to be evaluated is:Ngo et al. (2021) leveraged language models (Beltagy et al., 2019) to capture term relationships semantically, leading to the encoding of any query.

Menuet et al. (2022) leverages the Cognitive Atlas (Poldrack et al., 2011) to improve the label quality of NeuroVault images, enabling the decoding of a large set of concepts.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

The figure is divided between the learning of the reconstruction process from peaks (A), the application of the reconstruction to neuroscientific studies (B), and the decoding of brain images with a model trained from those reconstructions in a zero-shot setting.

Peaks2Image outputs images from any set of positions.

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting.

In the following experiments, we predict only \(l{=}81\) terms from the 6308 in the tf-idf vocabulary. Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

It also transforms each study text into tf-idf features across a vocabulary of 6308 terms.

Among the 240 tags from the NeuroVault test collections, 189 are present in the vocabulary. We focus on those 189 tags without adding any additional mappings between the left-out tags and words from the vocabulary. Additionally, we filter out terms that contain less than 50 positive examples in the test set to prevent any non-significant effects, keeping 81 terms to decode.

The limitation of terms is induced by the evaluation against NeuroVault, but Peaks2Image could extend the prediction to any word included in the tf-idf vocabulary.

We noted an effect of NNoD being able to decode some terms unseen during training.

Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim is moderately overstated. While Peaks2Image can theoretically decode terms from its 6308-word vocabulary, several key limitations exist:

1. The evidence shows evaluation was only done on 81 terms out of 189 possible tags, with terms requiring >50 positive examples.
2. Figure 5 reveals variable performance across different cognitive terms, with some showing poor decoding accuracy.
3. The reviewer notes "quite poor" reconstruction performance with correlation of only 0.3.
4. The authors themselves acknowledge limitations due to lack of extensively annotated brain images.

While the model has zero-shot capabilities, claiming it can decode "any term" overstates the demonstrated performance and reliability.

<score>4</score>

Justification:
- The claim suggests universal capability ("can decode in a zero-shot setting any term") but evidence shows limited testing (81/6308 terms)
- Quote: "While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts"
- Performance varies significantly across terms (Fig 5)
- Reviewer highlights poor reconstruction performance
- Insufficient evidence demonstrating reliable decoding across the full vocabulary


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

While the evaluation has been performed on a limited set of terms, Peaks2Image can decode in a zero-shot setting any term from its vocabulary.

The review comment to be evaluated is:

This paper contains several controversial claims.

The paper uses Nilearn to estimate peaks and then uses these peaks to learn a model from data. However, there is no evaluation of the error associated with the selection of these peaks. Further, what evidence shows that you are more likely to see neural responses in these peaks than examining the whole brain data?

Another concern is the proposed formulation. As an example, what is the definition of the function f() in Equation 1? To learn f*, what constraints are considered? The same question can be asked for D in Equation 2. Due to the fact that D without any constraint may increase the chance of overfitting.

Empirical studies are another concern. The proposed method only compares with some classic machine learning approaches, such as Kernel Ridge or Space Gaussian. There are several groups of state-of-the-art techniques that aim at similar types of analysis and should be compared with the proposed approach, e.g., HTFA (Manning 2018), BRSA (Cai 2016), and DRD (Wu 2019).

In addition to the aforementioned concerns, the proposed approach should clarify some related concepts for multi-subject and multi-resource analysis. It seems that this study uses multi-subject data. However, I could not find anywhere that discusses how the author(s) functionally align brain responses from different subjects. Alternatively, Neuro Vault is a multi-resource repository where data are provided from different machines and environments. I also did not find any information on how the proposed method deals with bach effects to learn a model from multi-site data. You may provide additional information in a preprocessing/preparation section.

The evidence to be evaluated is:Ngo et al. (2021) leveraged language models (Beltagy et al., 2019) to capture term relationships semantically, leading to the encoding of any query.

Menuet et al. (2022) leverages the Cognitive Atlas (Poldrack et al., 2011) to improve the label quality of NeuroVault images, enabling the decoding of a large set of concepts.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

The figure is divided between the learning of the reconstruction process from peaks (A), the application of the reconstruction to neuroscientific studies (B), and the decoding of brain images with a model trained from those reconstructions in a zero-shot setting.

Peaks2Image outputs images from any set of positions.

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting.

In the following experiments, we predict only \(l{=}81\) terms from the 6308 in the tf-idf vocabulary. Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

It also transforms each study text into tf-idf features across a vocabulary of 6308 terms.

Among the 240 tags from the NeuroVault test collections, 189 are present in the vocabulary. We focus on those 189 tags without adding any additional mappings between the left-out tags and words from the vocabulary. Additionally, we filter out terms that contain less than 50 positive examples in the test set to prevent any non-significant effects, keeping 81 terms to decode.

The limitation of terms is induced by the evaluation against NeuroVault, but Peaks2Image could extend the prediction to any word included in the tf-idf vocabulary.

We noted an effect of NNoD being able to decode some terms unseen during training.

Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find this claim significantly overstated. While the evidence shows Peaks2Image can decode terms beyond its training set, there are important limitations:

1. The evaluation was limited to 81 terms out of 6308 vocabulary terms
2. Only terms with >50 positive examples were tested
3. Performance varies significantly across terms (Figure 5)
4. The reviewer raises valid concerns about lack of validation for peak selection and multi-subject alignment
5. No experimental validation is shown for the full vocabulary claim

The evidence demonstrates zero-shot capability for some terms, but does not support the broad claim of decoding "any term from its vocabulary." The limitations are acknowledged in the paper but not in this claim.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

While the evaluation has been performed on a limited set of terms, Peaks2Image can decode in a zero-shot setting any term from its vocabulary.

The review comment to be evaluated is:

Strength:
1.This paper is well-written and easy to read. The definition of tasks and problems are very clear.
2.For a decoding task, the idea that fully utilize the rich descriptions of the studied cognitive tasks in scientific report is novel.
Weaknesses
1.My major concern is the necessity of image reconstruction from peaks. As illustrated in the paper, the authors train a reconstruction network to convert the peak coordinates into images. In my opinion, this ill-posed problem is too hard to be reasonable. Reconstructing the fMRI from only a few peak coordinates (less than 100 according to the paper) seems impossible to recover the details of images. Thus, the reconstruction is unnecessary. If the peak coordinates contain enough information for the following decoding tasks, reconstruction would introduce inevitable noise. If the peak coordinates are not enough, reconstruction still can’t help the decoding since extra information comes from the ‘guessing’ of the trained network. Can the authors give more explanation about this?
2.Also, the experiments do not verify that image reconstruction can help in improving the decoding performance. The ablation study should compare with model using peak coordinates as input. So, I am wondering about the performance of model that uses peak coordinates as input.
3.Experimental results are all shown in figures, which is good for analysis. However, giving some extra tables with exact values can be better.

The evidence to be evaluated is:Ngo et al. (2021) leveraged language models (Beltagy et al., 2019) to capture term relationships semantically, leading to the encoding of any query.

Menuet et al. (2022) leverages the Cognitive Atlas (Poldrack et al., 2011) to improve the label quality of NeuroVault images, enabling the decoding of a large set of concepts.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

The figure is divided between the learning of the reconstruction process from peaks (A), the application of the reconstruction to neuroscientific studies (B), and the decoding of brain images with a model trained from those reconstructions in a zero-shot setting.

Peaks2Image outputs images from any set of positions.

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting.

In the following experiments, we predict only \(l{=}81\) terms from the 6308 in the tf-idf vocabulary. Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

It also transforms each study text into tf-idf features across a vocabulary of 6308 terms.

Among the 240 tags from the NeuroVault test collections, 189 are present in the vocabulary. We focus on those 189 tags without adding any additional mappings between the left-out tags and words from the vocabulary. Additionally, we filter out terms that contain less than 50 positive examples in the test set to prevent any non-significant effects, keeping 81 terms to decode.

The limitation of terms is induced by the evaluation against NeuroVault, but Peaks2Image could extend the prediction to any word included in the tf-idf vocabulary.

We noted an effect of NNoD being able to decode some terms unseen during training.

Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that "Peaks2Image can decode in a zero-shot setting any term from its vocabulary" appears overstated. While the system has a vocabulary of 6,308 terms, the actual evaluation was limited to only 81 terms. The evidence shows significant limitations:

1. The evaluation was restricted due to needing sufficient positive examples (>50) in test data
2. Performance varies significantly across terms (as shown in Figure 5)
3. A reviewer raised valid concerns about the reconstruction process from limited peak coordinates
4. The paper itself acknowledges evaluation was "reduced to a limited number of concepts"

While the system shows promise for zero-shot decoding, claiming it can decode "any term" is not supported by the current evidence.

<score>4</score>

Justification:
- The claim uses absolute language ("any term") without acknowledging limitations
- Evidence shows testing on only 81/6308 terms (~1.3% of vocabulary)
- The paper states "While Peaks2Image could extend to any word..." but doesn't provide evidence supporting this capability
- Reviewer highlights fundamental concerns about reconstruction from limited peak coordinates
- Figure 5 shows variable performance across different terms, suggesting inconsistent capabilities


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this work, we propose Peaks2Image, a neural-network approach to reconstruct continuous spatial representations of brain activity from peak activation tables.

The evidence to be evaluated is:Although there has been interest in enriching brain images to increase the amount of data (Zhuang et al., 2019), to generate or reconstruct fMRI images (Manning et al., 2018; Cai et al., 2016), there has been limited interest in generating reconstructions of statistical brain maps from peaks (Gorgolewski et al., 2019), probably due to the daunting data dimensionality. In particular, we found no attempt at merging rich textual information from neuroscientific studies with the dense spatial information of brain images. We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted. From a given set of \(l\) peaks \(\mathbf{P_{y}}\in\mathbb{R}^{3\times l}\) extracted from an image \(\mathbf{y}\in\mathbb{R}^{m}\) with \(m\) voxels, we learn the transformation \(f:\mathbb{R}^{3\times l}\mapsto\mathbb{R}^{m}\) by minimization of the \(\mathbf{L_{2}}\) error over the original image: \[f^{*}=\operatorname*{arg\,min}_{f}\mathbf{E_{y}}\|f(\mathbf{P_{y}})-\mathbf{ y}\|_{2}^{2} \tag{1}\] To solve the minimization problem in eq 1, we leverage a set of statistical maps taken from NeuroVault, an online repository of fMRI maps, from which we extract peak coordinates. This leads to a training set of peaks and the corresponding brain images to be reconstructed. Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

Instead of leveraging the peak coordinates directly as a sets problem (Zaheer et al., 2017), Peaks2Image builds an internal sparse Gaussian representation similar to the literature as by Laird et al. (2005); Dockes et al. (2020); Ngo et al. (2021).

Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information. In the rest of the paper, the term _brain image_ refers to its 1024-dimensional DiFuMo representation. Mapping voxels \(\mathbf{x}\in\mathbb{R}^{m}\) to coefficients \(\alpha\in R^{k}\) along DiFuMo dictionary \(\mathbf{D}\in\mathbb{R}^{m\times k}\) of dimension \(k\) is a mere linear regression problem following equation 2. All the following steps are performed in the \(k\)-dimensional DiFuMo space.

The DiFuMo-encoded sparse Gaussian representation is turned into a dense representation with a 3-layer neural network where the input and output are DiFuMo coefficients.

To train this reconstruction process, we use around 6k images from the #4438 NeuroVault collection ([https://neurovault.org/collections/4438](https://neurovault.org/collections/4438)). The reconstructed images are obtained from the predicted DiFuMo components by linear combinations of the dictionary components: \(\hat{\mathbf{y}}=\mathbf{D}f^{*}(\alpha(\mathbf{x}_{\mathbf{y}}))\). Peaks2Image outputs images from any set of positions.

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting. Indeed, the ability to decode cognitive concepts from the produced database ensures its validity and circumvents the biases inherent to image reconstruction quality metrics, such as the confounding effect of smoothness.

We leverage a 3-layer dense neural network to predict the set of binary labels \(\mathbf{L}=\{0,1\}_{1}\times\cdots\times\{0,1\}_{l}\), \(l\) denoting the total number of classes, from the input image \(\mathbf{X}\in\mathbb{R}^{d}\) compressed in DiFuMo space.

Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost. We leverage the NeuroVault database.

We take collection #4438, without labels (6k maps), to train the reconstruction task.

We use NeuroQuery Data Collection (Dockes et al., 2020) to fetch around 13k neuroscientific studies from PubMed Central. The tool helps us parse the activation tables of those studies, totalling 400k peaks. It also transforms each study text into tf-idf features across a vocabulary of 6308 terms. For validation, we restrict the analysis to shared terms between the tags of NeuroVault images and the vocabulary from the tf-idf. Among the 240 tags from the NeuroVault test collections, 189 are present in the vocabulary. We focus on those 189 tags without adding any additional mappings between the left-out tags and words from the vocabulary. Additionally, we filter out terms that contain less than 50 positive examples in the test set to prevent any non-significant effects, keeping 81 terms to decode.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020). Similarly to the NeuroVault images, we extract the DiFuMo representations of the sparse Gaussian maps using Nilearn.

The image reconstruction model is an MLP built from 3 dense layers with 1024 units and sigmoid activation implemented using PyTorch (Paszke et al., 2019). It is trained for 50 epochs using Adam optimizer and a learning rate of 1e-3. The loss is the mean-squared error (MSE) over the DiFuMo components. We select the best set of weights over the validation set, which is composed of 20% of collection #4438.

The decoder model is an MLP composed of one hidden layer of 300 units and ReLU activation.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We validate the relevance of the reconstructions by decoding NeuroVault images using reconstructed neuroscientific studies as training set. The problem is a multi-label classification, which we evaluate by computing the ROC AUC over each label. We exclude collection #4438 that has been used as training set in the reconstruction task, and evaluate the decoding of all remaining images from NeuroVault. We apply label enrichment to the original image labels. This includes regular expressions to map different terminologies for a same concept, and a hierarchy-based inference of labels based on the Cognitive Atlas ontology. For a given term, we assign parent terms from the ontology used in (Menuet et al., 2022). This results in 43k images, totalling around 200 different cognitive processes.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task. We observe a statistically significant gain of 0.1 on the mean AUC across labels between Peaks2Image and sparse Gaussian representations.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases. Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding. In particular, neuroscientific studies could become a challenging application for recent methods in text-to-image representation learning such as Radford et al. (2021).

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 2: Correlation between original images and reconstructions: We evaluate the reconstruction for different models (Linear regression, kernel Ridge regression, sparse Gaussian representation and Peaks2Image) using correlation in voxel space. Evaluation is performed on collection 4337 from NeuroVault (18k images).

Input: Figure 3: Zero-shot decoding performance on NeuroVault: evaluation is performed on all NeuroVault except collection #4438. We exclude categories with less than 50 positive examples for stability. Peaks2Image decodes better than chance 65 out of 81 cognitive processes over 43k images (AUC > 0.55).

Input: Figure 4: Distribution of relative AUC performance across tasks of Peaks2Image compared to other representations: We show the relative AUC gain of Peaks2Image for each task, along with the difference of mean AUC across labels vs different methods. We perform a t-test to probe the statistical significance of the difference between Peaks2Image and other methods. We also display the average AUC difference (∆). For NeuroSynth, we compute the difference on a subset of shared terms.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim and evidence, this appears to be a well-stated claim. The authors clearly describe Peaks2Image as a neural network approach to reconstruct continuous spatial brain activity from peak activation tables. The evidence thoroughly documents the methodology, including the mathematical framework, training process, and validation through both reconstruction quality metrics and downstream decoding tasks. The extensive results shown in multiple figures demonstrate that the approach works as claimed, with clear performance advantages over baseline methods. The authors don't overstate capabilities and acknowledge limitations.

<score>1</score>

Justification:
1. The evidence directly supports the core claim through:
- Detailed mathematical formulation showing how peak coordinates are transformed into brain images
- Comprehensive validation using 6k NeuroVault images for training and 43k for testing
- Quantitative performance metrics showing improved reconstruction vs baselines (Fig 2)
- Successful application to decoding 65 out of 81 cognitive processes (Fig 3)
- Visual examples demonstrating reconstruction quality (Fig 6-7)

2. The claim is appropriately scoped:
- Focuses specifically on reconstruction capability
- Does not make unsupported generalizations
- Matches experimental evidence
- No exaggerated language or unsupported statements about capabilities

3. Results clearly demonstrate the stated functionality through multiple evaluation approaches and detailed error analysis.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this work, we propose Peaks2Image, a neural-network approach to reconstruct continuous spatial representations of brain activity from peak activation tables.

The review comment to be evaluated is:

Strengths
1. The authors proposed an interesting workflow. If I understand it correctly, the authors are using the following priors: (1) fMRI brain maps is tightly correlated to the underlying cognitive states, and (2) distribution of peak locations in fMRI brain maps is highly correlated to the overall global pattern in fMRI brain maps. With these priors, the authors proposed to train a neural network model to predict the global fMRI pattern from sparse locations of peaks, and with that trained model they synthesized a paired dataset of (cognitive descriptions, global fMRI patterns) using the (cognitive descriptions, sparse locations of peaks) from the neuroscientific literatures. Finally, they trained a separate decoder with the (cognitive descriptions, global fMRI patterns) pairs to decode the former from the latter. The decoder is the ultimate piece of mastery that can describe the underlying cognitive states of an arbitrary fMRI image.

Weaknesses
1. In my humble opinion, the abstract may need to be better articulated. I was very disoriented and confused by the abstract until I finished reading the introduction section. Specifically, I was confused by (1) what the two complementary sources of data are, (2) why they are complementary, and (3) what is the proposed method trying to accomplish. For (1) and (2), even though the two sources of data are both explicitly mentioned, I did not have enough context to parse their relationship. In contrary, the introduction section has a very natural flow and is painless to follow.
2. It is a bit misleading that when the authors mentioned the word “image” they did not necessarily mean “image”, but rather the “1x1024 vector representation in the DeFuMo space of the image”. I believe it might be helpful to provide a clarifying statement somewhere early in the paper to avoid potential misunderstanding.
3. Figure 1 can be further polished. (1) Figure 1A did not clearly show what data format is provided and what are derived. It might be better if that information is illustrated more explicitly. (2) A lot of things are represented as “blue thin rectangle”, yet some of them are modules/operations while others are entities. It may be better to replace some of them. For example, “Reconstruction” and “decoder” can be replaced by sketches of the neural networks, and “Labels” can be replaced by some clipart object, etc. (3) The caption mentioned that in Figure 1B, the brain map is reconstructed using the trained model, so it would be better to show the sketch of model in an appropriate place and indicate that its weights are frozen.
4. The reconstruction performance (Section 5.1, Figure 2) is quite poor. a correlation of 0.3 does not seem too promising.

Minor Things
1. Is there some typo in the following sentence? Page 5, 3rd paragraph. “When the percentile corresponds to a To avoid selecting studies with a null tf-idf value, we replace the percentile threshold with a strictly positive tf-idf value criteria.”

The evidence to be evaluated is:Although there has been interest in enriching brain images to increase the amount of data (Zhuang et al., 2019), to generate or reconstruct fMRI images (Manning et al., 2018; Cai et al., 2016), there has been limited interest in generating reconstructions of statistical brain maps from peaks (Gorgolewski et al., 2019), probably due to the daunting data dimensionality. In particular, we found no attempt at merging rich textual information from neuroscientific studies with the dense spatial information of brain images. We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations.

We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted. From a given set of \(l\) peaks \(\mathbf{P_{y}}\in\mathbb{R}^{3\times l}\) extracted from an image \(\mathbf{y}\in\mathbb{R}^{m}\) with \(m\) voxels, we learn the transformation \(f:\mathbb{R}^{3\times l}\mapsto\mathbb{R}^{m}\) by minimization of the \(\mathbf{L_{2}}\) error over the original image: \[f^{*}=\operatorname*{arg\,min}_{f}\mathbf{E_{y}}\|f(\mathbf{P_{y}})-\mathbf{ y}\|_{2}^{2} \tag{1}\] To solve the minimization problem in eq 1, we leverage a set of statistical maps taken from NeuroVault, an online repository of fMRI maps, from which we extract peak coordinates. This leads to a training set of peaks and the corresponding brain images to be reconstructed. Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

Instead of leveraging the peak coordinates directly as a sets problem (Zaheer et al., 2017), Peaks2Image builds an internal sparse Gaussian representation similar to the literature as by Laird et al. (2005); Dockes et al. (2020); Ngo et al. (2021).

Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information. In the rest of the paper, the term _brain image_ refers to its 1024-dimensional DiFuMo representation. Mapping voxels \(\mathbf{x}\in\mathbb{R}^{m}\) to coefficients \(\alpha\in R^{k}\) along DiFuMo dictionary \(\mathbf{D}\in\mathbb{R}^{m\times k}\) of dimension \(k\) is a mere linear regression problem following equation 2. All the following steps are performed in the \(k\)-dimensional DiFuMo space.

The DiFuMo-encoded sparse Gaussian representation is turned into a dense representation with a 3-layer neural network where the input and output are DiFuMo coefficients.

To train this reconstruction process, we use around 6k images from the #4438 NeuroVault collection ([https://neurovault.org/collections/4438](https://neurovault.org/collections/4438)). The reconstructed images are obtained from the predicted DiFuMo components by linear combinations of the dictionary components: \(\hat{\mathbf{y}}=\mathbf{D}f^{*}(\alpha(\mathbf{x}_{\mathbf{y}}))\). Peaks2Image outputs images from any set of positions.

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting. Indeed, the ability to decode cognitive concepts from the produced database ensures its validity and circumvents the biases inherent to image reconstruction quality metrics, such as the confounding effect of smoothness.

We leverage a 3-layer dense neural network to predict the set of binary labels \(\mathbf{L}=\{0,1\}_{1}\times\cdots\times\{0,1\}_{l}\), \(l\) denoting the total number of classes, from the input image \(\mathbf{X}\in\mathbb{R}^{d}\) compressed in DiFuMo space.

Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost. We leverage the NeuroVault database.

We take collection #4438, without labels (6k maps), to train the reconstruction task.

We use NeuroQuery Data Collection (Dockes et al., 2020) to fetch around 13k neuroscientific studies from PubMed Central. The tool helps us parse the activation tables of those studies, totalling 400k peaks. It also transforms each study text into tf-idf features across a vocabulary of 6308 terms. For validation, we restrict the analysis to shared terms between the tags of NeuroVault images and the vocabulary from the tf-idf. Among the 240 tags from the NeuroVault test collections, 189 are present in the vocabulary. We focus on those 189 tags without adding any additional mappings between the left-out tags and words from the vocabulary. Additionally, we filter out terms that contain less than 50 positive examples in the test set to prevent any non-significant effects, keeping 81 terms to decode.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020). Similarly to the NeuroVault images, we extract the DiFuMo representations of the sparse Gaussian maps using Nilearn.

The image reconstruction model is an MLP built from 3 dense layers with 1024 units and sigmoid activation implemented using PyTorch (Paszke et al., 2019). It is trained for 50 epochs using Adam optimizer and a learning rate of 1e-3. The loss is the mean-squared error (MSE) over the DiFuMo components. We select the best set of weights over the validation set, which is composed of 20% of collection #4438.

The decoder model is an MLP composed of one hidden layer of 300 units and ReLU activation.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We validate the relevance of the reconstructions by decoding NeuroVault images using reconstructed neuroscientific studies as training set. The problem is a multi-label classification, which we evaluate by computing the ROC AUC over each label. We exclude collection #4438 that has been used as training set in the reconstruction task, and evaluate the decoding of all remaining images from NeuroVault. We apply label enrichment to the original image labels. This includes regular expressions to map different terminologies for a same concept, and a hierarchy-based inference of labels based on the Cognitive Atlas ontology. For a given term, we assign parent terms from the ontology used in (Menuet et al., 2022). This results in 43k images, totalling around 200 different cognitive processes.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task. We observe a statistically significant gain of 0.1 on the mean AUC across labels between Peaks2Image and sparse Gaussian representations.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases. Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding. In particular, neuroscientific studies could become a challenging application for recent methods in text-to-image representation learning such as Radford et al. (2021).

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 2: Correlation between original images and reconstructions: We evaluate the reconstruction for different models (Linear regression, kernel Ridge regression, sparse Gaussian representation and Peaks2Image) using correlation in voxel space. Evaluation is performed on collection 4337 from NeuroVault (18k images).

Input: Figure 3: Zero-shot decoding performance on NeuroVault: evaluation is performed on all NeuroVault except collection #4438. We exclude categories with less than 50 positive examples for stability. Peaks2Image decodes better than chance 65 out of 81 cognitive processes over 43k images (AUC > 0.55).

Input: Figure 4: Distribution of relative AUC performance across tasks of Peaks2Image compared to other representations: We show the relative AUC gain of Peaks2Image for each task, along with the difference of mean AUC across labels vs different methods. We perform a t-test to probe the statistical significance of the difference between Peaks2Image and other methods. We also display the average AUC difference (∆). For NeuroSynth, we compute the difference on a subset of shared terms.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "In this work, we propose Peaks2Image, a neural-network approach to reconstruct continuous spatial representations of brain activity from peak activation tables."

Summary of evaluation:
The evidence strongly supports this claim. The paper thoroughly documents the development and implementation of Peaks2Image, including detailed methodology of how it reconstructs brain images from peak coordinates using neural networks. While the reconstruction performance shows moderate correlation (around 0.3-0.4), the claim itself is well-stated as it simply describes what the approach does without overstating its capabilities or performance. The claim is factual and accurately reflects the evidence without exaggeration.

<score>1</score>

Detailed justification:
1. The evidence directly demonstrates the development of a neural network approach (3-layer MLP) to reconstruct brain images from peak coordinates
2. The methodology is well-documented, including the DiFuMo representation, training process, and evaluation
3. The claim makes no exaggerated statements about performance or capabilities
4. Visual evidence in Figures 6 and 7 demonstrates the actual reconstruction process
5. While performance metrics are moderate, this does not affect the validity of the claim itself which simply states what the approach does

The reviewer comments highlight some weaknesses in other aspects of the paper but do not challenge the validity of this specific claim.