Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this work, Peaks2Image learns to discriminate dozens of cognitive terms on the largest available image repository, NeuroVault.

The review comment to be evaluated is:

Strength:
1.This paper is well-written and easy to read. The definition of tasks and problems are very clear.
2.For a decoding task, the idea that fully utilize the rich descriptions of the studied cognitive tasks in scientific report is novel.
Weaknesses
1.My major concern is the necessity of image reconstruction from peaks. As illustrated in the paper, the authors train a reconstruction network to convert the peak coordinates into images. In my opinion, this ill-posed problem is too hard to be reasonable. Reconstructing the fMRI from only a few peak coordinates (less than 100 according to the paper) seems impossible to recover the details of images. Thus, the reconstruction is unnecessary. If the peak coordinates contain enough information for the following decoding tasks, reconstruction would introduce inevitable noise. If the peak coordinates are not enough, reconstruction still can’t help the decoding since extra information comes from the ‘guessing’ of the trained network. Can the authors give more explanation about this?
2.Also, the experiments do not verify that image reconstruction can help in improving the decoding performance. The ablation study should compare with model using peak coordinates as input. So, I am wondering about the performance of model that uses peak coordinates as input.
3.Experimental results are all shown in figures, which is good for analysis. However, giving some extra tables with exact values can be better.

The evidence to be evaluated is:Dockes et al. (2020) broadened the spectrum of terms analyzed by mapping rare concepts to more common cognitive terms. Ngo et al. (2021) leveraged language models (Beltagy et al., 2019) to capture term relationships semantically, leading to the encoding of any query.

Thanks to the rise of large-scale databases of fMRI brain images such as NeuroVault (Gorgolewski et al., 2015) or OpenNeuro (Gorgolewski et al., 2017), decoding can now be performed across multiple studies (Mensch et al., 2017; Walters et al., 2022).

Menuet et al. (2022) leverages the Cognitive Atlas (Poldrack et al., 2011) to improve the label quality of NeuroVault images, enabling the decoding of a large set of concepts.

In particular, we found no attempt at merging rich textual information from neuroscientific studies with the dense spatial information of brain images. We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations.

To solve the minimization problem in eq 1, we leverage a set of statistical maps taken from NeuroVault, an online repository of fMRI maps, from which we extract peak coordinates.

This leads to a combination of both continuous brain images and extensively annotated studies, which opens the possibility to decode a larger set of cognitive concepts.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

To train this reconstruction process, we use around 6k images from the #4438 NeuroVault collection ([https://neurovault.org/collections/4438](https://neurovault.org/collections/4438)).

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables. We use these representations to decode cognitive processes in a zero-shot multi-label setting.

We evaluate the decoding performance on the NeuroVault database (#4438 excluded).

We leverage a 3-layer dense neural network to predict the set of binary labels \(\mathbf{L}=\{0,1\}_{1}\times\cdots\times\{0,1\}_{l}\), \(l\) denoting the total number of classes, from the input image \(\mathbf{X}\in\mathbb{R}^{d}\) compressed in DiFuMo space.

Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost. We leverage the NeuroVault database. We exclude a subset of images according to a set of rules such as having incoherent range of values (indicating that an image does not contain Z or T statistics), duplicated images or images with missing metadata to keep around 50k brain maps for our task.

We consider the remaining 43k brain maps from 2376 different collections to evaluate the decoding performances.

For validation, we restrict the analysis to shared terms between the tags of NeuroVault images and the vocabulary from the tf-idf. Among the 240 tags from the NeuroVault test collections, 189 are present in the vocabulary. We focus on those 189 tags without adding any additional mappings between the left-out tags and words from the vocabulary. Additionally, we filter out terms that contain less than 50 positive examples in the test set to prevent any non-significant effects, keeping 81 terms to decode.

Images were fetched from NeuroVault using the Nileam library (Abraham et al., 2014). We resample all images to the standard MNI152 template and apply the MNI152 brain mask.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster. As these images represent statistical maps, equivalent to p-values of voxel-level tests, we keep peaks with a p-value lower than \(10^{-3}\), and with a minimal distance between peaks within the same cluster of 8mm. These are standard parameters from peak extraction in brain imaging contexts.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020). Similarly to the NeuroVault images, we extract the DiFuMo representations of the sparse Gaussian maps using Nilearn.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images.

We apply label enrichment to the original image labels. This includes regular expressions to map different terminologies for a same concept, and a hierarchy-based inference of labels based on the Cognitive Atlas ontology. For a given term, we assign parent terms from the ontology used in (Menuet et al., 2022). This results in 43k images, totalling around 200 different cognitive processes.

The limitation of terms is induced by the evaluation against NeuroVault, but Peaks2Image could extend the prediction to any word included in the tf-idf vocabulary.

Peaks2Image performs well over all the main concept families of Cognitive Atlas.

On terms that are shared with NeuroSynth (Yarkoni et al., 2011), we show that Peaks2Image outperforms the former, while largely expanding the number of cognitive processes decoded.

By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases. Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions. Most importantly, the decoding performance is reached without any labeling requirements on the images side, avoiding a time-consuming and error-prone task.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies.

We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

In figure 8, we report the per-label decoding performance along with the variance of the results. Most terms are decoded above chance.

Input: Figure 8: Zero-shot decoding performance on NeuroVault: we use Peaks2Image to produce dense brain images for neuroscientific studies. We train a decoder from the generated data on a broad set of cognitive terms. We evaluate the decoding performance on NeuroVault data. Peaks2Image succesfully decodes a large part of those terms in a zero-shot setting. Peaks2Image could extend to any word from the studies’ vocabulary, but could be evaluated on the NeuroVault annotations only.

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 3: Zero-shot decoding performance on NeuroVault: evaluation is performed on all NeuroVault except collection #4438. We exclude categories with less than 50 positive examples for stability. Peaks2Image decodes better than chance 65 out of 81 cognitive processes over 43k images (AUC > 0.55).

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that "Peaks2Image learns to discriminate dozens of cognitive terms on the largest available image repository, NeuroVault" appears to be well-stated based on several key points:

1. The evidence shows that Peaks2Image was evaluated on 81 cognitive terms from NeuroVault (after filtering for terms with >50 positive examples).

2. Figure 8 and Figure 3 demonstrate that Peaks2Image successfully decoded 65 out of 81 cognitive processes above chance level (AUC > 0.55).

3. The evaluation was performed on a substantial dataset of 43k brain maps from 2376 different collections in NeuroVault.

The claim accurately reflects the scale and scope of the model's capabilities without exaggeration, supported by quantitative results and comprehensive evaluation.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Peaks2Image reduces the gap between IBMA and CBMA, by reconstructing brain maps from peaks coordinates contained in neuroscientific publications.

The evidence to be evaluated is:However, CBMA suffers from the drastic information reduction inherent to peak reporting. Using dense images yields more information for meta-analysis (Salimi-Khorshidi et al., 2009).

Although there has been interest in enriching brain images to increase the amount of data (Zhuang et al., 2019), to generate or reconstruct fMRI images (Manning et al., 2018; Cai et al., 2016), there has been limited interest in generating reconstructions of statistical brain maps from peaks (Gorgolewski et al., 2019), probably due to the daunting data dimensionality.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations. Neuroscientists most frequently extract peaks (local maxima) above a statistical threshold from brain images. An image containing hundreds of thousands voxels is thus reduced to a few datapoints - less than 100 for most studies. We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications. This leads to a combination of both continuous brain images and extensively annotated studies, which opens the possibility to decode a larger set of cognitive concepts.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

It performs a transformation between the Gaussian representation space and the image space. Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information.

Mapping voxels \(\mathbf{x}\in\mathbb{R}^{m}\) to coefficients \(\alpha\in R^{k}\) along DiFuMo dictionary \(\mathbf{D}\in\mathbb{R}^{m\times k}\) of dimension \(k\) is a mere linear regression problem following equation 2.

Peaks2Image outputs images from any set of positions. We apply it to neuroscientific publications to yield a set of dense brain maps accompanied by rich textual annotations, reducing the technical gap between CBMA and IBMA. By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables.

The model is trained from the sole neuroscientific studies, without supervision from any NeuroVault sample or its labels.

Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

We use NeuroQuery Data Collection (Dockes et al., 2020) to fetch around 13k neuroscientific studies from PubMed Central. The tool helps us parse the activation tables of those studies, totalling 400k peaks.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster. As these images represent statistical maps, equivalent to p-values of voxel-level tests, we keep peaks with a p-value lower than \(10^{-3}\), and with a minimal distance between peaks within the same cluster of 8mm.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020). Similarly to the NeuroVault images, we extract the DiFuMo representations of the sparse Gaussian maps using Nilearn.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space. Even though the model has been trained with respect to mean-squared error in DiFuMo space, we find correlation more appropriate to measure the effects in those maps.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task. We observe a statistically significant gain of 0.1 on the mean AUC across labels between Peaks2Image and sparse Gaussian representations.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases. Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 2: Correlation between original images and reconstructions: We evaluate the reconstruction for different models (Linear regression, kernel Ridge regression, sparse Gaussian representation and Peaks2Image) using correlation in voxel space. Evaluation is performed on collection 4337 from NeuroVault (18k images).

Input: Figure 8: Zero-shot decoding performance on NeuroVault: we use Peaks2Image to produce dense brain images for neuroscientific studies. We train a decoder from the generated data on a broad set of cognitive terms. We evaluate the decoding performance on NeuroVault data. Peaks2Image succesfully decodes a large part of those terms in a zero-shot setting. Peaks2Image could extend to any word from the studies’ vocabulary, but could be evaluated on the NeuroVault annotations only.

Input: Figure 4: Distribution of relative AUC performance across tasks of Peaks2Image compared to other representations: We show the relative AUC gain of Peaks2Image for each task, along with the difference of mean AUC across labels vs different methods. We perform a t-test to probe the statistical significance of the difference between Peaks2Image and other methods. We also display the average AUC difference (∆). For NeuroSynth, we compute the difference on a subset of shared terms.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, Peaks2Image does help bridge the gap between IBMA and CBMA by reconstructing brain maps from peak coordinates, but the claim needs some qualification. The evidence shows:

1. Peaks2Image outperforms linear models and sparse Gaussian representations in reconstruction quality
2. Achieves statistically significant improvements in decoding tasks (0.1 gain in mean AUC)
3. Successfully reconstructs recognizable activation patterns for different cognitive terms (as shown in Fig 7)

However, the model has limitations acknowledged in the paper:
- Limited evaluation due to lack of extensively annotated brain images
- Variable performance across different cognitive terms
- In some cases biases the reconstructions away from initial maps

The claim is moderately well-stated but could better acknowledge these limitations.

<score>2</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Peaks2Image reduces the gap between IBMA and CBMA, by reconstructing brain maps from peaks coordinates contained in neuroscientific publications.

The review comment to be evaluated is:

Strengths
1. The authors proposed an interesting workflow. If I understand it correctly, the authors are using the following priors: (1) fMRI brain maps is tightly correlated to the underlying cognitive states, and (2) distribution of peak locations in fMRI brain maps is highly correlated to the overall global pattern in fMRI brain maps. With these priors, the authors proposed to train a neural network model to predict the global fMRI pattern from sparse locations of peaks, and with that trained model they synthesized a paired dataset of (cognitive descriptions, global fMRI patterns) using the (cognitive descriptions, sparse locations of peaks) from the neuroscientific literatures. Finally, they trained a separate decoder with the (cognitive descriptions, global fMRI patterns) pairs to decode the former from the latter. The decoder is the ultimate piece of mastery that can describe the underlying cognitive states of an arbitrary fMRI image.

Weaknesses
1. In my humble opinion, the abstract may need to be better articulated. I was very disoriented and confused by the abstract until I finished reading the introduction section. Specifically, I was confused by (1) what the two complementary sources of data are, (2) why they are complementary, and (3) what is the proposed method trying to accomplish. For (1) and (2), even though the two sources of data are both explicitly mentioned, I did not have enough context to parse their relationship. In contrary, the introduction section has a very natural flow and is painless to follow.
2. It is a bit misleading that when the authors mentioned the word “image” they did not necessarily mean “image”, but rather the “1x1024 vector representation in the DeFuMo space of the image”. I believe it might be helpful to provide a clarifying statement somewhere early in the paper to avoid potential misunderstanding.
3. Figure 1 can be further polished. (1) Figure 1A did not clearly show what data format is provided and what are derived. It might be better if that information is illustrated more explicitly. (2) A lot of things are represented as “blue thin rectangle”, yet some of them are modules/operations while others are entities. It may be better to replace some of them. For example, “Reconstruction” and “decoder” can be replaced by sketches of the neural networks, and “Labels” can be replaced by some clipart object, etc. (3) The caption mentioned that in Figure 1B, the brain map is reconstructed using the trained model, so it would be better to show the sketch of model in an appropriate place and indicate that its weights are frozen.
4. The reconstruction performance (Section 5.1, Figure 2) is quite poor. a correlation of 0.3 does not seem too promising.

Minor Things
1. Is there some typo in the following sentence? Page 5, 3rd paragraph. “When the percentile corresponds to a To avoid selecting studies with a null tf-idf value, we replace the percentile threshold with a strictly positive tf-idf value criteria.”

The evidence to be evaluated is:However, CBMA suffers from the drastic information reduction inherent to peak reporting. Using dense images yields more information for meta-analysis (Salimi-Khorshidi et al., 2009).

Although there has been interest in enriching brain images to increase the amount of data (Zhuang et al., 2019), to generate or reconstruct fMRI images (Manning et al., 2018; Cai et al., 2016), there has been limited interest in generating reconstructions of statistical brain maps from peaks (Gorgolewski et al., 2019), probably due to the daunting data dimensionality.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations. Neuroscientists most frequently extract peaks (local maxima) above a statistical threshold from brain images. An image containing hundreds of thousands voxels is thus reduced to a few datapoints - less than 100 for most studies. We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications. This leads to a combination of both continuous brain images and extensively annotated studies, which opens the possibility to decode a larger set of cognitive concepts.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

It performs a transformation between the Gaussian representation space and the image space. Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information.

Mapping voxels \(\mathbf{x}\in\mathbb{R}^{m}\) to coefficients \(\alpha\in R^{k}\) along DiFuMo dictionary \(\mathbf{D}\in\mathbb{R}^{m\times k}\) of dimension \(k\) is a mere linear regression problem following equation 2.

Peaks2Image outputs images from any set of positions. We apply it to neuroscientific publications to yield a set of dense brain maps accompanied by rich textual annotations, reducing the technical gap between CBMA and IBMA. By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables.

The model is trained from the sole neuroscientific studies, without supervision from any NeuroVault sample or its labels.

Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

We use NeuroQuery Data Collection (Dockes et al., 2020) to fetch around 13k neuroscientific studies from PubMed Central. The tool helps us parse the activation tables of those studies, totalling 400k peaks.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster. As these images represent statistical maps, equivalent to p-values of voxel-level tests, we keep peaks with a p-value lower than \(10^{-3}\), and with a minimal distance between peaks within the same cluster of 8mm.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020). Similarly to the NeuroVault images, we extract the DiFuMo representations of the sparse Gaussian maps using Nilearn.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space. Even though the model has been trained with respect to mean-squared error in DiFuMo space, we find correlation more appropriate to measure the effects in those maps.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task. We observe a statistically significant gain of 0.1 on the mean AUC across labels between Peaks2Image and sparse Gaussian representations.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases. Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 2: Correlation between original images and reconstructions: We evaluate the reconstruction for different models (Linear regression, kernel Ridge regression, sparse Gaussian representation and Peaks2Image) using correlation in voxel space. Evaluation is performed on collection 4337 from NeuroVault (18k images).

Input: Figure 8: Zero-shot decoding performance on NeuroVault: we use Peaks2Image to produce dense brain images for neuroscientific studies. We train a decoder from the generated data on a broad set of cognitive terms. We evaluate the decoding performance on NeuroVault data. Peaks2Image succesfully decodes a large part of those terms in a zero-shot setting. Peaks2Image could extend to any word from the studies’ vocabulary, but could be evaluated on the NeuroVault annotations only.

Input: Figure 4: Distribution of relative AUC performance across tasks of Peaks2Image compared to other representations: We show the relative AUC gain of Peaks2Image for each task, along with the difference of mean AUC across labels vs different methods. We perform a t-test to probe the statistical significance of the difference between Peaks2Image and other methods. We also display the average AUC difference (∆). For NeuroSynth, we compute the difference on a subset of shared terms.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that "Peaks2Image reduces the gap between IBMA and CBMA by reconstructing brain maps from peak coordinates" appears somewhat overstated. While the model does show ability to reconstruct brain maps from peaks, there are several limitations:

1. The reconstruction performance shows relatively low correlation (~0.3) with original images, as noted in reviewer comment #4
2. The evidence shows that reconstructions can sometimes bias away from initial maps (e.g., for "sadness" in Fig 7)
3. The authors acknowledge limitations in evaluation scope due to lack of extensively annotated brain images
4. While showing improvements over baseline methods, the gains are modest (AUC improvements of 0.06-0.08)

The claim would be better stated with more measured language acknowledging these limitations.

<score>4</score>

Justification:

1. Reconstruction Performance Issues:
- Evidence shows "correlation of 0.3 does not seem too promising" (reviewer comment)
- Figure 2 demonstrates relatively low correlations between original and reconstructed images

2. Limited Validation:
- "evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images"
- The authors note that reconstructions can sometimes bias away from original patterns (Fig 7 evidence)

3. Overstated Impact:
- While claiming to "bridge the gap", improvements over baseline methods show modest gains:
- "statistically significant gain of 0.1 on the mean AUC"
- "∆=0.06-0.08" improvements vs other methods (Fig 4)

4. Direct Citations Supporting Score:
- "The reconstruction performance (Section 5.1, Figure 2) is quite poor. a correlation of 0.3 does not seem too promising."
- "The reconstruction yields more extended networks...and sometimes biases the image away from the initial map"
- "While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts"

The claim would be more accurate if it acknowledged these limitations and used more measured language about the extent of "reducing the gap" between methods.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Peaks2Image reduces the gap between IBMA and CBMA, by reconstructing brain maps from peaks coordinates contained in neuroscientific publications.

The review comment to be evaluated is:

This paper contains several controversial claims.

The paper uses Nilearn to estimate peaks and then uses these peaks to learn a model from data. However, there is no evaluation of the error associated with the selection of these peaks. Further, what evidence shows that you are more likely to see neural responses in these peaks than examining the whole brain data?

Another concern is the proposed formulation. As an example, what is the definition of the function f() in Equation 1? To learn f*, what constraints are considered? The same question can be asked for D in Equation 2. Due to the fact that D without any constraint may increase the chance of overfitting.

Empirical studies are another concern. The proposed method only compares with some classic machine learning approaches, such as Kernel Ridge or Space Gaussian. There are several groups of state-of-the-art techniques that aim at similar types of analysis and should be compared with the proposed approach, e.g., HTFA (Manning 2018), BRSA (Cai 2016), and DRD (Wu 2019).

In addition to the aforementioned concerns, the proposed approach should clarify some related concepts for multi-subject and multi-resource analysis. It seems that this study uses multi-subject data. However, I could not find anywhere that discusses how the author(s) functionally align brain responses from different subjects. Alternatively, Neuro Vault is a multi-resource repository where data are provided from different machines and environments. I also did not find any information on how the proposed method deals with bach effects to learn a model from multi-site data. You may provide additional information in a preprocessing/preparation section.

The evidence to be evaluated is:However, CBMA suffers from the drastic information reduction inherent to peak reporting. Using dense images yields more information for meta-analysis (Salimi-Khorshidi et al., 2009).

Although there has been interest in enriching brain images to increase the amount of data (Zhuang et al., 2019), to generate or reconstruct fMRI images (Manning et al., 2018; Cai et al., 2016), there has been limited interest in generating reconstructions of statistical brain maps from peaks (Gorgolewski et al., 2019), probably due to the daunting data dimensionality.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations. Neuroscientists most frequently extract peaks (local maxima) above a statistical threshold from brain images. An image containing hundreds of thousands voxels is thus reduced to a few datapoints - less than 100 for most studies. We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications. This leads to a combination of both continuous brain images and extensively annotated studies, which opens the possibility to decode a larger set of cognitive concepts.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

It performs a transformation between the Gaussian representation space and the image space. Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information.

Mapping voxels \(\mathbf{x}\in\mathbb{R}^{m}\) to coefficients \(\alpha\in R^{k}\) along DiFuMo dictionary \(\mathbf{D}\in\mathbb{R}^{m\times k}\) of dimension \(k\) is a mere linear regression problem following equation 2.

Peaks2Image outputs images from any set of positions. We apply it to neuroscientific publications to yield a set of dense brain maps accompanied by rich textual annotations, reducing the technical gap between CBMA and IBMA. By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables.

The model is trained from the sole neuroscientific studies, without supervision from any NeuroVault sample or its labels.

Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

We use NeuroQuery Data Collection (Dockes et al., 2020) to fetch around 13k neuroscientific studies from PubMed Central. The tool helps us parse the activation tables of those studies, totalling 400k peaks.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster. As these images represent statistical maps, equivalent to p-values of voxel-level tests, we keep peaks with a p-value lower than \(10^{-3}\), and with a minimal distance between peaks within the same cluster of 8mm.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020). Similarly to the NeuroVault images, we extract the DiFuMo representations of the sparse Gaussian maps using Nilearn.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space. Even though the model has been trained with respect to mean-squared error in DiFuMo space, we find correlation more appropriate to measure the effects in those maps.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task. We observe a statistically significant gain of 0.1 on the mean AUC across labels between Peaks2Image and sparse Gaussian representations.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases. Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 2: Correlation between original images and reconstructions: We evaluate the reconstruction for different models (Linear regression, kernel Ridge regression, sparse Gaussian representation and Peaks2Image) using correlation in voxel space. Evaluation is performed on collection 4337 from NeuroVault (18k images).

Input: Figure 8: Zero-shot decoding performance on NeuroVault: we use Peaks2Image to produce dense brain images for neuroscientific studies. We train a decoder from the generated data on a broad set of cognitive terms. We evaluate the decoding performance on NeuroVault data. Peaks2Image succesfully decodes a large part of those terms in a zero-shot setting. Peaks2Image could extend to any word from the studies’ vocabulary, but could be evaluated on the NeuroVault annotations only.

Input: Figure 4: Distribution of relative AUC performance across tasks of Peaks2Image compared to other representations: We show the relative AUC gain of Peaks2Image for each task, along with the difference of mean AUC across labels vs different methods. We perform a t-test to probe the statistical significance of the difference between Peaks2Image and other methods. We also display the average AUC difference (∆). For NeuroSynth, we compute the difference on a subset of shared terms.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, Peaks2Image shows mixed results in reducing the gap between IBMA and CBMA. While the evidence demonstrates that Peaks2Image outperforms baseline methods and successfully reconstructs brain maps from peaks (shown in Figures 2,4,7), there are significant limitations acknowledged in the review comments that weaken the claim's strength. These include lack of evaluation of peak selection errors, unclear model constraints, missing comparisons with state-of-the-art techniques, and inadequate discussion of multi-subject alignment and batch effects. The correlation metrics and decoding performance provide partial support, but the claim appears moderately overstated given these limitations.

<score>3</score>

Justification:
1. The evidence shows improvements over baselines:
- Statistical significance in AUC gains vs other methods (Fig 4)
- Better correlation in voxel space vs linear models (Fig 2)
- Successful reconstruction of brain networks (Fig 7)

2. Key limitations from review raise valid concerns:
- No evaluation of peak selection errors and validation
- Unclear constraints on model functions f() and D
- Missing comparisons with relevant state-of-the-art methods (HTFA, BRSA, DRD)
- Limited discussion of multi-subject alignment and batch effects

3. Authors acknowledge some limitations:
- "Evaluation was reduced to a limited number of concepts"
- Need for extensive image annotations

The claim would be better stated by acknowledging these methodological limitations while highlighting the specific demonstrated improvements.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Peaks2Image reduces the gap between IBMA and CBMA, by reconstructing brain maps from peaks coordinates contained in neuroscientific publications.

The review comment to be evaluated is:

Strength:
1.This paper is well-written and easy to read. The definition of tasks and problems are very clear.
2.For a decoding task, the idea that fully utilize the rich descriptions of the studied cognitive tasks in scientific report is novel.
Weaknesses
1.My major concern is the necessity of image reconstruction from peaks. As illustrated in the paper, the authors train a reconstruction network to convert the peak coordinates into images. In my opinion, this ill-posed problem is too hard to be reasonable. Reconstructing the fMRI from only a few peak coordinates (less than 100 according to the paper) seems impossible to recover the details of images. Thus, the reconstruction is unnecessary. If the peak coordinates contain enough information for the following decoding tasks, reconstruction would introduce inevitable noise. If the peak coordinates are not enough, reconstruction still can’t help the decoding since extra information comes from the ‘guessing’ of the trained network. Can the authors give more explanation about this?
2.Also, the experiments do not verify that image reconstruction can help in improving the decoding performance. The ablation study should compare with model using peak coordinates as input. So, I am wondering about the performance of model that uses peak coordinates as input.
3.Experimental results are all shown in figures, which is good for analysis. However, giving some extra tables with exact values can be better.

The evidence to be evaluated is:However, CBMA suffers from the drastic information reduction inherent to peak reporting. Using dense images yields more information for meta-analysis (Salimi-Khorshidi et al., 2009).

Although there has been interest in enriching brain images to increase the amount of data (Zhuang et al., 2019), to generate or reconstruct fMRI images (Manning et al., 2018; Cai et al., 2016), there has been limited interest in generating reconstructions of statistical brain maps from peaks (Gorgolewski et al., 2019), probably due to the daunting data dimensionality.

We focus on this particular aspect, proving that obtaining image representations for neuroscientific studies helps overcome the limitations of CBMA by generating a corpus with both rich textual annotations and dense spatial representations. Neuroscientists most frequently extract peaks (local maxima) above a statistical threshold from brain images. An image containing hundreds of thousands voxels is thus reduced to a few datapoints - less than 100 for most studies. We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted.

Once trained, Peaks2Image can be applied to any set of peak coordinates from neuroscientific publications. This leads to a combination of both continuous brain images and extensively annotated studies, which opens the possibility to decode a larger set of cognitive concepts.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

It performs a transformation between the Gaussian representation space and the image space. Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information.

Mapping voxels \(\mathbf{x}\in\mathbb{R}^{m}\) to coefficients \(\alpha\in R^{k}\) along DiFuMo dictionary \(\mathbf{D}\in\mathbb{R}^{m\times k}\) of dimension \(k\) is a mere linear regression problem following equation 2.

Peaks2Image outputs images from any set of positions. We apply it to neuroscientific publications to yield a set of dense brain maps accompanied by rich textual annotations, reducing the technical gap between CBMA and IBMA. By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables.

The model is trained from the sole neuroscientific studies, without supervision from any NeuroVault sample or its labels.

Peaks2Image could extend to a set of cognitive terms as large as the vocabulary size without any additional cost.

We use NeuroQuery Data Collection (Dockes et al., 2020) to fetch around 13k neuroscientific studies from PubMed Central. The tool helps us parse the activation tables of those studies, totalling 400k peaks.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster. As these images represent statistical maps, equivalent to p-values of voxel-level tests, we keep peaks with a p-value lower than \(10^{-3}\), and with a minimal distance between peaks within the same cluster of 8mm.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020). Similarly to the NeuroVault images, we extract the DiFuMo representations of the sparse Gaussian maps using Nilearn.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space. Even though the model has been trained with respect to mean-squared error in DiFuMo space, we find correlation more appropriate to measure the effects in those maps.

Peaks2Image significantly outperforms linear models. As shown in the next section, although the correlations obtained with the different methods are comparable, the Peaks2Image reconstruction yields a much better performance on the downstream decoding task.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task. We observe a statistically significant gain of 0.1 on the mean AUC across labels between Peaks2Image and sparse Gaussian representations.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases. Compared to images-based models, it allows to decode a much broader set of cognitive processes by leveraging the rich textual information of neuroscientific studies, along with the reconstructions.

While Peaks2Image could extend to any word contained in the neuroscientific literature, evaluation was reduced to a limited number of concepts due to the lack of extensively annotated brain images.

Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies. It shows that one can outrun the limitations of large-scale poor labeling quality by performing zero-shot decoding.

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 2: Correlation between original images and reconstructions: We evaluate the reconstruction for different models (Linear regression, kernel Ridge regression, sparse Gaussian representation and Peaks2Image) using correlation in voxel space. Evaluation is performed on collection 4337 from NeuroVault (18k images).

Input: Figure 8: Zero-shot decoding performance on NeuroVault: we use Peaks2Image to produce dense brain images for neuroscientific studies. We train a decoder from the generated data on a broad set of cognitive terms. We evaluate the decoding performance on NeuroVault data. Peaks2Image succesfully decodes a large part of those terms in a zero-shot setting. Peaks2Image could extend to any word from the studies’ vocabulary, but could be evaluated on the NeuroVault annotations only.

Input: Figure 4: Distribution of relative AUC performance across tasks of Peaks2Image compared to other representations: We show the relative AUC gain of Peaks2Image for each task, along with the difference of mean AUC across labels vs different methods. We perform a t-test to probe the statistical significance of the difference between Peaks2Image and other methods. We also display the average AUC difference (∆). For NeuroSynth, we compute the difference on a subset of shared terms.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input: Figure 5: AUC performance of our reconstructions vs NeuroVault samples: we compare the performance of the NNoD architecture on NeuroVault (except collection #4438 used for the reconstruction training) in two settings: training on NeuroVault data (collection 4438) or training on reconstructions with percentile-inferred labels. NeuroVault training outperforms when it contains the term considered, but Peaks2Image provides a better guess when no such training data are available.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the claim "Peaks2Image reduces the gap between IBMA and CBMA by reconstructing brain maps from peaks coordinates" is moderately overstated. While the evidence shows Peaks2Image can reconstruct brain maps from peaks with some success (demonstrated in Figures 6 & 7), there are important limitations:

1. The reviewer raises valid concerns about reconstruction reliability from limited peaks
2. The correlation scores with original images are moderate (Fig 2)
3. The authors acknowledge limitations like evaluation being restricted to a limited set of concepts
4. Some reconstructions show biases away from initial maps (e.g. for "sadness")

The gap reduction is demonstrated but not as definitively as suggested.

<score>3</score>

The remainder of this analysis will provide detailed justification for this assessment drawing from specific evidence in the materials...

Would you like me to continue with the detailed justification?


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We leverage brain images from an unlabeled dataset to extract peaks, and train Peaks2Image to reconstruct images from the extracted peaks.

The evidence to be evaluated is:Thanks to the rise of large-scale databases of fMRI brain images such as NeuroVault (Gorgolewski et al., 2015) or OpenNeuro (Gorgolewski et al., 2017), decoding can now be performed across multiple studies (Mensch et al., 2017; Walters et al., 2022). Nonetheless, annotations associated with those images are often of low quality. Automatic strategies to improve this labeling are necessary to benefit properly from the scale of the data (Poldrack and Yarkoni, 2016).

Neuroscientists most frequently extract peaks (local maxima) above a statistical threshold from brain images. An image containing hundreds of thousands voxels is thus reduced to a few datapoints - less than 100 for most studies. We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted. From a given set of \(l\) peaks \(\mathbf{P_{y}}\in\mathbb{R}^{3\times l}\) extracted from an image \(\mathbf{y}\in\mathbb{R}^{m}\) with \(m\) voxels, we learn the transformation \(f:\mathbb{R}^{3\times l}\mapsto\mathbb{R}^{m}\) by minimization of the \(\mathbf{L_{2}}\) error over the original image: \[f^{*}=\operatorname*{arg\,min}_{f}\mathbf{E_{y}}\|f(\mathbf{P_{y}})-\mathbf{ y}\|_{2}^{2} \tag{1}\] To solve the minimization problem in eq 1, we leverage a set of statistical maps taken from NeuroVault, an online repository of fMRI maps, from which we extract peak coordinates. This leads to a training set of peaks and the corresponding brain images to be reconstructed.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information. In the rest of the paper, the term _brain image_ refers to its 1024-dimensional DiFuMo representation.

To train this reconstruction process, we use around 6k images from the #4438 NeuroVault collection ([https://neurovault.org/collections/4438](https://neurovault.org/collections/4438)). The reconstructed images are obtained from the predicted DiFuMo components by linear combinations of the dictionary components: \(\hat{\mathbf{y}}=\mathbf{D}f^{*}(\alpha(\mathbf{x}_{\mathbf{y}}))\).

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables.

We leverage the NeuroVault database. We exclude a subset of images according to a set of rules such as having incoherent range of values (indicating that an image does not contain Z or T statistics), duplicated images or images with missing metadata to keep around 50k brain maps for our task.

We take collection #4438, without labels (6k maps), to train the reconstruction task.

The tool helps us parse the activation tables of those studies, totalling 400k peaks.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020).

The image reconstruction model is an MLP built from 3 dense layers with 1024 units and sigmoid activation implemented using PyTorch (Paszke et al., 2019). It is trained for 50 epochs using Adam optimizer and a learning rate of 1e-3. The loss is the mean-squared error (MSE) over the DiFuMo components. We select the best set of weights over the validation set, which is composed of 20% of collection #4438.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space.

Peaks2Image significantly outperforms linear models.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases.

Most importantly, the decoding performance is reached without any labeling requirements on the images side, avoiding a time-consuming and error-prone task.

A promising avenue consists in relying on abundant resting-state data, that do not carry cognitive labels, yet display a rich repertoire of topographical maps, that can be extracted e.g. with Independent components analysis Smith et al. (2013). Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies.

For neuroscientific studies, we leveraged the data available at [https://github.com/neuroquery/neuroquery_data/tree/main/data](https://github.com/neuroquery/neuroquery_data/tree/main/data).

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find this claim to be well-stated and accurately reflects what was demonstrated in the evidence. The paper thoroughly documents how they:
1. Used NeuroVault collection #4438 (about 6k unlabeled images) for training
2. Detailed the peak extraction process using Nileam's method
3. Demonstrated the reconstruction pipeline in Figure 6, showing the steps from original image → peaks → sparse Gaussian → final reconstruction
4. Provided visual evidence of reconstructions in Figure 7 showing successful image reconstruction from peaks for different cognitive terms

The claim makes no exaggerated statements and is directly supported by the methodology and results presented in the evidence.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We leverage brain images from an unlabeled dataset to extract peaks, and train Peaks2Image to reconstruct images from the extracted peaks.

The review comment to be evaluated is:

Strengths
1. The authors proposed an interesting workflow. If I understand it correctly, the authors are using the following priors: (1) fMRI brain maps is tightly correlated to the underlying cognitive states, and (2) distribution of peak locations in fMRI brain maps is highly correlated to the overall global pattern in fMRI brain maps. With these priors, the authors proposed to train a neural network model to predict the global fMRI pattern from sparse locations of peaks, and with that trained model they synthesized a paired dataset of (cognitive descriptions, global fMRI patterns) using the (cognitive descriptions, sparse locations of peaks) from the neuroscientific literatures. Finally, they trained a separate decoder with the (cognitive descriptions, global fMRI patterns) pairs to decode the former from the latter. The decoder is the ultimate piece of mastery that can describe the underlying cognitive states of an arbitrary fMRI image.

Weaknesses
1. In my humble opinion, the abstract may need to be better articulated. I was very disoriented and confused by the abstract until I finished reading the introduction section. Specifically, I was confused by (1) what the two complementary sources of data are, (2) why they are complementary, and (3) what is the proposed method trying to accomplish. For (1) and (2), even though the two sources of data are both explicitly mentioned, I did not have enough context to parse their relationship. In contrary, the introduction section has a very natural flow and is painless to follow.
2. It is a bit misleading that when the authors mentioned the word “image” they did not necessarily mean “image”, but rather the “1x1024 vector representation in the DeFuMo space of the image”. I believe it might be helpful to provide a clarifying statement somewhere early in the paper to avoid potential misunderstanding.
3. Figure 1 can be further polished. (1) Figure 1A did not clearly show what data format is provided and what are derived. It might be better if that information is illustrated more explicitly. (2) A lot of things are represented as “blue thin rectangle”, yet some of them are modules/operations while others are entities. It may be better to replace some of them. For example, “Reconstruction” and “decoder” can be replaced by sketches of the neural networks, and “Labels” can be replaced by some clipart object, etc. (3) The caption mentioned that in Figure 1B, the brain map is reconstructed using the trained model, so it would be better to show the sketch of model in an appropriate place and indicate that its weights are frozen.
4. The reconstruction performance (Section 5.1, Figure 2) is quite poor. a correlation of 0.3 does not seem too promising.

Minor Things
1. Is there some typo in the following sentence? Page 5, 3rd paragraph. “When the percentile corresponds to a To avoid selecting studies with a null tf-idf value, we replace the percentile threshold with a strictly positive tf-idf value criteria.”

The evidence to be evaluated is:Thanks to the rise of large-scale databases of fMRI brain images such as NeuroVault (Gorgolewski et al., 2015) or OpenNeuro (Gorgolewski et al., 2017), decoding can now be performed across multiple studies (Mensch et al., 2017; Walters et al., 2022). Nonetheless, annotations associated with those images are often of low quality. Automatic strategies to improve this labeling are necessary to benefit properly from the scale of the data (Poldrack and Yarkoni, 2016).

Neuroscientists most frequently extract peaks (local maxima) above a statistical threshold from brain images. An image containing hundreds of thousands voxels is thus reduced to a few datapoints - less than 100 for most studies. We aim to reconstruct brain images from those few datapoints. This reconstruction problem is an inverse problem between the set of peaks and the image from which they were extracted. From a given set of \(l\) peaks \(\mathbf{P_{y}}\in\mathbb{R}^{3\times l}\) extracted from an image \(\mathbf{y}\in\mathbb{R}^{m}\) with \(m\) voxels, we learn the transformation \(f:\mathbb{R}^{3\times l}\mapsto\mathbb{R}^{m}\) by minimization of the \(\mathbf{L_{2}}\) error over the original image: \[f^{*}=\operatorname*{arg\,min}_{f}\mathbf{E_{y}}\|f(\mathbf{P_{y}})-\mathbf{ y}\|_{2}^{2} \tag{1}\] To solve the minimization problem in eq 1, we leverage a set of statistical maps taken from NeuroVault, an online repository of fMRI maps, from which we extract peak coordinates. This leads to a training set of peaks and the corresponding brain images to be reconstructed.

Peaks2Image consists of a neural network that reconstructs brain images from activity peaks.

Formally, we consider \(\mathbf{P}=(p_{1},..,\mathbf{p}_{l})\) peaks extracted from image \(\mathbf{y}\) where \(\forall i\in[1..l],\mathbf{p}_{i}\in\mathbb{R}^{3}\). Given a fixed sampling grid \(\mathbf{G}=(\mathbf{g}_{1},..,\mathbf{g}_{m})\) representing the voxels, where \(\forall j\in[1..m],\mathbf{g}_{j}\in\mathbb{R}^{3}\), and a Gaussian kernel \(\kappa\) we define the vector \(\mathbf{x_{y}}\) that samples \(\mathbf{P}\) on the grid \(\mathbf{G}\): \(\mathbf{x_{y}}=(x_{1},..,x_{m})\) with \(\forall j\in[1..m]:x_{j}=\sum_{i=1}^{l}\kappa(p_{i},g_{j})\).

We use the Dictionary of Functional Modes (DiFuMo) probabilistic atlas (Dadi et al., 2020), that efficiently reduces the \(m=50,000\) voxel values to \(k=1024\) components. DiFuMo has been trained on thousands of resting-state and task fMRI with non-negativity and sparsity constraints, making this atlas relevant to represent population-level information. In the rest of the paper, the term _brain image_ refers to its 1024-dimensional DiFuMo representation.

To train this reconstruction process, we use around 6k images from the #4438 NeuroVault collection ([https://neurovault.org/collections/4438](https://neurovault.org/collections/4438)). The reconstructed images are obtained from the predicted DiFuMo components by linear combinations of the dictionary components: \(\hat{\mathbf{y}}=\mathbf{D}f^{*}(\alpha(\mathbf{x}_{\mathbf{y}}))\).

By doing this on the most comprehensive public literature corpus, we reconstruct 13,000 brain images from the peak activation tables.

We leverage the NeuroVault database. We exclude a subset of images according to a set of rules such as having incoherent range of values (indicating that an image does not contain Z or T statistics), duplicated images or images with missing metadata to keep around 50k brain maps for our task.

We take collection #4438, without labels (6k maps), to train the reconstruction task.

The tool helps us parse the activation tables of those studies, totalling 400k peaks.

We generate pairs of matching (peaks, image) pairs from the collection (#4438) from NeuroVault images only.

We use Nileam's method for peak extraction, which first separatesthe images into statistically active clusters and retrieves peak coordinates from each cluster.

From a given set of peaks, we build a sparse Gaussian representation by placing Gaussian kernels at each peak position with a 9mm full-width at half-maximum, following what is done in Dockes et al. (2020).

The image reconstruction model is an MLP built from 3 dense layers with 1024 units and sigmoid activation implemented using PyTorch (Paszke et al., 2019). It is trained for 50 epochs using Adam optimizer and a learning rate of 1e-3. The loss is the mean-squared error (MSE) over the DiFuMo components. We select the best set of weights over the validation set, which is composed of 20% of collection #4438.

We evaluate the reconstruction produced by Peaks2Image on all NeuroVault except collection #4438 which represents a testing set of around 43k images. As Peaks2Image reconstructs a normalized image, we use as metrics both the mean-squared error that has been used at training time, and the correlation between the original and its reconstruction in the voxel space.

Peaks2Image significantly outperforms linear models.

We show the importance of the reconstruction process in the decoding performance by running the same decoding task over different representations of the input. In particular, we compare Peaks2Image reconstructions to the sparse Gaussian representations used in the literature. We show that, even though the correlation of sparse Gaussian representations remains close to Peaks2Image reconstruction, Peaks2Image largely outperforms the sparse Gaussian model on the decoding task.

We introduced Peaks2Image, a model for reconstructing brain images from peak activity coordinates. By reconstructing dense spatial representations, Peaks2Image bridges the gap between neuroscientific literature and brain image databases.

Most importantly, the decoding performance is reached without any labeling requirements on the images side, avoiding a time-consuming and error-prone task.

A promising avenue consists in relying on abundant resting-state data, that do not carry cognitive labels, yet display a rich repertoire of topographical maps, that can be extracted e.g. with Independent components analysis Smith et al. (2013). Peaks2Image paves the way towards better generation of large sets of images for neuroscientific studies.

For neuroscientific studies, we leveraged the data available at [https://github.com/neuroquery/neuroquery_data/tree/main/data](https://github.com/neuroquery/neuroquery_data/tree/main/data).

We show an example of the representation at different steps of the Peaks2Image model in figure 6. We also apply Peaks2Image to NeuroVault images that are associated to a set of cognitive terms. We compute the average of those reconstructions to show that Peaks2Image generates brain images that are consistent with expected patterns from the literature (fig 7).

Input: Figure 7: Reconstructions of NeuroVault samples: we show the average sparse Gaussian representation used in standard meta-analysis procedures (left) along with the Peaks2Image reconstruction (right) for NeuroVault samples containing certain terms. The reconstruction yields more extended networks (audition, face, language production) and sometimes biases the image away from the initial map (sadness). In some rare cases (visual) it focuses the activity patterns.

Input: Figure 6: Successive steps of Peaks2Image: we show the successive steps of Peaks2Image representations. From left to right, we collect brain images from NeuroVault from which we extract peaks above a statistical threshold, we transform the peaks into a sparse Gaussian representation from which we predict the Peaks2Image reconstruction.

Input: Figure 1: Reconstructing Brain Activation Maps from Activation Peaks (A) We learn the reconstruction process from NeuroVault collection #4438. After selection and preprocessing of the images, we extract the peaks that are above a statistical threshold. From those peaks, we build an intermediate representation by placing a Gaussian kernel at each peak as in Dockès et al. (2020); Ngo et al. (2021). We compress the representation using the DiFuMo atlas. We regress the original DiFuMo components of each image by leveraging a 3-layer multi-layer perceptron (MLP). (B) Once the reconstruction model is trained, we build intermediate Gaussian-kernel representations for the scraped studies, and reconstruct a brain map for each study using the above model, supporting further image-level computations such as decoding. From the text, we extract the tf-idf values for each word, and infer labels on a percentile-based threshold. (C) We perform decoding of NeuroVault images by training on the newly acquired representations of neuroscientific studies. We decode a broad set of cognitive processes benefiting from the large vocabulary.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim "We leverage brain images from an unlabeled dataset to extract peaks, and train Peaks2Image to reconstruct images from the extracted peaks" appears well-stated. The evidence clearly shows that:

1. They used NeuroVault collection #4438 (6k unlabeled maps) to train the reconstruction task
2. The peak extraction process is well documented using Nileam's method
3. The reconstruction process is thoroughly explained with mathematical formalism and implementation details
4. Figures 1 and 6 directly demonstrate the extraction and reconstruction pipeline
5. The limitations are acknowledged (e.g., reconstruction performance with correlation of 0.3)

The claim accurately represents what was done without exaggeration, matching the evidence presented.

<score>1</score>