Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

2. we improve the "at-birth" synaptic connectivity to show that even low-capacity evolutionarily encoded information might lead to reasonable initial representations with no training at all,

The review comment to be evaluated is:

Strengths:
- The paper is clear and well written
- The analysis is thorough and supports the authors' arguments
- The method of initializing weights via clusters of trained kernels is interesting
- The "critical training" method of training only selected layers is also interesting

Weaknesses:
- "Recently, such models have been criticized due to how their learning departs from brain development (Marcus, 2004; Grossberg, 2020; Zador, 2019)." This seems to be the motivation for the work, but those authors don't raise the issues that are addressed in this paper. The Marcus book predates deep learning. Grossberg focuses on catastrophic forgetting. Zador focuses on the sparsity of labelled data in life. I am not sure whether the number of weight updates needed to train deep networks presents a substantial independent concern about their biological plausibility, particularly given the possibility of constant unsupervised learning. If the issue has been raised in the literature, more specific references would help here. If not, I think the authors should argue more specifically that it is a problem, before going on to solve it.
- The cluster-based initialization method seems to be presented as a way of bringing some learning results through a genetic bottleneck, but the biological plausibility isn't obvious. Further discussion of how this relates to and departs from biology would be helpful.
- It's interesting that critical training worked so well, but in the context of the paper's goals, the biological relevance of this method should also be laid out more clearly.

Minor:
- I don't think it was shown how classification performance is affected by the cluster-based initialization method, but it would be nice to see that.
- "37 bits per synapse (Zador, 2019)" This refers to the binary connectivity pattern (an issue of architecture) rather than the weights (which are addressed in this paper).

The evidence to be evaluated is:Concretely relating supervised updates to primate ventral stream development, Seibert (2018) establishes that no more than \(\sim\)4 months - or 10 million seconds - of waking visual experience is needed to reach adult-level primate IT cortex (as assessed by its capability to support adult level object recognition).

For instance, the fully trained model's 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans (Yarbus, 1967; Gibaldi & Sabatini, 2020). If few supervised updates can get model representations fairly close to a fully trained model (Figure 1), how close are the initial representations without any training? In relation to biology and following the introduced framework of treating all consecutive training as developmental learning, these "at-birth" synaptic connections would result from information encoded in the genome as a product of evolution. Due to the genome's capacity bottleneck, it is infeasible to precisely encode every synapse. Primary visual cortex alone contains \(\sim\)1.4E8 neurons per hemisphere (Leuba & Krafts, 1994), \(\sim\)1E3 synapses per neuron, each requiring \(\sim\)37 bits per synapse (Zador, 2019). Thus, without any clever rules, specifying the connections in only one hemisphere of V1 could require up to \(\sim\)5.2E12 bits - orders of magnitude more than the entire genome's 1GB = 8E9 bits (Zador, 2019). Sampling synaptic weights from reasonably compressed distributions on the other hand places little memory requirements on genetic encoding while potentially yielding useful initial weights.

To test the hypothesis that the genome might already encode more powerful initial representations with synaptic wiring sampled from distributions specified by only few bits, we explored multi-dimensional distributions as a more expressive alternative.

For the first convolutional layer only, we employ a Gabor prior on the weights following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987) (Appendix B). This results in 33 KB (4,166 parameters) to specify network initialization, compared to 423 MB for a trained model's weights (assuming 8 byte per parameter).

This model is _not trained at all_ and we only evaluate the goodness of its initial wiring on the suite of Brain-Score benchmarks. Strikingly, we find that even without any training, CORnet-S\({}_{\text{WC}}\) achieves \(54\pm 1.5\%\)of the brain predictivity score relative to a fully-trained model (Figure 2), representing a 12 percent point improvement (\(n=10\) seeds; permutation test \(p<1E-5\)) over the Kaiming Normal initialized model with a score of \(43\pm 1.7\%\).

_Weight Compression_ explores the hypothesis that evolution may have discovered an initialization strategy with improved at-birth representations (relative to current initializations). _WC_ is most likely not how evolution found the at-birth synaptic connections, but shows that with nearly identical capacity, an alternative initialization distribution leads to networks that are more brain-like in their adult state - revealing a new space of possibilities (hypotheses) that should be considered (see Appendix B.4 for more details on biological plausibility).

While improved "at-birth" connectivity can reach 54% of a fully-trained model's score, additional experience dependent updates appear necessary to reach higher predictivities.

As a proof-of-principle that training a reduced set of layers can retain high performance, we propose a novel thin training technique, which we term _Critical Training (CT: Figure 3A). CT_ updates only the weights in critical layers, instead of updating every single model synapse.

By reducing the number of trained parameters, _Critical Training_ also yields engineering benefits with more than 40% of the ImageNet score maintained at \(<3\%\) of parameters trained - a significant improvement over 22% accuracy with _BatchNormNorm_(Frankle et al., 2021) while adding only a small number of additional weights.

Reducing the number of supervised updates minimizes the number of epochs and images (Section 3); _Weight Compression (WC)_ improves the at-birth synaptic connectivity for high initial scores with no training at all (Section 4); and _Critical Training (CT)_ reduces the number of synapses that are updated during training (Section 5).

We developed a range of models with neural and behavioral scores approaching those of the current leading model of the adult ventral visual stream as quantified in Brain-Score, while requiring only a fraction of supervised synaptic updates. These models were built by complementarily 1) reducing the number of supervised updates, i.e. training epochs and labeled images; 2) improving the "at birth" distribution of synaptic connectivity; and 3) training only critical synapses at the end of each model area.

With the capacity bottleneck in the genome, mechanisms for wiring up would likely be shared between similar systems. The fact that early visual areas converge earlier during training (Figure 1) and are better predicted than higher areas by _WC_ initialization is consistent with developmental studies of the primate ventral stream. In humans, behaviors that rely on low-level spatial and temporal processing of visual inputs reach adult-like performance considerably earlier than complex visual behaviors that rely on higher cortical regions, such as face perception (Ellenberg et al., 1999; Grill-Spector et al., 2008). A critical component in more closely modeling primate development is to reduce the dependence on labels altogether.

The changes to model initialization and training presented here serve as a proof-of-principle that models can be changed to more closely align with primate development by reducing training steps with labeled images and improving initialization.

We expect future work in this direction to further close the gap with improved evolutionarily encoded wiring mechanisms and developmental learning rules.

For the very first layer of size \(7\times 7\) we found a Gabor filter most effective following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987). To generate the Gabor kernels we fit trained channel weights to a Gabor function

Weight Compression (WC)'s cluster-based initialization generates an "at-birth" network that already captures some useful aspects of the visual inputs in a compressed manner and allows the system to learn faster.

However, independent of _how_ these clusters are determined, WC shows that it is possible to encode certain priors about a system's wiring diagram in a very compressed manner. Evolution also acquired aspects of the visual inputs (with a different strategy) and encoded them in the genome in a lower information regime. For example, we know that there are multiple neuro-developmental mechanisms, such as spontaneous retinal waves and axon guidance cues, that significantly shape the architecture and function of the visual system requiring no (or very little) visual experience ((Huberman et al., 2008)). These mechanisms depend on a relatively small number of proteins encoded in a genome and give rise to a highly complex pattern of at-birth synaptic connectivity representing a very large compression of information.

A model initialized with weight compression achieves (only _WC_) 54% brain predictivity score with 0 supervised synaptic updates.

Notably, ImageNet performance of these networks seems to not be predictive of their brain predictivity, since even untrained networks with at-chance ImageNet performance correspond reasonably well to e.g. V1 (Figures 1 and 2).

* **KN** A model initialized by standard _Kaiming Normal_ initialization without training.

* WC A model initialized by our _Weight Compression_ initialization, described in Section 4, without training.

Input: Figure 1: High scores in brain predictivity can be achieved with few supervised updates (log xaxes). A Average brain predictivity of models trained with a range of supervised updates (different combinations of epochs × images). Each dot is a different hypothesis of how the ventral visual stream might have developed and shows the adult brain-likeness score that is achieved by that model. Fairly brain-like representations are already realized with few supervised updates, relative to a fully trained model (black dot; see also Figure 6). Standardly trained CORnet-S is set to be 100% score in brain predictivity on the benchmarks. A “pixels” baseline achieves 7% of this brain predictivity score. B Individual brain predictivity scores over epochs. Models start to approximate the primate ventral stream with few training epochs. Lower visual areas (V1, V2) are approximated earlier in training. C Like B, but number of training images instead of epochs. Few images are sufficient to approximate especially early visual areas.

Input: Figure 2: “At-birth” synaptic connectivity yields reasonable scores in brain predictivity. A Specifying the initial weight distribution: Kaiming Normal (KN, He et al., 2015) samples from a generic Gaussian. Weight Compression (WC) compresses trained weights into low-parameter clustered distributions that weights can be initialized from. B Visualization of WC compressed parameters: Gabor filters for first layer and cluster centers for following layers with kernel size > 3. The cluster centers capture an intuitive variety of kernel types. C “At-birth” representations with WC achieve 54% score in brain predictivity of a fully trained model, with no training at all. Scores after training remain virtually unchanged (shaded bars).

Input: Figure 7: Alternative weight compression methods Comparison of different initializations that compress weights, “at birth” i.e. without any training (gray) and after training critical layers (shades of blue) for 6 epochs. Our best clustering-based approach WC achieved similar results as the Mixture Gaussian approach (∼3 percent points mean difference) but leads to more diverse clusters. Performance drops when solely sampling weights from kernel based normal distributions (Kernel normal) and additionally disabling the Gabor prior (No Gabor prior)

Input: Figure 4: Benefits of combining training reductions for high brain predictivity with few supervised synaptic updates and limitations of techniques in high-update regimes. A (log x-axis) By reducing updates with a combination of fewer supervised updates (Figure 1), improved initialization WC (Figure 2), and training only down-sampling layers CT (Figure 3), the resulting models (dark blue dots; fewer supervised updates alone in light blue) maintain high brain predictivity scores while requiring only a fraction of supervised synaptic updates compared to standard CORnet-S (black dot, top right). B Comparison between WC-initialized models trained with CT versus standardly initialized models training all weights, when varying training epochs and labeled images. Colors represent their percent point difference in brain predictivity scores. WC+CT improve performance in regimes with few epochs and images, but fall behind in regimes with many labeled images and epochs.

Input: Figure 9: Individual brain benchmark scores for WC+CT model A Individual brain predictivitiy scores of WC+CT models trained with a range of epochs on all images. These models score especially high on V1, V2 and V4 already after one epoch in comparison to a model with standard initialization training all weights. IT and Behavior benchmarks continuously improve over later epochs as well but fall short of a fully trained model. B Like A, but with models trained until convergence on different numbers of labeled images, up to the full dataset of 1.28M images (rightmost points). As in A we see > 80% V1, V2 and V4 scores with only 100,000 images. For comparable IT and behavioral scores, more images are required.

Input: Figure 8: Detailed analysis of WC+CT. A When reducing the number of supervised synaptic updates, adding critical training (dark grey) and adding weight compression initialization (dark blue) both improve scores in brain predictivity at the same number of supervised synaptic updates, in comparison to a model with standard initialization and training all weights (bright blue). B Brain predictivities for the WC+CT model when trained with a range of epochs and labeled images. C Same as B, but for a standardly initialized (KN) model training all weights.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim that "we improve the 'at-birth' synaptic connectivity to show that even low-capacity evolutionarily encoded information might lead to reasonable initial representations with no training at all."

The evidence strongly supports this claim through:
1. Quantitative results showing 54% brain predictivity score with no training using Weight Compression (WC) initialization vs 43% with standard initialization
2. Demonstration that this is achieved with minimal parameters (33KB vs 423MB for trained model)
3. Clear figures showing improved performance of WC over standard initialization
4. Appropriate acknowledgment of limitations by noting this may not be how evolution actually works

The claim is carefully worded with "might lead to" and "reasonable" rather than making stronger assertions. The evidence directly demonstrates improved initial representations while maintaining appropriate uncertainty about biological mechanisms.

<score>1</score>

Justification:
- The claim is well-stated and accurately reflects the evidence without exaggeration
- Results are quantified with specific metrics
- Limitations and uncertainties are acknowledged
- The genomic capacity constraints are clearly explained
- Multiple visualization methods and metrics support the findings
- Reviewer comments don't dispute this specific claim's validity


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

2. we improve the "at-birth" synaptic connectivity to show that even low-capacity evolutionarily encoded information might lead to reasonable initial representations with no training at all,

The review comment to be evaluated is:

### Strength:

First of all, the question addressed in the paper is essential for both the machine learning and neuroscience community and the author aim at addressing in a well-written piece of work that also clearly contextualizes and motivates the issue at hand. This work uses a state-of-the-art neural architecture, instead of a reduced version of models as is often the case in this type of work. The authors also propose a good analysis and clearly lay out their results, although their interpretation is one of the weaknesses of the paper.

### Weakness:

Although it is true that this work has many strengths it is hard to overcome the following weaknesses, which question the motivation of the approach undertaken. First, there exists a set of more principled approaches that have been developed over the last 4-5 years for training ANNs in a biologically plausible way.  Second, the results obtained are very subjective to the authors as the result of the improvement of the score is minor.

Although it is true that we don’t expect the authors to solve how the visual system develops and that this work corresponds to a proof of principle, it is still hard to navigate the plausibility of the assumptions made among all possibilities. On the other hand, there is a large set of work that has been proposed to build biologically plausible deep neural networks starting from clear objective function minimization and enforcing locality of learning rules. Compared to this set of works this one seems more adhoc than principled. This represents an important weakness when addressing the more neuroscience-friendly community.

Indeed the approach proposed is more of a combination of existing approaches rather than a novel idea: the use of training of a subset of layers, different weights initialization based on the distribution of trained weights. It is not clear how this provides new insight to the statistical learning community.

The evidence to be evaluated is:Concretely relating supervised updates to primate ventral stream development, Seibert (2018) establishes that no more than \(\sim\)4 months - or 10 million seconds - of waking visual experience is needed to reach adult-level primate IT cortex (as assessed by its capability to support adult level object recognition).

For instance, the fully trained model's 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans (Yarbus, 1967; Gibaldi & Sabatini, 2020). If few supervised updates can get model representations fairly close to a fully trained model (Figure 1), how close are the initial representations without any training? In relation to biology and following the introduced framework of treating all consecutive training as developmental learning, these "at-birth" synaptic connections would result from information encoded in the genome as a product of evolution. Due to the genome's capacity bottleneck, it is infeasible to precisely encode every synapse. Primary visual cortex alone contains \(\sim\)1.4E8 neurons per hemisphere (Leuba & Krafts, 1994), \(\sim\)1E3 synapses per neuron, each requiring \(\sim\)37 bits per synapse (Zador, 2019). Thus, without any clever rules, specifying the connections in only one hemisphere of V1 could require up to \(\sim\)5.2E12 bits - orders of magnitude more than the entire genome's 1GB = 8E9 bits (Zador, 2019). Sampling synaptic weights from reasonably compressed distributions on the other hand places little memory requirements on genetic encoding while potentially yielding useful initial weights.

To test the hypothesis that the genome might already encode more powerful initial representations with synaptic wiring sampled from distributions specified by only few bits, we explored multi-dimensional distributions as a more expressive alternative.

For the first convolutional layer only, we employ a Gabor prior on the weights following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987) (Appendix B). This results in 33 KB (4,166 parameters) to specify network initialization, compared to 423 MB for a trained model's weights (assuming 8 byte per parameter).

This model is _not trained at all_ and we only evaluate the goodness of its initial wiring on the suite of Brain-Score benchmarks. Strikingly, we find that even without any training, CORnet-S\({}_{\text{WC}}\) achieves \(54\pm 1.5\%\)of the brain predictivity score relative to a fully-trained model (Figure 2), representing a 12 percent point improvement (\(n=10\) seeds; permutation test \(p<1E-5\)) over the Kaiming Normal initialized model with a score of \(43\pm 1.7\%\).

_Weight Compression_ explores the hypothesis that evolution may have discovered an initialization strategy with improved at-birth representations (relative to current initializations). _WC_ is most likely not how evolution found the at-birth synaptic connections, but shows that with nearly identical capacity, an alternative initialization distribution leads to networks that are more brain-like in their adult state - revealing a new space of possibilities (hypotheses) that should be considered (see Appendix B.4 for more details on biological plausibility).

While improved "at-birth" connectivity can reach 54% of a fully-trained model's score, additional experience dependent updates appear necessary to reach higher predictivities.

As a proof-of-principle that training a reduced set of layers can retain high performance, we propose a novel thin training technique, which we term _Critical Training (CT: Figure 3A). CT_ updates only the weights in critical layers, instead of updating every single model synapse.

By reducing the number of trained parameters, _Critical Training_ also yields engineering benefits with more than 40% of the ImageNet score maintained at \(<3\%\) of parameters trained - a significant improvement over 22% accuracy with _BatchNormNorm_(Frankle et al., 2021) while adding only a small number of additional weights.

Reducing the number of supervised updates minimizes the number of epochs and images (Section 3); _Weight Compression (WC)_ improves the at-birth synaptic connectivity for high initial scores with no training at all (Section 4); and _Critical Training (CT)_ reduces the number of synapses that are updated during training (Section 5).

We developed a range of models with neural and behavioral scores approaching those of the current leading model of the adult ventral visual stream as quantified in Brain-Score, while requiring only a fraction of supervised synaptic updates. These models were built by complementarily 1) reducing the number of supervised updates, i.e. training epochs and labeled images; 2) improving the "at birth" distribution of synaptic connectivity; and 3) training only critical synapses at the end of each model area.

With the capacity bottleneck in the genome, mechanisms for wiring up would likely be shared between similar systems. The fact that early visual areas converge earlier during training (Figure 1) and are better predicted than higher areas by _WC_ initialization is consistent with developmental studies of the primate ventral stream. In humans, behaviors that rely on low-level spatial and temporal processing of visual inputs reach adult-like performance considerably earlier than complex visual behaviors that rely on higher cortical regions, such as face perception (Ellenberg et al., 1999; Grill-Spector et al., 2008). A critical component in more closely modeling primate development is to reduce the dependence on labels altogether.

The changes to model initialization and training presented here serve as a proof-of-principle that models can be changed to more closely align with primate development by reducing training steps with labeled images and improving initialization.

We expect future work in this direction to further close the gap with improved evolutionarily encoded wiring mechanisms and developmental learning rules.

For the very first layer of size \(7\times 7\) we found a Gabor filter most effective following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987). To generate the Gabor kernels we fit trained channel weights to a Gabor function

Weight Compression (WC)'s cluster-based initialization generates an "at-birth" network that already captures some useful aspects of the visual inputs in a compressed manner and allows the system to learn faster.

However, independent of _how_ these clusters are determined, WC shows that it is possible to encode certain priors about a system's wiring diagram in a very compressed manner. Evolution also acquired aspects of the visual inputs (with a different strategy) and encoded them in the genome in a lower information regime. For example, we know that there are multiple neuro-developmental mechanisms, such as spontaneous retinal waves and axon guidance cues, that significantly shape the architecture and function of the visual system requiring no (or very little) visual experience ((Huberman et al., 2008)). These mechanisms depend on a relatively small number of proteins encoded in a genome and give rise to a highly complex pattern of at-birth synaptic connectivity representing a very large compression of information.

A model initialized with weight compression achieves (only _WC_) 54% brain predictivity score with 0 supervised synaptic updates.

Notably, ImageNet performance of these networks seems to not be predictive of their brain predictivity, since even untrained networks with at-chance ImageNet performance correspond reasonably well to e.g. V1 (Figures 1 and 2).

* **KN** A model initialized by standard _Kaiming Normal_ initialization without training.

* WC A model initialized by our _Weight Compression_ initialization, described in Section 4, without training.

Input: Figure 1: High scores in brain predictivity can be achieved with few supervised updates (log xaxes). A Average brain predictivity of models trained with a range of supervised updates (different combinations of epochs × images). Each dot is a different hypothesis of how the ventral visual stream might have developed and shows the adult brain-likeness score that is achieved by that model. Fairly brain-like representations are already realized with few supervised updates, relative to a fully trained model (black dot; see also Figure 6). Standardly trained CORnet-S is set to be 100% score in brain predictivity on the benchmarks. A “pixels” baseline achieves 7% of this brain predictivity score. B Individual brain predictivity scores over epochs. Models start to approximate the primate ventral stream with few training epochs. Lower visual areas (V1, V2) are approximated earlier in training. C Like B, but number of training images instead of epochs. Few images are sufficient to approximate especially early visual areas.

Input: Figure 2: “At-birth” synaptic connectivity yields reasonable scores in brain predictivity. A Specifying the initial weight distribution: Kaiming Normal (KN, He et al., 2015) samples from a generic Gaussian. Weight Compression (WC) compresses trained weights into low-parameter clustered distributions that weights can be initialized from. B Visualization of WC compressed parameters: Gabor filters for first layer and cluster centers for following layers with kernel size > 3. The cluster centers capture an intuitive variety of kernel types. C “At-birth” representations with WC achieve 54% score in brain predictivity of a fully trained model, with no training at all. Scores after training remain virtually unchanged (shaded bars).

Input: Figure 7: Alternative weight compression methods Comparison of different initializations that compress weights, “at birth” i.e. without any training (gray) and after training critical layers (shades of blue) for 6 epochs. Our best clustering-based approach WC achieved similar results as the Mixture Gaussian approach (∼3 percent points mean difference) but leads to more diverse clusters. Performance drops when solely sampling weights from kernel based normal distributions (Kernel normal) and additionally disabling the Gabor prior (No Gabor prior)

Input: Figure 4: Benefits of combining training reductions for high brain predictivity with few supervised synaptic updates and limitations of techniques in high-update regimes. A (log x-axis) By reducing updates with a combination of fewer supervised updates (Figure 1), improved initialization WC (Figure 2), and training only down-sampling layers CT (Figure 3), the resulting models (dark blue dots; fewer supervised updates alone in light blue) maintain high brain predictivity scores while requiring only a fraction of supervised synaptic updates compared to standard CORnet-S (black dot, top right). B Comparison between WC-initialized models trained with CT versus standardly initialized models training all weights, when varying training epochs and labeled images. Colors represent their percent point difference in brain predictivity scores. WC+CT improve performance in regimes with few epochs and images, but fall behind in regimes with many labeled images and epochs.

Input: Figure 9: Individual brain benchmark scores for WC+CT model A Individual brain predictivitiy scores of WC+CT models trained with a range of epochs on all images. These models score especially high on V1, V2 and V4 already after one epoch in comparison to a model with standard initialization training all weights. IT and Behavior benchmarks continuously improve over later epochs as well but fall short of a fully trained model. B Like A, but with models trained until convergence on different numbers of labeled images, up to the full dataset of 1.28M images (rightmost points). As in A we see > 80% V1, V2 and V4 scores with only 100,000 images. For comparable IT and behavioral scores, more images are required.

Input: Figure 8: Detailed analysis of WC+CT. A When reducing the number of supervised synaptic updates, adding critical training (dark grey) and adding weight compression initialization (dark blue) both improve scores in brain predictivity at the same number of supervised synaptic updates, in comparison to a model with standard initialization and training all weights (bright blue). B Brain predictivities for the WC+CT model when trained with a range of epochs and labeled images. C Same as B, but for a standardly initialized (KN) model training all weights.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence provided, I'll evaluate the claim that "we improve the 'at-birth' synaptic connectivity to show that even low-capacity evolutionarily encoded information might lead to reasonable initial representations with no training at all."

The evidence strongly supports this claim through multiple findings:
1. The Weight Compression (WC) initialization achieves 54±1.5% brain predictivity without any training, compared to 43±1.7% with standard Kaiming Normal initialization
2. This is achieved with only 33 KB of parameters vs 423 MB for trained weights
3. The results are statistically significant (p<1E-5)
4. The improvements are particularly strong in early visual areas, aligning with biological development patterns

The claim appears well-stated, as it carefully qualifies the findings as "reasonable" initial representations and provides concrete evidence of improvement while acknowledging limitations.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

2. we improve the "at-birth" synaptic connectivity to show that even low-capacity evolutionarily encoded information might lead to reasonable initial representations with no training at all,

The review comment to be evaluated is:

## Strengths
The paper addresses an important problem in theoretical neuroscience: while trained deep networks appear to be good models of the visual stream, the amount of training they need to get there is incompatible with biology.

Two of the proposed approaches to reduce the amount of training are somewhat straightforward: training for fewer epochs and freezing the amount of trained layers. However, I think the provided results are important as they show good fit to brain data can be achieved without closely following the training procedures of deep learning.

The last approach, weight clustering, is perhaps more interesting to the theoretical neuro community, as it works in the same way as genomically encoding the network (the “at birth” state).

Importantly, the proposed ways to reduce the amount of training are tested not only using ImageNet performance, but also ventral stream data, which is arguably more important for neuroscience.

Overall, the paper presents a novel set of results for an important problem, and supports its claims with a significant number of experiments.

## Weaknesses
As mentioned before, I think the weight compression part is the most valuable one, and therefore it should receive more attention in the paper.
1. Clustering should be better explained in the main text. Especially how the number of clusters was chosen.
2. In the beginning of Sec. 4, you discuss how many bits it would take to encode different networks. I think the same should be done for your experiments. E.g. what is the size of CORnet-S vs. the size of a compressed network?
3. I think similar weight compression techniques have been used before, see http://proceedings.mlr.press/v80/wu18h.html for example
4. In general, it’d be interesting to see how at-birth performance changes with the amount of “genetically encoded” information, i.e. the number of clusters passed through compression. E.g., would doubling the amount of clusters you use drastically improve performance? I’m not expecting this experiment to appear in the paper, but perhaps it’s worth discussing.

The majority of results is limited to one architecture, CORnet-S, which is not a very standard deep learning model. However, I think it is OK as repeating the experiments on something like ResNet50 would be very resource-consuming, and CORnet-S appears to be a better model of the visual stream anyway.

## Recommendation
As the paper addresses an important problem with a novel approach and extensive experiments, I recommend to accept the paper.

## Minor points
In Fig.1, were learning rates tuned individually for each length of training, or were short runs just snapshots of the longer ones? If it’s the latter, I’d imagine you can get better results by cranking up the learning rate and later decreasing it before you hit a plateau (e.g., by spacing out the same three LR decreases within a shorter run). I guess doing it for each point in Fig.1A is too much training, but repeating all points in Fig.1B-C should be possible?

> For instance, the fully trained model’s 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans

I would say that a more correct conclusion is that supervised learning is a bad concept for development, but I also think your results are applicable beyond supervised learning. Basically, you show that reducing the amount of updates in backprop retains good Brain-Score performance. But that probably tells more about SGD dynamics in deep networks rather than a specific loss function, and might easily translate to, say, more realistic self-supervised learning. (I guess you briefly mention this point in the discussion.)

> Abstract

> First, we find that only 2% of supervised updates (epochs and images) are needed to achieve ∼80% of the match to adult ventral stream.

80% of the best performing network -- currently it reads as 80% of absolute performance, which would presumably be 1.0 Brain-Score?

The caption of Fig.3C-D should better explain what exactly is trained in each setting

In Sec.6, you should specify that you used MobileNet V1 and ResNet-50.

The evidence to be evaluated is:Concretely relating supervised updates to primate ventral stream development, Seibert (2018) establishes that no more than \(\sim\)4 months - or 10 million seconds - of waking visual experience is needed to reach adult-level primate IT cortex (as assessed by its capability to support adult level object recognition).

For instance, the fully trained model's 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans (Yarbus, 1967; Gibaldi & Sabatini, 2020). If few supervised updates can get model representations fairly close to a fully trained model (Figure 1), how close are the initial representations without any training? In relation to biology and following the introduced framework of treating all consecutive training as developmental learning, these "at-birth" synaptic connections would result from information encoded in the genome as a product of evolution. Due to the genome's capacity bottleneck, it is infeasible to precisely encode every synapse. Primary visual cortex alone contains \(\sim\)1.4E8 neurons per hemisphere (Leuba & Krafts, 1994), \(\sim\)1E3 synapses per neuron, each requiring \(\sim\)37 bits per synapse (Zador, 2019). Thus, without any clever rules, specifying the connections in only one hemisphere of V1 could require up to \(\sim\)5.2E12 bits - orders of magnitude more than the entire genome's 1GB = 8E9 bits (Zador, 2019). Sampling synaptic weights from reasonably compressed distributions on the other hand places little memory requirements on genetic encoding while potentially yielding useful initial weights.

To test the hypothesis that the genome might already encode more powerful initial representations with synaptic wiring sampled from distributions specified by only few bits, we explored multi-dimensional distributions as a more expressive alternative.

For the first convolutional layer only, we employ a Gabor prior on the weights following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987) (Appendix B). This results in 33 KB (4,166 parameters) to specify network initialization, compared to 423 MB for a trained model's weights (assuming 8 byte per parameter).

This model is _not trained at all_ and we only evaluate the goodness of its initial wiring on the suite of Brain-Score benchmarks. Strikingly, we find that even without any training, CORnet-S\({}_{\text{WC}}\) achieves \(54\pm 1.5\%\)of the brain predictivity score relative to a fully-trained model (Figure 2), representing a 12 percent point improvement (\(n=10\) seeds; permutation test \(p<1E-5\)) over the Kaiming Normal initialized model with a score of \(43\pm 1.7\%\).

_Weight Compression_ explores the hypothesis that evolution may have discovered an initialization strategy with improved at-birth representations (relative to current initializations). _WC_ is most likely not how evolution found the at-birth synaptic connections, but shows that with nearly identical capacity, an alternative initialization distribution leads to networks that are more brain-like in their adult state - revealing a new space of possibilities (hypotheses) that should be considered (see Appendix B.4 for more details on biological plausibility).

While improved "at-birth" connectivity can reach 54% of a fully-trained model's score, additional experience dependent updates appear necessary to reach higher predictivities.

As a proof-of-principle that training a reduced set of layers can retain high performance, we propose a novel thin training technique, which we term _Critical Training (CT: Figure 3A). CT_ updates only the weights in critical layers, instead of updating every single model synapse.

By reducing the number of trained parameters, _Critical Training_ also yields engineering benefits with more than 40% of the ImageNet score maintained at \(<3\%\) of parameters trained - a significant improvement over 22% accuracy with _BatchNormNorm_(Frankle et al., 2021) while adding only a small number of additional weights.

Reducing the number of supervised updates minimizes the number of epochs and images (Section 3); _Weight Compression (WC)_ improves the at-birth synaptic connectivity for high initial scores with no training at all (Section 4); and _Critical Training (CT)_ reduces the number of synapses that are updated during training (Section 5).

We developed a range of models with neural and behavioral scores approaching those of the current leading model of the adult ventral visual stream as quantified in Brain-Score, while requiring only a fraction of supervised synaptic updates. These models were built by complementarily 1) reducing the number of supervised updates, i.e. training epochs and labeled images; 2) improving the "at birth" distribution of synaptic connectivity; and 3) training only critical synapses at the end of each model area.

With the capacity bottleneck in the genome, mechanisms for wiring up would likely be shared between similar systems. The fact that early visual areas converge earlier during training (Figure 1) and are better predicted than higher areas by _WC_ initialization is consistent with developmental studies of the primate ventral stream. In humans, behaviors that rely on low-level spatial and temporal processing of visual inputs reach adult-like performance considerably earlier than complex visual behaviors that rely on higher cortical regions, such as face perception (Ellenberg et al., 1999; Grill-Spector et al., 2008). A critical component in more closely modeling primate development is to reduce the dependence on labels altogether.

The changes to model initialization and training presented here serve as a proof-of-principle that models can be changed to more closely align with primate development by reducing training steps with labeled images and improving initialization.

We expect future work in this direction to further close the gap with improved evolutionarily encoded wiring mechanisms and developmental learning rules.

For the very first layer of size \(7\times 7\) we found a Gabor filter most effective following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987). To generate the Gabor kernels we fit trained channel weights to a Gabor function

Weight Compression (WC)'s cluster-based initialization generates an "at-birth" network that already captures some useful aspects of the visual inputs in a compressed manner and allows the system to learn faster.

However, independent of _how_ these clusters are determined, WC shows that it is possible to encode certain priors about a system's wiring diagram in a very compressed manner. Evolution also acquired aspects of the visual inputs (with a different strategy) and encoded them in the genome in a lower information regime. For example, we know that there are multiple neuro-developmental mechanisms, such as spontaneous retinal waves and axon guidance cues, that significantly shape the architecture and function of the visual system requiring no (or very little) visual experience ((Huberman et al., 2008)). These mechanisms depend on a relatively small number of proteins encoded in a genome and give rise to a highly complex pattern of at-birth synaptic connectivity representing a very large compression of information.

A model initialized with weight compression achieves (only _WC_) 54% brain predictivity score with 0 supervised synaptic updates.

Notably, ImageNet performance of these networks seems to not be predictive of their brain predictivity, since even untrained networks with at-chance ImageNet performance correspond reasonably well to e.g. V1 (Figures 1 and 2).

* **KN** A model initialized by standard _Kaiming Normal_ initialization without training.

* WC A model initialized by our _Weight Compression_ initialization, described in Section 4, without training.

Input: Figure 1: High scores in brain predictivity can be achieved with few supervised updates (log xaxes). A Average brain predictivity of models trained with a range of supervised updates (different combinations of epochs × images). Each dot is a different hypothesis of how the ventral visual stream might have developed and shows the adult brain-likeness score that is achieved by that model. Fairly brain-like representations are already realized with few supervised updates, relative to a fully trained model (black dot; see also Figure 6). Standardly trained CORnet-S is set to be 100% score in brain predictivity on the benchmarks. A “pixels” baseline achieves 7% of this brain predictivity score. B Individual brain predictivity scores over epochs. Models start to approximate the primate ventral stream with few training epochs. Lower visual areas (V1, V2) are approximated earlier in training. C Like B, but number of training images instead of epochs. Few images are sufficient to approximate especially early visual areas.

Input: Figure 2: “At-birth” synaptic connectivity yields reasonable scores in brain predictivity. A Specifying the initial weight distribution: Kaiming Normal (KN, He et al., 2015) samples from a generic Gaussian. Weight Compression (WC) compresses trained weights into low-parameter clustered distributions that weights can be initialized from. B Visualization of WC compressed parameters: Gabor filters for first layer and cluster centers for following layers with kernel size > 3. The cluster centers capture an intuitive variety of kernel types. C “At-birth” representations with WC achieve 54% score in brain predictivity of a fully trained model, with no training at all. Scores after training remain virtually unchanged (shaded bars).

Input: Figure 7: Alternative weight compression methods Comparison of different initializations that compress weights, “at birth” i.e. without any training (gray) and after training critical layers (shades of blue) for 6 epochs. Our best clustering-based approach WC achieved similar results as the Mixture Gaussian approach (∼3 percent points mean difference) but leads to more diverse clusters. Performance drops when solely sampling weights from kernel based normal distributions (Kernel normal) and additionally disabling the Gabor prior (No Gabor prior)

Input: Figure 4: Benefits of combining training reductions for high brain predictivity with few supervised synaptic updates and limitations of techniques in high-update regimes. A (log x-axis) By reducing updates with a combination of fewer supervised updates (Figure 1), improved initialization WC (Figure 2), and training only down-sampling layers CT (Figure 3), the resulting models (dark blue dots; fewer supervised updates alone in light blue) maintain high brain predictivity scores while requiring only a fraction of supervised synaptic updates compared to standard CORnet-S (black dot, top right). B Comparison between WC-initialized models trained with CT versus standardly initialized models training all weights, when varying training epochs and labeled images. Colors represent their percent point difference in brain predictivity scores. WC+CT improve performance in regimes with few epochs and images, but fall behind in regimes with many labeled images and epochs.

Input: Figure 9: Individual brain benchmark scores for WC+CT model A Individual brain predictivitiy scores of WC+CT models trained with a range of epochs on all images. These models score especially high on V1, V2 and V4 already after one epoch in comparison to a model with standard initialization training all weights. IT and Behavior benchmarks continuously improve over later epochs as well but fall short of a fully trained model. B Like A, but with models trained until convergence on different numbers of labeled images, up to the full dataset of 1.28M images (rightmost points). As in A we see > 80% V1, V2 and V4 scores with only 100,000 images. For comparable IT and behavioral scores, more images are required.

Input: Figure 8: Detailed analysis of WC+CT. A When reducing the number of supervised synaptic updates, adding critical training (dark grey) and adding weight compression initialization (dark blue) both improve scores in brain predictivity at the same number of supervised synaptic updates, in comparison to a model with standard initialization and training all weights (bright blue). B Brain predictivities for the WC+CT model when trained with a range of epochs and labeled images. C Same as B, but for a standardly initialized (KN) model training all weights.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim "we improve the 'at-birth' synaptic connectivity to show that even low-capacity evolutionarily encoded information might lead to reasonable initial representations with no training at all," I find the following:

The evidence strongly supports this claim through:
- Quantitative results showing WC initialization achieves 54% brain predictivity without any training, a 12% improvement over standard initialization
- Clear demonstration of compression (33KB vs 423MB)
- Alignment with biological constraints (genome capacity bottleneck)
- Multiple experimental validations including ablation studies
- Appropriate acknowledgment of limitations ("WC is most likely not how evolution found the at-birth synaptic connections")

The claim is well-stated and accurately reflects the evidence without exaggeration.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

2. we improve the "at-birth" synaptic connectivity to show that even low-capacity evolutionarily encoded information might lead to reasonable initial representations with no training at all,

The review comment to be evaluated is:

The authors present thorough analyses of standard feedforward networks that quantitatively demonstrate that such networks can model the visual stream well while requiring far fewer coordinated parameter changes than occur with traditional training methods.  In particular, they show that very little supervision is required for strong matches to neural responses, that plausible distributions of initial random weights can match neural responses very well, and that weights can be updated very sparsely while still achieving quite high neural response matches.

Such findings are highly relevant to allowing for such networks to be more plausibly accepted by the neuroscience community as valid models of cortical networks.  The striking finding that well-chosen random weights (Weight Compression) can predict neural responses with more than 50% accuracy is  especially interesting, as it suggests that indeed there may be less training required in mammalian brains compared to randomly initialized neuronal networks. This possibility opens up further avenues of study relating to the ability of real brains to learn with far fewer examples compared to ANNs.

Figure 4b further demonstrates this possibility, showing that, under the ethologically relevant condition of little supervision, WC + Critical Training, where only a small subset of weights in so-called critical layers are trained, greatly outperforms traditional training.  We note that, in the ideal case, a biologically plausible method might be found that performs well both under little supervision, as WC+CT does here, and under a large amount of supervision, as traditional backprop does, the demonstration of principle for the former case already represents appreciable progress.

However, little in the way of tying model predictivity to accuracy on another task was explored. While tying brain predictivity to network performance is of course a very active area in its own right, we feel that addressing this directly within the context of the present work would facilitate readers' abilities to contextualize the present study with respect to ANN performance.

Overall, we find the authors' presented work to be of great interest to those interested in applying ML models to theoretical questions in neuroscience and to have a thorough set of analyses that both illustrates progress in this area while suggesting future avenues of work.

Minor:
- opening quotes are backwards throughout paper

The evidence to be evaluated is:Concretely relating supervised updates to primate ventral stream development, Seibert (2018) establishes that no more than \(\sim\)4 months - or 10 million seconds - of waking visual experience is needed to reach adult-level primate IT cortex (as assessed by its capability to support adult level object recognition).

For instance, the fully trained model's 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans (Yarbus, 1967; Gibaldi & Sabatini, 2020). If few supervised updates can get model representations fairly close to a fully trained model (Figure 1), how close are the initial representations without any training? In relation to biology and following the introduced framework of treating all consecutive training as developmental learning, these "at-birth" synaptic connections would result from information encoded in the genome as a product of evolution. Due to the genome's capacity bottleneck, it is infeasible to precisely encode every synapse. Primary visual cortex alone contains \(\sim\)1.4E8 neurons per hemisphere (Leuba & Krafts, 1994), \(\sim\)1E3 synapses per neuron, each requiring \(\sim\)37 bits per synapse (Zador, 2019). Thus, without any clever rules, specifying the connections in only one hemisphere of V1 could require up to \(\sim\)5.2E12 bits - orders of magnitude more than the entire genome's 1GB = 8E9 bits (Zador, 2019). Sampling synaptic weights from reasonably compressed distributions on the other hand places little memory requirements on genetic encoding while potentially yielding useful initial weights.

To test the hypothesis that the genome might already encode more powerful initial representations with synaptic wiring sampled from distributions specified by only few bits, we explored multi-dimensional distributions as a more expressive alternative.

For the first convolutional layer only, we employ a Gabor prior on the weights following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987) (Appendix B). This results in 33 KB (4,166 parameters) to specify network initialization, compared to 423 MB for a trained model's weights (assuming 8 byte per parameter).

This model is _not trained at all_ and we only evaluate the goodness of its initial wiring on the suite of Brain-Score benchmarks. Strikingly, we find that even without any training, CORnet-S\({}_{\text{WC}}\) achieves \(54\pm 1.5\%\)of the brain predictivity score relative to a fully-trained model (Figure 2), representing a 12 percent point improvement (\(n=10\) seeds; permutation test \(p<1E-5\)) over the Kaiming Normal initialized model with a score of \(43\pm 1.7\%\).

_Weight Compression_ explores the hypothesis that evolution may have discovered an initialization strategy with improved at-birth representations (relative to current initializations). _WC_ is most likely not how evolution found the at-birth synaptic connections, but shows that with nearly identical capacity, an alternative initialization distribution leads to networks that are more brain-like in their adult state - revealing a new space of possibilities (hypotheses) that should be considered (see Appendix B.4 for more details on biological plausibility).

While improved "at-birth" connectivity can reach 54% of a fully-trained model's score, additional experience dependent updates appear necessary to reach higher predictivities.

As a proof-of-principle that training a reduced set of layers can retain high performance, we propose a novel thin training technique, which we term _Critical Training (CT: Figure 3A). CT_ updates only the weights in critical layers, instead of updating every single model synapse.

By reducing the number of trained parameters, _Critical Training_ also yields engineering benefits with more than 40% of the ImageNet score maintained at \(<3\%\) of parameters trained - a significant improvement over 22% accuracy with _BatchNormNorm_(Frankle et al., 2021) while adding only a small number of additional weights.

Reducing the number of supervised updates minimizes the number of epochs and images (Section 3); _Weight Compression (WC)_ improves the at-birth synaptic connectivity for high initial scores with no training at all (Section 4); and _Critical Training (CT)_ reduces the number of synapses that are updated during training (Section 5).

We developed a range of models with neural and behavioral scores approaching those of the current leading model of the adult ventral visual stream as quantified in Brain-Score, while requiring only a fraction of supervised synaptic updates. These models were built by complementarily 1) reducing the number of supervised updates, i.e. training epochs and labeled images; 2) improving the "at birth" distribution of synaptic connectivity; and 3) training only critical synapses at the end of each model area.

With the capacity bottleneck in the genome, mechanisms for wiring up would likely be shared between similar systems. The fact that early visual areas converge earlier during training (Figure 1) and are better predicted than higher areas by _WC_ initialization is consistent with developmental studies of the primate ventral stream. In humans, behaviors that rely on low-level spatial and temporal processing of visual inputs reach adult-like performance considerably earlier than complex visual behaviors that rely on higher cortical regions, such as face perception (Ellenberg et al., 1999; Grill-Spector et al., 2008). A critical component in more closely modeling primate development is to reduce the dependence on labels altogether.

The changes to model initialization and training presented here serve as a proof-of-principle that models can be changed to more closely align with primate development by reducing training steps with labeled images and improving initialization.

We expect future work in this direction to further close the gap with improved evolutionarily encoded wiring mechanisms and developmental learning rules.

For the very first layer of size \(7\times 7\) we found a Gabor filter most effective following studies in V1 (Hubel and Wiesel, 1962; Jones and Palmer, 1987). To generate the Gabor kernels we fit trained channel weights to a Gabor function

Weight Compression (WC)'s cluster-based initialization generates an "at-birth" network that already captures some useful aspects of the visual inputs in a compressed manner and allows the system to learn faster.

However, independent of _how_ these clusters are determined, WC shows that it is possible to encode certain priors about a system's wiring diagram in a very compressed manner. Evolution also acquired aspects of the visual inputs (with a different strategy) and encoded them in the genome in a lower information regime. For example, we know that there are multiple neuro-developmental mechanisms, such as spontaneous retinal waves and axon guidance cues, that significantly shape the architecture and function of the visual system requiring no (or very little) visual experience ((Huberman et al., 2008)). These mechanisms depend on a relatively small number of proteins encoded in a genome and give rise to a highly complex pattern of at-birth synaptic connectivity representing a very large compression of information.

A model initialized with weight compression achieves (only _WC_) 54% brain predictivity score with 0 supervised synaptic updates.

Notably, ImageNet performance of these networks seems to not be predictive of their brain predictivity, since even untrained networks with at-chance ImageNet performance correspond reasonably well to e.g. V1 (Figures 1 and 2).

* **KN** A model initialized by standard _Kaiming Normal_ initialization without training.

* WC A model initialized by our _Weight Compression_ initialization, described in Section 4, without training.

Input: Figure 1: High scores in brain predictivity can be achieved with few supervised updates (log xaxes). A Average brain predictivity of models trained with a range of supervised updates (different combinations of epochs × images). Each dot is a different hypothesis of how the ventral visual stream might have developed and shows the adult brain-likeness score that is achieved by that model. Fairly brain-like representations are already realized with few supervised updates, relative to a fully trained model (black dot; see also Figure 6). Standardly trained CORnet-S is set to be 100% score in brain predictivity on the benchmarks. A “pixels” baseline achieves 7% of this brain predictivity score. B Individual brain predictivity scores over epochs. Models start to approximate the primate ventral stream with few training epochs. Lower visual areas (V1, V2) are approximated earlier in training. C Like B, but number of training images instead of epochs. Few images are sufficient to approximate especially early visual areas.

Input: Figure 2: “At-birth” synaptic connectivity yields reasonable scores in brain predictivity. A Specifying the initial weight distribution: Kaiming Normal (KN, He et al., 2015) samples from a generic Gaussian. Weight Compression (WC) compresses trained weights into low-parameter clustered distributions that weights can be initialized from. B Visualization of WC compressed parameters: Gabor filters for first layer and cluster centers for following layers with kernel size > 3. The cluster centers capture an intuitive variety of kernel types. C “At-birth” representations with WC achieve 54% score in brain predictivity of a fully trained model, with no training at all. Scores after training remain virtually unchanged (shaded bars).

Input: Figure 7: Alternative weight compression methods Comparison of different initializations that compress weights, “at birth” i.e. without any training (gray) and after training critical layers (shades of blue) for 6 epochs. Our best clustering-based approach WC achieved similar results as the Mixture Gaussian approach (∼3 percent points mean difference) but leads to more diverse clusters. Performance drops when solely sampling weights from kernel based normal distributions (Kernel normal) and additionally disabling the Gabor prior (No Gabor prior)

Input: Figure 4: Benefits of combining training reductions for high brain predictivity with few supervised synaptic updates and limitations of techniques in high-update regimes. A (log x-axis) By reducing updates with a combination of fewer supervised updates (Figure 1), improved initialization WC (Figure 2), and training only down-sampling layers CT (Figure 3), the resulting models (dark blue dots; fewer supervised updates alone in light blue) maintain high brain predictivity scores while requiring only a fraction of supervised synaptic updates compared to standard CORnet-S (black dot, top right). B Comparison between WC-initialized models trained with CT versus standardly initialized models training all weights, when varying training epochs and labeled images. Colors represent their percent point difference in brain predictivity scores. WC+CT improve performance in regimes with few epochs and images, but fall behind in regimes with many labeled images and epochs.

Input: Figure 9: Individual brain benchmark scores for WC+CT model A Individual brain predictivitiy scores of WC+CT models trained with a range of epochs on all images. These models score especially high on V1, V2 and V4 already after one epoch in comparison to a model with standard initialization training all weights. IT and Behavior benchmarks continuously improve over later epochs as well but fall short of a fully trained model. B Like A, but with models trained until convergence on different numbers of labeled images, up to the full dataset of 1.28M images (rightmost points). As in A we see > 80% V1, V2 and V4 scores with only 100,000 images. For comparable IT and behavioral scores, more images are required.

Input: Figure 8: Detailed analysis of WC+CT. A When reducing the number of supervised synaptic updates, adding critical training (dark grey) and adding weight compression initialization (dark blue) both improve scores in brain predictivity at the same number of supervised synaptic updates, in comparison to a model with standard initialization and training all weights (bright blue). B Brain predictivities for the WC+CT model when trained with a range of epochs and labeled images. C Same as B, but for a standardly initialized (KN) model training all weights.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim and evidence:

The claim about improving "at-birth" synaptic connectivity leading to reasonable initial representations without training appears well-supported by the evidence. The data shows that Weight Compression initialization achieves 54% brain predictivity score with zero training, representing a 12% improvement over standard initialization. The evidence thoroughly documents the theoretical basis (genome capacity constraints), methodology (Gabor priors, compressed distributions), and empirical results (Figure 2). The reviewer comment also highlights this as a "striking finding." While there are some limitations noted in the review regarding task performance links, the core claim about improved initial representations is carefully stated and well-grounded in the presented evidence.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

3. we propose a thin, "critical training" technique which reduces the number of trained synapses while maintaining high brain predictivity and improves over previous computer vision attempts to minimize trained components,

The evidence to be evaluated is:We evaluated the brain predictivity scores of CORnet-S variants that were trained with a combination of fewer epochs and images.

Similarly, when training until convergence with fractions of the 1.28M total images, 50,000 images are sufficient to obtain high neural scores (80% of full training in V1, V2, V4, IT).

For instance, the fully trained model's 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans (Yarbus, 1967; Gibaldi & Sabatini, 2020).

Sampling synaptic weights from reasonably compressed distributions on the other hand places little memory requirements on genetic encoding while potentially yielding useful initial weights.

To test the hypothesis that the genome might already encode more powerful initial representations with synaptic wiring sampled from distributions specified by only few bits, we explored multi-dimensional distributions as a more expressive alternative. These distributions only require a small number of parameters, but unlike current generic initializers, we explicitly specify them for each layer. To determine the right parameterization, we compress a trained model's weights into clusters which we then sample from ("_Weight Compression, WC_"). More specifically, for all convolutional layers except the first, we cluster the kernel weights and later sample from the clusters.

Applying _WC_ to CORnet-S, we first obtain a compressed and clustered set of parameters, from which we sample entirely new weights to yield a new model CORnet-S\({}_{\text{WC}}\). This model is _not trained at all_ and we only evaluate the goodness of its initial wiring on the suite of Brain-Score benchmarks.

Training thin down-sampling layers reduces the number of updated synapses while maintaining high brain predictivity

As a proof-of-principle that training a reduced set of layers can retain high performance, we propose a novel thin training technique, which we term _Critical Training (CT: Figure 3A). CT_ updates only the weights in critical layers, instead of updating every single model synapse.

We explore successive variants of applying _CT_ up to a block in the architecture and then training the following blocks, e.g. freezing V1, V2, V4 with critical training of the respective down-sampling layers and additional IT training. The final _CT_ ventral stream model is almost completely frozen and only the synapses generating each cortical area's output are trained.

However, while the average score with _DT_ (gray) already drops below 65% with over a quarter of trained parameters remaining and _BN_ drops to 62% with very few parameters, _CT_ (blue) maintains over 75% with only 1.4 out of 52.8 million parameters trained.

By reducing the number of trained parameters, _Critical Training_ also yields engineering benefits with more than 40% of the ImageNet score maintained at \(<3\%\) of parameters trained - a significant improvement over 22% accuracy with _BatchNormNorm_(Frankle et al., 2021) while adding only a small number of additional weights.

High brain predictivity can be achieved with a relatively small number of supervised synaptic updates

Reducing the number of supervised updates minimizes the number of epochs and images (Section 3); _Weight Compression (WC)_ improves the at-birth synaptic connectivity for high initial scores with no training at all (Section 4); and _Critical Training (CT)_ reduces the number of synapses that are updated during training (Section 5).

_WC+CT_ yield strong benefits (green numbers) in a regime with few supervised updates, improving by up to 27 percent points when training for only 1 epoch on 1,000 images.

_WC+CT_ therefore most positively interact with a small budget of supervised updates (which is the focus of this work).

_WC+CT_ maintain 91% of the score in ResNet despite an almost 80% reduction in parameters.

We developed a range of models with neural and behavioral scores approaching those of the current leading model of the adult ventral visual stream as quantified in Brain-Score, while requiring only a fraction of supervised synaptic updates. These models were built by complementarily 1) reducing the number of supervised updates, i.e. training epochs and labeled images; 2) improving the "at birth" distribution of synaptic connectivity; and 3) training only critical synapses at the end of each model area.

We here started from CORnet-S, one of the leading models on Brain-Score, but does not fully predict all brain measurements (0.42 absolute score).

A critical component in more closely modeling primate development is to reduce the dependence on labels altogether. Recent unsupervised approaches are starting to rival the classification performance of supervised models (Caron et al., 2018; Henaff et al., 2019; Zhuang et al., 2020) and combining them with the advances presented here could further reduce the number of synaptic updates.

Local learning rules (Lowe et al., 2019; Xiong et al., 2020) might alleviate these concerns and with critical training (Figure 3), it could be sufficient to learn in only a subset of layers. The changes to model initialization and training presented here serve as a proof-of-principle that models can be changed to more closely align with primate development by reducing training steps with labeled images and improving initialization. It is also possible to achieve high brain predictivity when training only a fraction of weights, but all these models are still far from the actual biological mechanisms.

The weight compression approach we use in Section 4 is based on different initialization techniques, applied to different layers.

We have explored a variety of weight compression methods applied to different layers and evaluate their performance "at birth" without training and when trained with critical training.

Our best model _WC+CT_ benefits from a combination of improved initialization through weight compression, and critical training. Figure 8A shows models with standard initialization and training all weights, but with fewer supervised updates (cf. Figure 1), models that only train down-sampling layers (_CT_), and models that combine critical training with weight compression (_WC+CT_).

Figure 8B and C show detailed brain predictivity scores, relative to a fully trained model, for models initialized and trained with _WC+CT_ (B) and models initialized with standard Kaiming Normal and training all weights (C) when trained with a range of epochs and labeled images.

To show the generalization of our approach we applied the weight compression methods to a ResNet-50 (He et al., 2016) and a MobileNet (Howard et al., 2017) (version 1, multiplier 1.0, image size 224) architecture.

Since CORnet-S is inspired by ResNet modules, we applied our critical training approach by training all conv3 layers (equivalent down sampling layers) of ResNet50.

To transfer our critical training approach, we therefore additionally train the \(1\times 1\) MobileNet layers corresponding to conv3. This training version allows for more training but still reduces the amount of trained parameters by 43% while maintaining 78% of the original score.

* KN+CT The _KN_-initialized model trained with _Critical Training_ until convergence, i.e. three downstream layers and the decoder are trained and all other layers remain unchanged.

* **WC+CT** The _WC_-initialized model with _Critical Training_.

Input: Figure 3: Training only critical layers reduces the number of updated synapses while maintaining high brain predictivity. A We could naively reduce the parameters of a fully-trained model by freezing layers from the bottom up, training only the top layers (“Downstream Training DT”; gray box). We instead propose Critical Training (CT) which only trains thin “critical” layers (e.g. downsampling layers, blue box). B CORnet-S circuitry. CT only trains critical layers, leaving the rest frozen. C Naively reducing parameters from standard training (black dot, top right) quickly deteriorates brain predictivity scores (DT, gray line) whereas Critical Training reduces parameters while retaining high scores (blue line, CT). D Like C, but measuring ImageNet score. CT retains nearly half the score with a fraction of parameters.

Input: Figure 1: High scores in brain predictivity can be achieved with few supervised updates (log xaxes). A Average brain predictivity of models trained with a range of supervised updates (different combinations of epochs × images). Each dot is a different hypothesis of how the ventral visual stream might have developed and shows the adult brain-likeness score that is achieved by that model. Fairly brain-like representations are already realized with few supervised updates, relative to a fully trained model (black dot; see also Figure 6). Standardly trained CORnet-S is set to be 100% score in brain predictivity on the benchmarks. A “pixels” baseline achieves 7% of this brain predictivity score. B Individual brain predictivity scores over epochs. Models start to approximate the primate ventral stream with few training epochs. Lower visual areas (V1, V2) are approximated earlier in training. C Like B, but number of training images instead of epochs. Few images are sufficient to approximate especially early visual areas.

Input: Figure 7: Alternative weight compression methods Comparison of different initializations that compress weights, “at birth” i.e. without any training (gray) and after training critical layers (shades of blue) for 6 epochs. Our best clustering-based approach WC achieved similar results as the Mixture Gaussian approach (∼3 percent points mean difference) but leads to more diverse clusters. Performance drops when solely sampling weights from kernel based normal distributions (Kernel normal) and additionally disabling the Gabor prior (No Gabor prior)

Input: Figure 4: Benefits of combining training reductions for high brain predictivity with few supervised synaptic updates and limitations of techniques in high-update regimes. A (log x-axis) By reducing updates with a combination of fewer supervised updates (Figure 1), improved initialization WC (Figure 2), and training only down-sampling layers CT (Figure 3), the resulting models (dark blue dots; fewer supervised updates alone in light blue) maintain high brain predictivity scores while requiring only a fraction of supervised synaptic updates compared to standard CORnet-S (black dot, top right). B Comparison between WC-initialized models trained with CT versus standardly initialized models training all weights, when varying training epochs and labeled images. Colors represent their percent point difference in brain predictivity scores. WC+CT improve performance in regimes with few epochs and images, but fall behind in regimes with many labeled images and epochs.

Input: Figure 8: Detailed analysis of WC+CT. A When reducing the number of supervised synaptic updates, adding critical training (dark grey) and adding weight compression initialization (dark blue) both improve scores in brain predictivity at the same number of supervised synaptic updates, in comparison to a model with standard initialization and training all weights (bright blue). B Brain predictivities for the WC+CT model when trained with a range of epochs and labeled images. C Same as B, but for a standardly initialized (KN) model training all weights.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review, the claim about proposing a "thin, critical training technique" that reduces trained synapses while maintaining high brain predictivity appears well-supported by the evidence. The data shows Critical Training (CT) maintains over 75% brain predictivity while training only 1.4 out of 52.8 million parameters. The claim about improving over previous attempts is supported by comparisons showing CT maintains higher scores (>75%) compared to BatchNorm (~62%) with fewer parameters. The evidence demonstrates clear improvements while accurately acknowledging limitations. The wording accurately reflects the findings without exaggeration.

<score>1</score>

Justification:
1. Direct evidence supports reduced parameters: "CT maintains over 75% with only 1.4 out of 52.8 million parameters trained"

2. Comparative data validates improvement claim: CT shows "40% of ImageNet score maintained at <3% of parameters trained - a significant improvement over 22% accuracy with BatchNormNorm"

3. Figures (especially Fig 3) clearly demonstrate maintained performance with reduced parameters

4. Authors acknowledge limitations appropriately: "all these models are still far from the actual biological mechanisms"

The claim's wording aligns precisely with the evidence without overstating results.