Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet demonstrate that HRBP (HRBP++) can almost match the accuracy of unstructured sparse training methods while achieving a huge acceleration on hardware.

The review comment to be evaluated is:

Strength:
1. The network pruning is an important topic, and the author has given sufficient background introduction.
2. The paper illustrate the implementation flow chart of HRBP and HRBP++ clearly.

Weaknesses:
1. There are spelling errors in the paper.
2. The paper mix up the programming languages and the math notations. It makes the presented methods difficult to follow. The definitions of symbols and math notations are missing.
3. The advantages of using the presented methods are unclear. What is the compression rate by using the methods? How is the impact on the accuracy? What is the current state of the art?

The evidence to be evaluated is:Even with the help of the dedicated sparse matrix representation technique such as CSR format (Buluc et al., 2009), the unstructured sparsity still expects to have over \(85\%\) sparsity ratio to acquire limited acceleration since the irregular weight distribution causes significant computation overhead due to poor data locality (Yuan et al., 2021). The recent weight regrouping (reorganization) (Rumi et al., 2020) can accelerate the unstructured sparse weights on hardware by extracting multiple smaller dense blocks in a large sparse matrix, which can improve the throughput with GEMM.

The regrouping algorithm (Rumi et al., 2020) finds similar rows and columns from the sparse weights matrix \(\mathbf{W}\odot m\) and brings them together into several dense blocks. In detail, it firstly clusters \(C_{O}\) rows of \(m\) into several groups based on the Jaccard similarity among non-zeros columns. For each group, it then picks out columns with the most non-zeros weights from all \(C_{I}K_{h}K_{w}\) columns and generates one dense block. For example, in Fig. 2(a), we can group filter \(\mathbf{K}_{0}\), \(\mathbf{K}_{1}\), and \(\mathbf{K}_{3}\) together, and select columns with at least two non-zero (_i.e._, \(0^{th}\), \(2^{th}\), and \(5^{th}\) columns), which leads to the dense block with orange color in Fig. 2(b).

The vanilla HRBP inevitably introduces too many zero kernels, which may result in layer collapse and a heavy accuracy drop (Tanaka et al., 2020). Thus, we further propose HRBP++ to reduce the number of zero kernels.

To this end, HRBP++ allows for zero values within kernels, resulting in fewer entirely discarded kernels for the same overall sparsity ratio.

With HRBP(++), we can extract dense blocks from sparse weights and accelerate both forward and backward of sparse training. For simplicity, we apply HRBP(++) to a random pruning mask, which randomly chooses connections to satisfy the sparsity ratio requirements. To this end, we design a strategy to create a random pruning mask in HRBP (++) style, named _HRBP(++)-based Random Mask_.

We implement our method with CUDA and measure the acceleration rate on GPU. In detail, we rewrite the nn.Conv2d module of Pytorch with CUDA to run CNN with sparse weights. The thread is the basic programmable unit that allows GPU programmers to use massive numbers of CUDA cores. CUDA threads are grouped at different levels such as warp, block, and grid. Given the dense blocks found by HRBP/HRBP++, the original convolution computation will be decomposed into several computations corresponding to dense blocks, and these computations will be performed in parallel by threads.

We follow the experimental settings in (Wang et al., 2020; Yuan et al., 2021) and conduct experiments on CIFAR-10, CIFAR-100 and ImageNet-1K (Deng et al., 2009). As the common setting in sparse training, we apply the WideResNet-32-2 and VGG-19 (Simonyan and Zisserman, 2014) for CIFAR-10/100 and ResNet-50 (He et al., 2016) for ImageNet.

We evaluate our method on NVIDIA Ampere A100 (108 SMs, 40GB).

We report the overall training time acceleration rate compared with the dense baselines, which includes both forward and backward calculations on hardware.

We compare HRBP(++) on static sparse training against random unstructured pruning (RU), random channel-wise pruning (RC) with an ERK ratio, and unstructured pruning with well-defined criteria such as SNIP (Lee et al., 2019) and GraSP (Wang et al., 2020).

Following (Wang et al., 2020), we set the sparsity ratio to \(90\%\) and \(95\%\).

From the accuracy perspective, HRBP++ achieves similar performance to unstructured pruning methods and much better performance than channel-wise pruning. For example, on CIFAR-10 with ResNet-32, HRBP++ achieves an accuracy of \(90.84\%\) at \(95\%\) sparsity, which is \(3.65\%\) higher than the RC and just \(0.17\%\) negligible drop to SNIP (Lee et al., 2019).

From the training time acceleration perspective, unstructured pruning cannot be accelerated due to its irregularity of non-zero weights. However, our HRBP can almost match the acceleration of channel-wise pruning, with the benefit of our optimized convolution implementation. Moreover, HRBP obtains more acceleration compared to the convention regrouping (Rumi et al., 2020), as the latter can only accelerate the forward pass. In summary, our HRBP(++) can successfully achieve comparable accuracy as unstructured pruning and bring training time acceleration on hardware.

We take VGG-19 with \(90\%\) overall sparsity on CIFAR-10 as an example and report the speedup of each layer in Figure 5. At shallow layers with a small sparsity ratio, channel-wise pruning achieves a slightly better acceleration. However, at deeper layers, HRBP achieves comparable or even better results, with at most \(7.5\)x acceleration on Conv14.

We further explore the dynamic sparse training with HRBP++, which prunes and grows the connections based on the proposed block-wise updating during the training time. We compare DST with HRBP++ to several unstructured DST methods, including DeepR (Bellec et al., 2018), DSR (Mostafa and Wang, 2019), SET (Mocanu et al., 2018), and RigL (Evci et al., 2020).

As the speedup is similar to static sparse training, we only show the accuracy comparison in Table 2 for simplicity. Noticeably, our HRBP++ can still match the accuracy of unstructured DST methods.

We evaluate the static sparse training with HRBP(++) on ImageNet.

As shown in Table 3, HRBP++ can also achieve similar performance as well-designed unstructured pruning methods. Meanwhile, it provides \(1.26\times\) and \(1.44\times\) training time acceleration on hardware when the pruning ratio is \(80\%\) and \(90\%\) respectively. To this end, our HRBP++ is still effective on complicated tasks like ImageNet.

As in Figure 7, our method can achieve comparable accuracy as unstructured pruning methods at all sparsity levels and all kinds of networks. This suggests the generalization ability of our method. For the acceleration rate on both training and inference, please see Figure 14 in Appendix.

We then propose the HRBP, aiming to accelerate the sparse training at both forward and backward passes by extracting dense blocks from the unstructured mask. Extensive results suggest that sparse training with HRBP(++) can achieve comparable accuracy with unstructured pruning methods. Meanwhile, it brings significant training acceleration. We believe our method can integrate the advantage of both unstructured pruning and structured pruning and make sparse training more hardware-friendly.

Input: unstructured mask \(m\), density ratio of mask \(r\), number of clusters \(t\), density ratio of kernel \(s\), number of input channel \(C_{I}\), the number of output channel \(C_{O}\), the width of the kernel \(K_{w}\), the height of the kernel \(K_{h}\), and the minimum number of rows of each group \(B_{1}\) Output: Dense groups

With HRBP, we can achieve \(72.7\%\) accuracy with \(60\%\) sparsity.

Our method can also achieve comparable or even better accuracy than DessiLBI at the same sparsity level. Note that, DessiLBI follows the LTH setting which still needs dense training at the first step, while our method directly trains the sparse subnetwork from scratch without dense training. We by default apply our method to random unstructured sparse mask with ERK sparse ratio distribution (Evci et al., 2020) of each layer.

We then apply HRBP++ to these unstructured masks and show the results in Figure 13. Noticeably, with SNIP and GraSP masks, HRBP++ can also achieve comparable accuracy at the same sparsity level.

This is consistent with the conclusion in (Evci et al., 2020) that ERK sparsity is beneficial.

Input: Table 3: Static sparse training on ImageNet. The dense ResNet-50 has 75.70% top-1 accuracy.

Input: Figure 5: The layer-wise speedup of VGG-19.

Input: Figure 7: The trade-off between sparsity and accuracy. All models are evaluated on CIFAR-10.

Input: Table 2: Comparison of dynamic sparse training methods on CIFAR-10 and CIFAR-100.

Input: Figure 14: Training and inference acceleration on hardware of ResNet-32 on CIFAR-10.

Input: Figure 13: Ablation study with different types of mask initialization of ResNet-32 on CIFAR-10.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet demonstrate that HRBP (HRBP++) can almost match the accuracy of unstructured sparse training methods while achieving a huge acceleration on hardware."

Analysis:
The evidence partially supports this claim but suggests some overstatement:

1. Accuracy matching:
- The evidence shows comparable accuracy on multiple datasets (CIFAR-10/100, ImageNet)
- Tables 2 and 3 demonstrate similar performance to unstructured methods
- Figure 7 shows comparable accuracy across different sparsity levels

2. Acceleration claims:
- The term "huge acceleration" is overstated:
- On ImageNet, only 1.26× and 1.44× speedups are shown at 80% and 90% sparsity
- Figure 14 shows modest training acceleration (around 1.5×) even at high sparsity
- Layer-wise speedups vary significantly (Figure 5)

The review comment also points out that the advantages and compression rates are unclear. While the method does show benefits, the term "huge acceleration" overstates the actual modest speedups shown in the evidence.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet demonstrate that HRBP (HRBP++) can almost match the accuracy of unstructured sparse training methods while achieving a huge acceleration on hardware.

The review comment to be evaluated is:

Strength:

1, generally this paper is well written,
2, the presented method is interesting.

weakness:

1, I had some doubts whether the compared baselines are reasonable. Why not directly using the group lasso penalty on the each convolutional filters like [r1,r2]. This method should be the naive way of pruning the filters.
[r1]Exploring Structural Sparsity of Deep Networks via Inverse Scale Spaces, TPAMI 2022
[r2] DessiLBI: Exploring Structural Sparsity on Deep Network via Differential Inclusion Paths. ICML2020
These should be compared, as group lass is exploring the structure sparsity of CNN filters.
I think the method of these papers is pretty strong competitor to the proposed method in this paper.

2, Page 4, HRBP paragraph, the third line: "weights, which is obtained by counting Thus, the". It seems missing something here?

3, As transformers make very good performance in many task, is it possible to apply the algorithm to transformer?  CNNs are a bit out of date (less attractive).

4, As this paper advocates the hardware-friendly algorithm. It should give some more implementation details: is the HRBP/HRBP++ implemented by cuda? How's the implementation of baselines in Fig. 5? Is the comparison apple-to-apple? Are the authours using C++ or Python to run the baselines in Fig. 5?

5. Fig.6 should be results from HRBP++. However the caption refers to HRBP instead.
6. It would be better if the authors could have some discussion about the practical usage of the proposed method. It seems that it is not so worth it  having only 2 times faster speed while dropping almost all of the original parameters and suffering from significant performance drop.

The evidence to be evaluated is:Even with the help of the dedicated sparse matrix representation technique such as CSR format (Buluc et al., 2009), the unstructured sparsity still expects to have over \(85\%\) sparsity ratio to acquire limited acceleration since the irregular weight distribution causes significant computation overhead due to poor data locality (Yuan et al., 2021). The recent weight regrouping (reorganization) (Rumi et al., 2020) can accelerate the unstructured sparse weights on hardware by extracting multiple smaller dense blocks in a large sparse matrix, which can improve the throughput with GEMM.

The regrouping algorithm (Rumi et al., 2020) finds similar rows and columns from the sparse weights matrix \(\mathbf{W}\odot m\) and brings them together into several dense blocks. In detail, it firstly clusters \(C_{O}\) rows of \(m\) into several groups based on the Jaccard similarity among non-zeros columns. For each group, it then picks out columns with the most non-zeros weights from all \(C_{I}K_{h}K_{w}\) columns and generates one dense block. For example, in Fig. 2(a), we can group filter \(\mathbf{K}_{0}\), \(\mathbf{K}_{1}\), and \(\mathbf{K}_{3}\) together, and select columns with at least two non-zero (_i.e._, \(0^{th}\), \(2^{th}\), and \(5^{th}\) columns), which leads to the dense block with orange color in Fig. 2(b).

The vanilla HRBP inevitably introduces too many zero kernels, which may result in layer collapse and a heavy accuracy drop (Tanaka et al., 2020). Thus, we further propose HRBP++ to reduce the number of zero kernels.

To this end, HRBP++ allows for zero values within kernels, resulting in fewer entirely discarded kernels for the same overall sparsity ratio.

With HRBP(++), we can extract dense blocks from sparse weights and accelerate both forward and backward of sparse training. For simplicity, we apply HRBP(++) to a random pruning mask, which randomly chooses connections to satisfy the sparsity ratio requirements. To this end, we design a strategy to create a random pruning mask in HRBP (++) style, named _HRBP(++)-based Random Mask_.

We implement our method with CUDA and measure the acceleration rate on GPU. In detail, we rewrite the nn.Conv2d module of Pytorch with CUDA to run CNN with sparse weights. The thread is the basic programmable unit that allows GPU programmers to use massive numbers of CUDA cores. CUDA threads are grouped at different levels such as warp, block, and grid. Given the dense blocks found by HRBP/HRBP++, the original convolution computation will be decomposed into several computations corresponding to dense blocks, and these computations will be performed in parallel by threads.

We follow the experimental settings in (Wang et al., 2020; Yuan et al., 2021) and conduct experiments on CIFAR-10, CIFAR-100 and ImageNet-1K (Deng et al., 2009). As the common setting in sparse training, we apply the WideResNet-32-2 and VGG-19 (Simonyan and Zisserman, 2014) for CIFAR-10/100 and ResNet-50 (He et al., 2016) for ImageNet.

We evaluate our method on NVIDIA Ampere A100 (108 SMs, 40GB).

We report the overall training time acceleration rate compared with the dense baselines, which includes both forward and backward calculations on hardware.

We compare HRBP(++) on static sparse training against random unstructured pruning (RU), random channel-wise pruning (RC) with an ERK ratio, and unstructured pruning with well-defined criteria such as SNIP (Lee et al., 2019) and GraSP (Wang et al., 2020).

Following (Wang et al., 2020), we set the sparsity ratio to \(90\%\) and \(95\%\).

From the accuracy perspective, HRBP++ achieves similar performance to unstructured pruning methods and much better performance than channel-wise pruning. For example, on CIFAR-10 with ResNet-32, HRBP++ achieves an accuracy of \(90.84\%\) at \(95\%\) sparsity, which is \(3.65\%\) higher than the RC and just \(0.17\%\) negligible drop to SNIP (Lee et al., 2019).

From the training time acceleration perspective, unstructured pruning cannot be accelerated due to its irregularity of non-zero weights. However, our HRBP can almost match the acceleration of channel-wise pruning, with the benefit of our optimized convolution implementation. Moreover, HRBP obtains more acceleration compared to the convention regrouping (Rumi et al., 2020), as the latter can only accelerate the forward pass. In summary, our HRBP(++) can successfully achieve comparable accuracy as unstructured pruning and bring training time acceleration on hardware.

We take VGG-19 with \(90\%\) overall sparsity on CIFAR-10 as an example and report the speedup of each layer in Figure 5. At shallow layers with a small sparsity ratio, channel-wise pruning achieves a slightly better acceleration. However, at deeper layers, HRBP achieves comparable or even better results, with at most \(7.5\)x acceleration on Conv14.

We further explore the dynamic sparse training with HRBP++, which prunes and grows the connections based on the proposed block-wise updating during the training time. We compare DST with HRBP++ to several unstructured DST methods, including DeepR (Bellec et al., 2018), DSR (Mostafa and Wang, 2019), SET (Mocanu et al., 2018), and RigL (Evci et al., 2020).

As the speedup is similar to static sparse training, we only show the accuracy comparison in Table 2 for simplicity. Noticeably, our HRBP++ can still match the accuracy of unstructured DST methods.

We evaluate the static sparse training with HRBP(++) on ImageNet.

As shown in Table 3, HRBP++ can also achieve similar performance as well-designed unstructured pruning methods. Meanwhile, it provides \(1.26\times\) and \(1.44\times\) training time acceleration on hardware when the pruning ratio is \(80\%\) and \(90\%\) respectively. To this end, our HRBP++ is still effective on complicated tasks like ImageNet.

As in Figure 7, our method can achieve comparable accuracy as unstructured pruning methods at all sparsity levels and all kinds of networks. This suggests the generalization ability of our method. For the acceleration rate on both training and inference, please see Figure 14 in Appendix.

We then propose the HRBP, aiming to accelerate the sparse training at both forward and backward passes by extracting dense blocks from the unstructured mask. Extensive results suggest that sparse training with HRBP(++) can achieve comparable accuracy with unstructured pruning methods. Meanwhile, it brings significant training acceleration. We believe our method can integrate the advantage of both unstructured pruning and structured pruning and make sparse training more hardware-friendly.

Input: unstructured mask \(m\), density ratio of mask \(r\), number of clusters \(t\), density ratio of kernel \(s\), number of input channel \(C_{I}\), the number of output channel \(C_{O}\), the width of the kernel \(K_{w}\), the height of the kernel \(K_{h}\), and the minimum number of rows of each group \(B_{1}\) Output: Dense groups

With HRBP, we can achieve \(72.7\%\) accuracy with \(60\%\) sparsity.

Our method can also achieve comparable or even better accuracy than DessiLBI at the same sparsity level. Note that, DessiLBI follows the LTH setting which still needs dense training at the first step, while our method directly trains the sparse subnetwork from scratch without dense training. We by default apply our method to random unstructured sparse mask with ERK sparse ratio distribution (Evci et al., 2020) of each layer.

We then apply HRBP++ to these unstructured masks and show the results in Figure 13. Noticeably, with SNIP and GraSP masks, HRBP++ can also achieve comparable accuracy at the same sparsity level.

This is consistent with the conclusion in (Evci et al., 2020) that ERK sparsity is beneficial.

Input: Table 3: Static sparse training on ImageNet. The dense ResNet-50 has 75.70% top-1 accuracy.

Input: Figure 5: The layer-wise speedup of VGG-19.

Input: Figure 7: The trade-off between sparsity and accuracy. All models are evaluated on CIFAR-10.

Input: Table 2: Comparison of dynamic sparse training methods on CIFAR-10 and CIFAR-100.

Input: Figure 14: Training and inference acceleration on hardware of ResNet-32 on CIFAR-10.

Input: Figure 13: Ablation study with different types of mask initialization of ResNet-32 on CIFAR-10.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim appears somewhat overstated. While the evidence does show HRBP/HRBP++ achieving comparable accuracy to unstructured methods on the mentioned datasets, the "huge acceleration" claim needs qualification. The actual acceleration rates shown are more modest:

- ImageNet: 1.26x-1.44x speedup at 80-90% sparsity (Table 3)
- Training acceleration peaks around 1.6x at 95% sparsity (Figure 14)
- Layer-wise acceleration varies significantly (Figure 5)

The word "huge" exaggerates the actual 1.2x-2x speedups demonstrated. Additionally, reviewer comment #6 points out the relatively modest 2x speedup comes with significant parameter reduction and performance tradeoffs.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Here, we show that by simply injecting symmetric, random, noise during training in reconstruction or classification tasks, artificial neural networks with ReLU activation functions eliminate this difference; the neurons converge to a sparse coding solution where only a small fraction are active for any input.

The evidence to be evaluated is:Gaussian noise injection has been interpreted as a form of model regularization, to help avoid overfitting (Zur et al., 2009) and improve generalization (Sietsma and Dow, 1991; Matsuoka, 1992). It was later shown that noise injection is in fact equivalent to having a regularization term that seeks to minimize the \(L_{2}\) norm of the network's Jacobian (Bishop, 1995; Rifai et al., 2011; Alain and Bengio, 2014). From this perspective, it makes sense that sparsity can be used to turn off neurons and minimize this Jacobian, providing a potential explanation for our results.

However, we are unaware of any existing work which finds that noisy training with a ReLU activation function not only results in sparsity but also converges to a sparse coding, Top-K network. Learning reconstructions of training data with sparse activations is the mainstay of sparse coding (Olshausen & Field, 1997). Sparse coding uses an overcomplete basis to learn a reconstruction of data with an \(L_{1}\) regularization penalty on neuron activity.

The objective function aims to minimize the mean squared-error between the reconstructed image and the original image while having sparse neuron activity: \(\operatorname*{arg\,min}_{\Phi,\mathbf{n}}(\mathbf{x}-\hat{\mathbf{x}})^{2}+ \lambda\sum_{i}^{N}|a_{i}|\). This sparse coding task is an example of a statistical model converging with neuroscience because, in addition to producing sparse representations, the basis functions learn to become Gabor filters like those found in area V1 of the visual system (Olshausen & Field, 1997). Interestingly, the ReLU activation function that is crucial for the results of this paper can be derived as the optimal solution to a simplified sparse coding problem when we assume that the basis vectors are orthogonal (Ba, 2020; Gregor & LeCun, 2010). This results in the optimal solution for activity \(a_{i}\) to be given by \(\operatorname*{ReLU}(u_{i}-\lambda)-\operatorname*{ReLU}(-u_{i}-\lambda)\) or, if we constrain \(a_{i}\) to be non-negative, \(\operatorname*{ReLU}(u_{i}-\lambda)\), where \(u_{i}\) is the value obtained by projecting the data onto basis \(\phi_{i}\). Intuitively, if the values of our target image \(\mathbf{x}\) reconstructed by \(a_{i}\phi_{i}\) are too small, it is better for \(a_{i}=0\) than to predict \(\mathbf{x}\), where the threshold for this tradeoff is given by \(\lambda\), which behaves as an activation threshold.

These Top-K networks, when trained with noise, are also closely related to associative memory models such as Sparse Distributed Memory (SDM) and Modern Hopfield Networks that activate a sparse subset of neurons and were developed to handle noisy queries (Kanerva, 1988; Krotov & Hopfield, 2016).

By default, we train a single hidden layer ReLU network with different amounts of random Gaussian noise applied to the input. The objective is to remove the noise and reconstruct the original input. Formally, noise is applied from an isotropic Gaussian with variance \(\sigma\) to the input \(\mathbf{x}\). The corrupted image is then put through our encoder and decoder.

\[\tilde{\mathbf{x}} =\mathbf{x}+\epsilon,\ \ \epsilon\sim N(0,\sigma I)\] \[\hat{\mathbf{x}} =W_{d}\ \text{ReLU}(W_{e}\tilde{\mathbf{x}}+\mathbf{b}_{e})+ \mathbf{b}_{d}\] \[\text{loss} =\sum_{\mathbf{x}\in X}(\mathbf{x}-\hat{\mathbf{x}})^{2}. \tag{1}\] We use Kaiming randomly initialized weights (He et al., 2015) and train until the fraction of active neurons converges.

Figure 2 shows that a single hidden layer ReLU network with 10,000 neurons learns to become a Top-K network with smaller \(k\) values as the amount of noise increases. This network was trained on the CIFAR10 embeddings to mimick training within a deeper network layer and where the manifold is smoother than when training on raw image pixels. While Fig. 2 only shows the mean number of neurons that are active across all 50,000 CIFAR10 inputs per epoch, Figure 3 shows the fraction of active neurons for every input.

It is clear that these values all synchronize to a value decided on by the network and are responsible for the sparse \(k\) neurons remaining active after the ReLU nonlinearity is applied. As noise increases, the bias terms become more negative to raise the activation threshold and increase sparsity. The \(L_{2}\) norms of the key vectors also become smaller. We hypothesize this is a counter-response to the \(L_{2}\) norms of the data increasing with noise and explains why in high noise settings (\(\sigma\geq 0.8\)) neuron activations linger around \(\sim\)50% sparsity before rapidly becoming very sparse (see Fig. 2). Before the key vector \(L_{2}\) norms are sufficiently small, noisy data with large \(L_{2}\) norms produce very large neuron activations (both positive and negative) and the bias terms initialized close to 0 have no real effect on which neurons are active. Because the key vectors are randomly initialized from a symmetric distribution around 0, in expectation half of the neurons are active after the ReLU nonlinearity is applied. This means the network must wait to first shrink its key vector \(L_{2}\) norms before the bias terms can affect activation sparsity.

Looking more closely at the sparse solutions found by our ReLU networks, we discovered that for \(\sigma\geq 0.3\), neurons begin to form receptive fields reminiscent of V1 Gabor filters and on/off center surround retinal ganglion cells (Sterling & Laughlin, 2015; Olshausen & Field, 1997).

This is a similar result to that of sparse coding (Olshausen & Field, 1997), however, here we use noise with the ReLU activation function instead of a sparsity penalty and train on full images instead of 8x8 patches.

When we tried training the network without any noise but instead with an \(L_{1}\) activation penalty motivated by the original sparse coding setup (Olshausen & Field, 1997), no biological receptive fields were observed. In addition, this approach resulted in unstable training, more dead neurons, less sparsity, and no synchronization of key vector norms or bias terms.1 Footnote 1: We did not implement the full sparse coding framework, only the \(L_{1}\) penalty, however, this shows that it is insufficient to implement sparse coding while noisy training is.

With these models we find that our networks become even more sparse and do so more quickly, as we would expect due to the explaining away of redundant neurons (see Appendix C).

Investigating our noise trained sparse networks, we find that even after the noise is removed, they remain sparse.

Fig. 6 shows that the sparsity levels remain largely unperturbed.

In fact, for classification \(\sigma\in\{0.1,0.3\}\) both get 94.3% validation accuracy instead of the 93.4% that the baseline gets, showing slightly better generalization while also being more sparse (66% for baseline versus 55% for \(\sigma=0.1\) and 45% for \(\sigma=0.3\)).

**Model Ablations -** Figure 7 shows the mean neuron sparsity at network convergence across a number of different experimental conditions: (i) three datasets: CIFAR10 pixels, CIFAR10 latent embeddings, MNIST; (ii) two noise distributions: Gaussian and Laplace; (iii) four numbers of neurons: 100, 1,000, 10,000 and 100,000; (iv) three training tasks: reconstruction, classification, and Transformer next token prediction.

We found that every optimizer with a high enough learning rate converged to a sparse Top-K solution without creating dead neurons.2

We found for the latent CIFAR10 dataset, both sigmoid and GELU (Hendrycks and Gimpel, 2016) had no sparsity but while sigmoid performs worse, GELU does just as well as ReLU. If we raised our arbitrary activation threshold to \(<0.01\) we found that GELU does become sparse with respect to noise but still not as sparse as ReLU while sigmoid remains fully dense (see Appendix E).

While still neither activation function becomes as sparse as ReLU, the GELU networks implement the same Top-K convergence as ReLU shown in Fig. 3 and form biologically plausible receptive fields like those of Fig. 5.

The correlation between noise and activation sparsity remains and is again retained even after noise is removed.

Figure 7(d) shows the final sparsity amounts of the first model layer with each of the other layers giving similar results with \(\sim\)3x to \(\sim\)9x more sparsity when \(\sigma=3.0\), compared to \(\sigma=0.0\).

An increase in noise will create more erroneous low-level activations in neurons that must be ignored to maximize signal to noise ratio. Therefore, during noisy training, the model must learn weight vectors to encode information about the data distribution that can be separated from the noise using only ReLU functions.

Similarly, we hypothesize that the encoding layer in our network is learning such projections, so that thresholding performs noise rejection while also producing sparse activations.

Our initial attempt at doing so did not produce performance gains, though it did increase sparsity.

Using deep learning to give insights into neuroscience, this result suggests that in the presence of noise, the biological sparse coding solution is optimal. Moreover, the fact that noise alone results in sparse coding rather than having a sparsity penalty in the loss function suggests that the primary driver behind sparse coding may be the handling of noise with metabolic efficiency as an added benefit, rather than the other way around.

**Conclusion -** We have shown that the realistic assumption of noisy training data is sufficient for feed-forward layers to organically become highly sparse and retain their sparsity, even after noise is removed. This result is particularly striking because of the way the networks converge with sparse coding, producing biological receptive fields and approximating an inhibitory interneuron that implements a Top-K operation. Combining noisy training with other approaches to induce sparsity, including taking additional ideas from sparse coding, may result in even higher degrees of sparsity. More broadly, this work builds a new bridge between artificial and biological neural networks by showing how they can be more similar than they may otherwise appear.

Here we show additional images of the receptive fields for networks trained with different amounts of noise (Fig. 8), the queries used to activate these neurons (Fig. 9) and how the receptive fields look more biological as the network is trained for longer (Fig. 10).

Training the same networks as before but using Eq. 2 with \(t^{*}=1\) and \(\gamma=1\) we present our results in Table 1. We show the sparsity amounts of the networks in Fig. 11.

Removing the bias terms and relying just on the inhibitory interneuron, we found that it performs well (Table 1) and becomes sparse as a function of noise but results in a number of dead neurons and is not as sparse as the baseline ReLU or Explaining Away models.

Neither of these networks are as sparse as the ReLU network. Also neither network creates sparsity through dead neurons, but given that all neurons have non-zero absolute activity, having dead neurons is impossible. While we just show the mean active neurons here, both solutions look like those of Fig. 3 with the neurons for all inputs converging to a Top-K solution where the bias terms become negative and synchronize along with the key vector \(L_{2}\) norms.

Input: Table 1: Reconstruction Validation Accuracy - The baseline ReLU model varying the number of neurons, compared to the shared bias, explaining away, and inhibitory interneuron networks. We only show noise up to σ = 0.8 because the higher noise levels present trivial solutions that all networks perform equally well on. Due to very high convergence between random seeds, we just present a single run for each model.

Input: Figure 7: Training ablations. The positive correlation between noise and sparsity is robust to (a) neuron counts, (b) datasets, (c) noise distributions, and (d) training tasks.

Input: Figure 2: There is a positive relationship between activation sparsity and noise. The average fraction of neurons active during each CIFAR10 latent input over the course of training for 800 epochs. Each line corresponds to training randomly initialized networks with different noise levels σ denoted by different colors. We show the average of three different random seeds and their standard error of the mean (not visible as the variance is so low). Interestingly, the higher noise levels take longer to sparsify but upon convergence become the sparsest.

Input: Figure 3: Noise induces the formation of Top-K networks. Shown column-wise are the fraction of neurons that are active for each training input (Left), the values of each of the 10,000 neurons bias terms (Middle) and the L2 norms of their key vectors (Right), as a function of number of training epochs. Each is shown for three different representative noise levels (rows). These plots show each input to the network and use the density of blue to represent how many times a particular value occurs.

Input: Figure 5: Biological receptive fields form with noise. Networks trained on CIFAR10 pixels with σ = 0.8 Gaussian noise. The most active 125 neuron receptive fields are reshaped into 32x32x3 and re-scaled so their values span the range of pixel values [0, 255]. Neurons are sorted by their activity levels (top left is the most active, going across rows then down columns) for a random noisy input of a car. Starting at σ ≥ 0.1 and particularly for σ = 0.8 (shown here) we observe the formation of many Gabor filters and on/off center surround receptive fields that are unique to noisy training.

Input: Figure 6: Noisy pretraining retains network sparsity. The vertical purple lines denote the start of noise annealing where each noise level is linearly decayed down to σ = 0. For both reconstruction (left) and classification (right) tasks the networks remain highly sparse even after all noise is removed. Moreover, there is no noticeable performance difference between any of the networks while some of those trained with moderate noise perform slightly better than the noise free baseline. We truncate the x-axis to start at epoch 600 for the sake of clarity. The classification networks continue to sparsify even when noise annealing has started because unlike in the reconstruction task, the sparsity level did not converge within the first 800 epochs.

Input: Figure 8: Receptive fields across noise levels. We show for all noise levels the 625 most active neurons for the truck query (middle of Fig. 9). For 0.8 ≥ σ which activate fewer neurons, we also show the cat query below it. Biological like receptive fields start to appear for σ = 0.1 and can be seen most clearly for σ ∈ {0.3, 0.8, 1.5}.

Input: Figure 12: Dead Neurons during the reconstruction task. During noise annealing, the higher noise levels σ ≥ 0.8 see a number of dead neurons. However, this does not full explain their sparsity levels and on this trivial task it does not hurt their reconstruction performance.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that it is generally well-stated but contains minor overstatement. The evidence strongly supports that noise injection during training leads to sparse coding solutions in ReLU networks, with comprehensive experimental results across different conditions (Figure 7), clear visualization of receptive fields (Figure 5), and detailed analysis of network behavior (Figures 2-3). However, the claim's use of "simply injecting" slightly understates the complexity, as the evidence shows the process requires specific conditions (ReLU activation, appropriate noise levels, sufficient training time) and doesn't always work perfectly (some dead neurons occur, Figure 12).

<score>2</score>

Justification:
1. Strong experimental support:
- Comprehensive ablation studies across datasets, noise types, neuron counts, and tasks (Figure 7)
- Clear demonstration of sparsification (Figures 2-3)
- Biological receptive field emergence (Figure 5)
- Retention of sparsity after noise removal (Figure 6)

2. Minor overstatements:
- "Simply injecting" downplays requirements like:
  - Specific activation function (ReLU works better than GELU/sigmoid)
  - Appropriate noise levels (σ ≥ 0.3 for biological receptive fields)
  - Training duration considerations
- Some limitations like dead neurons (Figure 12) could be better acknowledged

3. The core claim about noise leading to sparse coding solutions is well-supported by the extensive evidence presented, making this only a minor overstatement rather than a significant one.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Here, we show that by simply injecting symmetric, random, noise during training in reconstruction or classification tasks, artificial neural networks with ReLU activation functions eliminate this difference; the neurons converge to a sparse coding solution where only a small fraction are active for any input.

The review comment to be evaluated is:

Strengths

The paper is simple and easy to follow.

The sparsity effects shown appear relatively large and robust.

Adding noise is a simple manipulation that can have benefits beyond sparsity, and so there is some appeal to using this manipulation as a way to generate sparsity.

Weaknesses

It is odd that the authors focus so much on a one-layer model. For this finding to be broadly relevant, it is important to test whether this effect occurs in large-scale DNNs and whether networks can generate sparsity while maintaining good performance on real-world tasks. The only thing that is done along these lines is with a single, small GPT2 model.

It appears you need fairly high levels of noise to generate substantial sparsity (e.g., sigma=0.3 to get below 20% activation in Figure 2, which based on Figure 4 is quite a lot of noise). It seems this might limit the practical utility of the method, though it is hard to say in part because experiments are mostly limited to the single-layer training regime.

The link with biological receptive fields is tenuous. There are multiple ways of generating center-surround and oriented/bandpass receptive fields (e.g., whitening for center-surround RFs, sparsity for oriented RFs, CNNs often exhibit similar structure at the first layer). It is not clear this approach provides a better fit to biological data than these alternatives and there is no quantification of the match to biology or model comparison. Moreover, there are so many differences between the models trained here and biological networks, that it is hard to know the relevance of this finding, particularly given that the results can vary with the activation function (e.g., the sigmoid activation function does not generate sparsity) and the optimization algorithm (e.g., SGD with a low learning rate apparently does not generate sparsity).

The evidence to be evaluated is:Gaussian noise injection has been interpreted as a form of model regularization, to help avoid overfitting (Zur et al., 2009) and improve generalization (Sietsma and Dow, 1991; Matsuoka, 1992). It was later shown that noise injection is in fact equivalent to having a regularization term that seeks to minimize the \(L_{2}\) norm of the network's Jacobian (Bishop, 1995; Rifai et al., 2011; Alain and Bengio, 2014). From this perspective, it makes sense that sparsity can be used to turn off neurons and minimize this Jacobian, providing a potential explanation for our results.

However, we are unaware of any existing work which finds that noisy training with a ReLU activation function not only results in sparsity but also converges to a sparse coding, Top-K network. Learning reconstructions of training data with sparse activations is the mainstay of sparse coding (Olshausen & Field, 1997). Sparse coding uses an overcomplete basis to learn a reconstruction of data with an \(L_{1}\) regularization penalty on neuron activity.

The objective function aims to minimize the mean squared-error between the reconstructed image and the original image while having sparse neuron activity: \(\operatorname*{arg\,min}_{\Phi,\mathbf{n}}(\mathbf{x}-\hat{\mathbf{x}})^{2}+ \lambda\sum_{i}^{N}|a_{i}|\). This sparse coding task is an example of a statistical model converging with neuroscience because, in addition to producing sparse representations, the basis functions learn to become Gabor filters like those found in area V1 of the visual system (Olshausen & Field, 1997). Interestingly, the ReLU activation function that is crucial for the results of this paper can be derived as the optimal solution to a simplified sparse coding problem when we assume that the basis vectors are orthogonal (Ba, 2020; Gregor & LeCun, 2010). This results in the optimal solution for activity \(a_{i}\) to be given by \(\operatorname*{ReLU}(u_{i}-\lambda)-\operatorname*{ReLU}(-u_{i}-\lambda)\) or, if we constrain \(a_{i}\) to be non-negative, \(\operatorname*{ReLU}(u_{i}-\lambda)\), where \(u_{i}\) is the value obtained by projecting the data onto basis \(\phi_{i}\). Intuitively, if the values of our target image \(\mathbf{x}\) reconstructed by \(a_{i}\phi_{i}\) are too small, it is better for \(a_{i}=0\) than to predict \(\mathbf{x}\), where the threshold for this tradeoff is given by \(\lambda\), which behaves as an activation threshold.

These Top-K networks, when trained with noise, are also closely related to associative memory models such as Sparse Distributed Memory (SDM) and Modern Hopfield Networks that activate a sparse subset of neurons and were developed to handle noisy queries (Kanerva, 1988; Krotov & Hopfield, 2016).

By default, we train a single hidden layer ReLU network with different amounts of random Gaussian noise applied to the input. The objective is to remove the noise and reconstruct the original input. Formally, noise is applied from an isotropic Gaussian with variance \(\sigma\) to the input \(\mathbf{x}\). The corrupted image is then put through our encoder and decoder.

\[\tilde{\mathbf{x}} =\mathbf{x}+\epsilon,\ \ \epsilon\sim N(0,\sigma I)\] \[\hat{\mathbf{x}} =W_{d}\ \text{ReLU}(W_{e}\tilde{\mathbf{x}}+\mathbf{b}_{e})+ \mathbf{b}_{d}\] \[\text{loss} =\sum_{\mathbf{x}\in X}(\mathbf{x}-\hat{\mathbf{x}})^{2}. \tag{1}\] We use Kaiming randomly initialized weights (He et al., 2015) and train until the fraction of active neurons converges.

Figure 2 shows that a single hidden layer ReLU network with 10,000 neurons learns to become a Top-K network with smaller \(k\) values as the amount of noise increases. This network was trained on the CIFAR10 embeddings to mimick training within a deeper network layer and where the manifold is smoother than when training on raw image pixels. While Fig. 2 only shows the mean number of neurons that are active across all 50,000 CIFAR10 inputs per epoch, Figure 3 shows the fraction of active neurons for every input.

It is clear that these values all synchronize to a value decided on by the network and are responsible for the sparse \(k\) neurons remaining active after the ReLU nonlinearity is applied. As noise increases, the bias terms become more negative to raise the activation threshold and increase sparsity. The \(L_{2}\) norms of the key vectors also become smaller. We hypothesize this is a counter-response to the \(L_{2}\) norms of the data increasing with noise and explains why in high noise settings (\(\sigma\geq 0.8\)) neuron activations linger around \(\sim\)50% sparsity before rapidly becoming very sparse (see Fig. 2). Before the key vector \(L_{2}\) norms are sufficiently small, noisy data with large \(L_{2}\) norms produce very large neuron activations (both positive and negative) and the bias terms initialized close to 0 have no real effect on which neurons are active. Because the key vectors are randomly initialized from a symmetric distribution around 0, in expectation half of the neurons are active after the ReLU nonlinearity is applied. This means the network must wait to first shrink its key vector \(L_{2}\) norms before the bias terms can affect activation sparsity.

Looking more closely at the sparse solutions found by our ReLU networks, we discovered that for \(\sigma\geq 0.3\), neurons begin to form receptive fields reminiscent of V1 Gabor filters and on/off center surround retinal ganglion cells (Sterling & Laughlin, 2015; Olshausen & Field, 1997).

This is a similar result to that of sparse coding (Olshausen & Field, 1997), however, here we use noise with the ReLU activation function instead of a sparsity penalty and train on full images instead of 8x8 patches.

When we tried training the network without any noise but instead with an \(L_{1}\) activation penalty motivated by the original sparse coding setup (Olshausen & Field, 1997), no biological receptive fields were observed. In addition, this approach resulted in unstable training, more dead neurons, less sparsity, and no synchronization of key vector norms or bias terms.1 Footnote 1: We did not implement the full sparse coding framework, only the \(L_{1}\) penalty, however, this shows that it is insufficient to implement sparse coding while noisy training is.

With these models we find that our networks become even more sparse and do so more quickly, as we would expect due to the explaining away of redundant neurons (see Appendix C).

Investigating our noise trained sparse networks, we find that even after the noise is removed, they remain sparse.

Fig. 6 shows that the sparsity levels remain largely unperturbed.

In fact, for classification \(\sigma\in\{0.1,0.3\}\) both get 94.3% validation accuracy instead of the 93.4% that the baseline gets, showing slightly better generalization while also being more sparse (66% for baseline versus 55% for \(\sigma=0.1\) and 45% for \(\sigma=0.3\)).

**Model Ablations -** Figure 7 shows the mean neuron sparsity at network convergence across a number of different experimental conditions: (i) three datasets: CIFAR10 pixels, CIFAR10 latent embeddings, MNIST; (ii) two noise distributions: Gaussian and Laplace; (iii) four numbers of neurons: 100, 1,000, 10,000 and 100,000; (iv) three training tasks: reconstruction, classification, and Transformer next token prediction.

We found that every optimizer with a high enough learning rate converged to a sparse Top-K solution without creating dead neurons.2

We found for the latent CIFAR10 dataset, both sigmoid and GELU (Hendrycks and Gimpel, 2016) had no sparsity but while sigmoid performs worse, GELU does just as well as ReLU. If we raised our arbitrary activation threshold to \(<0.01\) we found that GELU does become sparse with respect to noise but still not as sparse as ReLU while sigmoid remains fully dense (see Appendix E).

While still neither activation function becomes as sparse as ReLU, the GELU networks implement the same Top-K convergence as ReLU shown in Fig. 3 and form biologically plausible receptive fields like those of Fig. 5.

The correlation between noise and activation sparsity remains and is again retained even after noise is removed.

Figure 7(d) shows the final sparsity amounts of the first model layer with each of the other layers giving similar results with \(\sim\)3x to \(\sim\)9x more sparsity when \(\sigma=3.0\), compared to \(\sigma=0.0\).

An increase in noise will create more erroneous low-level activations in neurons that must be ignored to maximize signal to noise ratio. Therefore, during noisy training, the model must learn weight vectors to encode information about the data distribution that can be separated from the noise using only ReLU functions.

Similarly, we hypothesize that the encoding layer in our network is learning such projections, so that thresholding performs noise rejection while also producing sparse activations.

Our initial attempt at doing so did not produce performance gains, though it did increase sparsity.

Using deep learning to give insights into neuroscience, this result suggests that in the presence of noise, the biological sparse coding solution is optimal. Moreover, the fact that noise alone results in sparse coding rather than having a sparsity penalty in the loss function suggests that the primary driver behind sparse coding may be the handling of noise with metabolic efficiency as an added benefit, rather than the other way around.

**Conclusion -** We have shown that the realistic assumption of noisy training data is sufficient for feed-forward layers to organically become highly sparse and retain their sparsity, even after noise is removed. This result is particularly striking because of the way the networks converge with sparse coding, producing biological receptive fields and approximating an inhibitory interneuron that implements a Top-K operation. Combining noisy training with other approaches to induce sparsity, including taking additional ideas from sparse coding, may result in even higher degrees of sparsity. More broadly, this work builds a new bridge between artificial and biological neural networks by showing how they can be more similar than they may otherwise appear.

Here we show additional images of the receptive fields for networks trained with different amounts of noise (Fig. 8), the queries used to activate these neurons (Fig. 9) and how the receptive fields look more biological as the network is trained for longer (Fig. 10).

Training the same networks as before but using Eq. 2 with \(t^{*}=1\) and \(\gamma=1\) we present our results in Table 1. We show the sparsity amounts of the networks in Fig. 11.

Removing the bias terms and relying just on the inhibitory interneuron, we found that it performs well (Table 1) and becomes sparse as a function of noise but results in a number of dead neurons and is not as sparse as the baseline ReLU or Explaining Away models.

Neither of these networks are as sparse as the ReLU network. Also neither network creates sparsity through dead neurons, but given that all neurons have non-zero absolute activity, having dead neurons is impossible. While we just show the mean active neurons here, both solutions look like those of Fig. 3 with the neurons for all inputs converging to a Top-K solution where the bias terms become negative and synchronize along with the key vector \(L_{2}\) norms.

Input: Table 1: Reconstruction Validation Accuracy - The baseline ReLU model varying the number of neurons, compared to the shared bias, explaining away, and inhibitory interneuron networks. We only show noise up to σ = 0.8 because the higher noise levels present trivial solutions that all networks perform equally well on. Due to very high convergence between random seeds, we just present a single run for each model.

Input: Figure 7: Training ablations. The positive correlation between noise and sparsity is robust to (a) neuron counts, (b) datasets, (c) noise distributions, and (d) training tasks.

Input: Figure 2: There is a positive relationship between activation sparsity and noise. The average fraction of neurons active during each CIFAR10 latent input over the course of training for 800 epochs. Each line corresponds to training randomly initialized networks with different noise levels σ denoted by different colors. We show the average of three different random seeds and their standard error of the mean (not visible as the variance is so low). Interestingly, the higher noise levels take longer to sparsify but upon convergence become the sparsest.

Input: Figure 3: Noise induces the formation of Top-K networks. Shown column-wise are the fraction of neurons that are active for each training input (Left), the values of each of the 10,000 neurons bias terms (Middle) and the L2 norms of their key vectors (Right), as a function of number of training epochs. Each is shown for three different representative noise levels (rows). These plots show each input to the network and use the density of blue to represent how many times a particular value occurs.

Input: Figure 5: Biological receptive fields form with noise. Networks trained on CIFAR10 pixels with σ = 0.8 Gaussian noise. The most active 125 neuron receptive fields are reshaped into 32x32x3 and re-scaled so their values span the range of pixel values [0, 255]. Neurons are sorted by their activity levels (top left is the most active, going across rows then down columns) for a random noisy input of a car. Starting at σ ≥ 0.1 and particularly for σ = 0.8 (shown here) we observe the formation of many Gabor filters and on/off center surround receptive fields that are unique to noisy training.

Input: Figure 6: Noisy pretraining retains network sparsity. The vertical purple lines denote the start of noise annealing where each noise level is linearly decayed down to σ = 0. For both reconstruction (left) and classification (right) tasks the networks remain highly sparse even after all noise is removed. Moreover, there is no noticeable performance difference between any of the networks while some of those trained with moderate noise perform slightly better than the noise free baseline. We truncate the x-axis to start at epoch 600 for the sake of clarity. The classification networks continue to sparsify even when noise annealing has started because unlike in the reconstruction task, the sparsity level did not converge within the first 800 epochs.

Input: Figure 8: Receptive fields across noise levels. We show for all noise levels the 625 most active neurons for the truck query (middle of Fig. 9). For 0.8 ≥ σ which activate fewer neurons, we also show the cat query below it. Biological like receptive fields start to appear for σ = 0.1 and can be seen most clearly for σ ∈ {0.3, 0.8, 1.5}.

Input: Figure 12: Dead Neurons during the reconstruction task. During noise annealing, the higher noise levels σ ≥ 0.8 see a number of dead neurons. However, this does not full explain their sparsity levels and on this trivial task it does not hurt their reconstruction performance.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I assess that while there is strong evidence that noise during training leads to sparsity in ReLU networks, some aspects of the claim are overstated. The evidence shows clear patterns of increased sparsity with noise across various conditions (Fig 2, 3, 7) and emergence of biological-like receptive fields (Fig 5, 8). However, the phrase "simply injecting" downplays important nuances - high noise levels are needed (σ≥0.3) for significant effects, and the reviewer correctly notes this may limit practical utility. Additionally, the biological parallel, while interesting, is not conclusively proven.

<score>3</score>

Justification:
1. The core finding about noise inducing sparsity is well-supported through extensive experiments and ablations.
2. However, key limitations are understated:
- Requires relatively high noise levels (σ≥0.3) for substantial effects
- Reviewer points out limited testing on large-scale DNNs and real-world tasks
- Biological relevance claims need stronger quantitative validation
- Results vary with activation function and optimization choices
3. The evidence indicates the phenomenon is robust but the practical constraints and biological implications are more complex than the claim suggests.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Here, we show that by simply injecting symmetric, random, noise during training in reconstruction or classification tasks, artificial neural networks with ReLU activation functions eliminate this difference; the neurons converge to a sparse coding solution where only a small fraction are active for any input.

The review comment to be evaluated is:

Strengths

1. The authors find a simple, implicit manner to create top-k networks that compares favorably to explicit methods

2. The method is tested in several different circumstances, with different tasks, numbers of neurons, and noise levels. Simple characteristics of the network, such as the fraction of active neurons and their biases, are well described.

Weaknesses

Major

1. As the authors observe, the results are recapitulated using a global negative bias term, acting as common inhibition that requires greater input values to activate neurons and resulting in top-k networks.  Thus, while such an observation may indeed be useful for those interested in such networks, the advance it comprises is a minor one.

Minor

1. While the presented results may bear on biological considerations, as suggested therein, caution is advised, as noise in biological networks is dynamic and thus not perfectly analogous to simply adding noise to inputs of a feedforward network.

The evidence to be evaluated is:Gaussian noise injection has been interpreted as a form of model regularization, to help avoid overfitting (Zur et al., 2009) and improve generalization (Sietsma and Dow, 1991; Matsuoka, 1992). It was later shown that noise injection is in fact equivalent to having a regularization term that seeks to minimize the \(L_{2}\) norm of the network's Jacobian (Bishop, 1995; Rifai et al., 2011; Alain and Bengio, 2014). From this perspective, it makes sense that sparsity can be used to turn off neurons and minimize this Jacobian, providing a potential explanation for our results.

However, we are unaware of any existing work which finds that noisy training with a ReLU activation function not only results in sparsity but also converges to a sparse coding, Top-K network. Learning reconstructions of training data with sparse activations is the mainstay of sparse coding (Olshausen & Field, 1997). Sparse coding uses an overcomplete basis to learn a reconstruction of data with an \(L_{1}\) regularization penalty on neuron activity.

The objective function aims to minimize the mean squared-error between the reconstructed image and the original image while having sparse neuron activity: \(\operatorname*{arg\,min}_{\Phi,\mathbf{n}}(\mathbf{x}-\hat{\mathbf{x}})^{2}+ \lambda\sum_{i}^{N}|a_{i}|\). This sparse coding task is an example of a statistical model converging with neuroscience because, in addition to producing sparse representations, the basis functions learn to become Gabor filters like those found in area V1 of the visual system (Olshausen & Field, 1997). Interestingly, the ReLU activation function that is crucial for the results of this paper can be derived as the optimal solution to a simplified sparse coding problem when we assume that the basis vectors are orthogonal (Ba, 2020; Gregor & LeCun, 2010). This results in the optimal solution for activity \(a_{i}\) to be given by \(\operatorname*{ReLU}(u_{i}-\lambda)-\operatorname*{ReLU}(-u_{i}-\lambda)\) or, if we constrain \(a_{i}\) to be non-negative, \(\operatorname*{ReLU}(u_{i}-\lambda)\), where \(u_{i}\) is the value obtained by projecting the data onto basis \(\phi_{i}\). Intuitively, if the values of our target image \(\mathbf{x}\) reconstructed by \(a_{i}\phi_{i}\) are too small, it is better for \(a_{i}=0\) than to predict \(\mathbf{x}\), where the threshold for this tradeoff is given by \(\lambda\), which behaves as an activation threshold.

These Top-K networks, when trained with noise, are also closely related to associative memory models such as Sparse Distributed Memory (SDM) and Modern Hopfield Networks that activate a sparse subset of neurons and were developed to handle noisy queries (Kanerva, 1988; Krotov & Hopfield, 2016).

By default, we train a single hidden layer ReLU network with different amounts of random Gaussian noise applied to the input. The objective is to remove the noise and reconstruct the original input. Formally, noise is applied from an isotropic Gaussian with variance \(\sigma\) to the input \(\mathbf{x}\). The corrupted image is then put through our encoder and decoder.

\[\tilde{\mathbf{x}} =\mathbf{x}+\epsilon,\ \ \epsilon\sim N(0,\sigma I)\] \[\hat{\mathbf{x}} =W_{d}\ \text{ReLU}(W_{e}\tilde{\mathbf{x}}+\mathbf{b}_{e})+ \mathbf{b}_{d}\] \[\text{loss} =\sum_{\mathbf{x}\in X}(\mathbf{x}-\hat{\mathbf{x}})^{2}. \tag{1}\] We use Kaiming randomly initialized weights (He et al., 2015) and train until the fraction of active neurons converges.

Figure 2 shows that a single hidden layer ReLU network with 10,000 neurons learns to become a Top-K network with smaller \(k\) values as the amount of noise increases. This network was trained on the CIFAR10 embeddings to mimick training within a deeper network layer and where the manifold is smoother than when training on raw image pixels. While Fig. 2 only shows the mean number of neurons that are active across all 50,000 CIFAR10 inputs per epoch, Figure 3 shows the fraction of active neurons for every input.

It is clear that these values all synchronize to a value decided on by the network and are responsible for the sparse \(k\) neurons remaining active after the ReLU nonlinearity is applied. As noise increases, the bias terms become more negative to raise the activation threshold and increase sparsity. The \(L_{2}\) norms of the key vectors also become smaller. We hypothesize this is a counter-response to the \(L_{2}\) norms of the data increasing with noise and explains why in high noise settings (\(\sigma\geq 0.8\)) neuron activations linger around \(\sim\)50% sparsity before rapidly becoming very sparse (see Fig. 2). Before the key vector \(L_{2}\) norms are sufficiently small, noisy data with large \(L_{2}\) norms produce very large neuron activations (both positive and negative) and the bias terms initialized close to 0 have no real effect on which neurons are active. Because the key vectors are randomly initialized from a symmetric distribution around 0, in expectation half of the neurons are active after the ReLU nonlinearity is applied. This means the network must wait to first shrink its key vector \(L_{2}\) norms before the bias terms can affect activation sparsity.

Looking more closely at the sparse solutions found by our ReLU networks, we discovered that for \(\sigma\geq 0.3\), neurons begin to form receptive fields reminiscent of V1 Gabor filters and on/off center surround retinal ganglion cells (Sterling & Laughlin, 2015; Olshausen & Field, 1997).

This is a similar result to that of sparse coding (Olshausen & Field, 1997), however, here we use noise with the ReLU activation function instead of a sparsity penalty and train on full images instead of 8x8 patches.

When we tried training the network without any noise but instead with an \(L_{1}\) activation penalty motivated by the original sparse coding setup (Olshausen & Field, 1997), no biological receptive fields were observed. In addition, this approach resulted in unstable training, more dead neurons, less sparsity, and no synchronization of key vector norms or bias terms.1 Footnote 1: We did not implement the full sparse coding framework, only the \(L_{1}\) penalty, however, this shows that it is insufficient to implement sparse coding while noisy training is.

With these models we find that our networks become even more sparse and do so more quickly, as we would expect due to the explaining away of redundant neurons (see Appendix C).

Investigating our noise trained sparse networks, we find that even after the noise is removed, they remain sparse.

Fig. 6 shows that the sparsity levels remain largely unperturbed.

In fact, for classification \(\sigma\in\{0.1,0.3\}\) both get 94.3% validation accuracy instead of the 93.4% that the baseline gets, showing slightly better generalization while also being more sparse (66% for baseline versus 55% for \(\sigma=0.1\) and 45% for \(\sigma=0.3\)).

**Model Ablations -** Figure 7 shows the mean neuron sparsity at network convergence across a number of different experimental conditions: (i) three datasets: CIFAR10 pixels, CIFAR10 latent embeddings, MNIST; (ii) two noise distributions: Gaussian and Laplace; (iii) four numbers of neurons: 100, 1,000, 10,000 and 100,000; (iv) three training tasks: reconstruction, classification, and Transformer next token prediction.

We found that every optimizer with a high enough learning rate converged to a sparse Top-K solution without creating dead neurons.2

We found for the latent CIFAR10 dataset, both sigmoid and GELU (Hendrycks and Gimpel, 2016) had no sparsity but while sigmoid performs worse, GELU does just as well as ReLU. If we raised our arbitrary activation threshold to \(<0.01\) we found that GELU does become sparse with respect to noise but still not as sparse as ReLU while sigmoid remains fully dense (see Appendix E).

While still neither activation function becomes as sparse as ReLU, the GELU networks implement the same Top-K convergence as ReLU shown in Fig. 3 and form biologically plausible receptive fields like those of Fig. 5.

The correlation between noise and activation sparsity remains and is again retained even after noise is removed.

Figure 7(d) shows the final sparsity amounts of the first model layer with each of the other layers giving similar results with \(\sim\)3x to \(\sim\)9x more sparsity when \(\sigma=3.0\), compared to \(\sigma=0.0\).

An increase in noise will create more erroneous low-level activations in neurons that must be ignored to maximize signal to noise ratio. Therefore, during noisy training, the model must learn weight vectors to encode information about the data distribution that can be separated from the noise using only ReLU functions.

Similarly, we hypothesize that the encoding layer in our network is learning such projections, so that thresholding performs noise rejection while also producing sparse activations.

Our initial attempt at doing so did not produce performance gains, though it did increase sparsity.

Using deep learning to give insights into neuroscience, this result suggests that in the presence of noise, the biological sparse coding solution is optimal. Moreover, the fact that noise alone results in sparse coding rather than having a sparsity penalty in the loss function suggests that the primary driver behind sparse coding may be the handling of noise with metabolic efficiency as an added benefit, rather than the other way around.

**Conclusion -** We have shown that the realistic assumption of noisy training data is sufficient for feed-forward layers to organically become highly sparse and retain their sparsity, even after noise is removed. This result is particularly striking because of the way the networks converge with sparse coding, producing biological receptive fields and approximating an inhibitory interneuron that implements a Top-K operation. Combining noisy training with other approaches to induce sparsity, including taking additional ideas from sparse coding, may result in even higher degrees of sparsity. More broadly, this work builds a new bridge between artificial and biological neural networks by showing how they can be more similar than they may otherwise appear.

Here we show additional images of the receptive fields for networks trained with different amounts of noise (Fig. 8), the queries used to activate these neurons (Fig. 9) and how the receptive fields look more biological as the network is trained for longer (Fig. 10).

Training the same networks as before but using Eq. 2 with \(t^{*}=1\) and \(\gamma=1\) we present our results in Table 1. We show the sparsity amounts of the networks in Fig. 11.

Removing the bias terms and relying just on the inhibitory interneuron, we found that it performs well (Table 1) and becomes sparse as a function of noise but results in a number of dead neurons and is not as sparse as the baseline ReLU or Explaining Away models.

Neither of these networks are as sparse as the ReLU network. Also neither network creates sparsity through dead neurons, but given that all neurons have non-zero absolute activity, having dead neurons is impossible. While we just show the mean active neurons here, both solutions look like those of Fig. 3 with the neurons for all inputs converging to a Top-K solution where the bias terms become negative and synchronize along with the key vector \(L_{2}\) norms.

Input: Table 1: Reconstruction Validation Accuracy - The baseline ReLU model varying the number of neurons, compared to the shared bias, explaining away, and inhibitory interneuron networks. We only show noise up to σ = 0.8 because the higher noise levels present trivial solutions that all networks perform equally well on. Due to very high convergence between random seeds, we just present a single run for each model.

Input: Figure 7: Training ablations. The positive correlation between noise and sparsity is robust to (a) neuron counts, (b) datasets, (c) noise distributions, and (d) training tasks.

Input: Figure 2: There is a positive relationship between activation sparsity and noise. The average fraction of neurons active during each CIFAR10 latent input over the course of training for 800 epochs. Each line corresponds to training randomly initialized networks with different noise levels σ denoted by different colors. We show the average of three different random seeds and their standard error of the mean (not visible as the variance is so low). Interestingly, the higher noise levels take longer to sparsify but upon convergence become the sparsest.

Input: Figure 3: Noise induces the formation of Top-K networks. Shown column-wise are the fraction of neurons that are active for each training input (Left), the values of each of the 10,000 neurons bias terms (Middle) and the L2 norms of their key vectors (Right), as a function of number of training epochs. Each is shown for three different representative noise levels (rows). These plots show each input to the network and use the density of blue to represent how many times a particular value occurs.

Input: Figure 5: Biological receptive fields form with noise. Networks trained on CIFAR10 pixels with σ = 0.8 Gaussian noise. The most active 125 neuron receptive fields are reshaped into 32x32x3 and re-scaled so their values span the range of pixel values [0, 255]. Neurons are sorted by their activity levels (top left is the most active, going across rows then down columns) for a random noisy input of a car. Starting at σ ≥ 0.1 and particularly for σ = 0.8 (shown here) we observe the formation of many Gabor filters and on/off center surround receptive fields that are unique to noisy training.

Input: Figure 6: Noisy pretraining retains network sparsity. The vertical purple lines denote the start of noise annealing where each noise level is linearly decayed down to σ = 0. For both reconstruction (left) and classification (right) tasks the networks remain highly sparse even after all noise is removed. Moreover, there is no noticeable performance difference between any of the networks while some of those trained with moderate noise perform slightly better than the noise free baseline. We truncate the x-axis to start at epoch 600 for the sake of clarity. The classification networks continue to sparsify even when noise annealing has started because unlike in the reconstruction task, the sparsity level did not converge within the first 800 epochs.

Input: Figure 8: Receptive fields across noise levels. We show for all noise levels the 625 most active neurons for the truck query (middle of Fig. 9). For 0.8 ≥ σ which activate fewer neurons, we also show the cat query below it. Biological like receptive fields start to appear for σ = 0.1 and can be seen most clearly for σ ∈ {0.3, 0.8, 1.5}.

Input: Figure 12: Dead Neurons during the reconstruction task. During noise annealing, the higher noise levels σ ≥ 0.8 see a number of dead neurons. However, this does not full explain their sparsity levels and on this trivial task it does not hurt their reconstruction performance.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "Here, we show that by simply injecting symmetric, random, noise during training in reconstruction or classification tasks, artificial neural networks with ReLU activation functions eliminate this difference; the neurons converge to a sparse coding solution where only a small fraction are active for any input."

The evidence strongly supports this claim through comprehensive experimental results. The authors demonstrate:
- Clear correlation between noise and sparsity across multiple conditions (Fig 7)
- Consistent convergence to Top-K networks (Fig 2, 3)
- Retention of sparsity even after noise removal (Fig 6)
- Robustness across datasets, architectures, and tasks
- Formation of biologically plausible receptive fields (Fig 5, 8)

The claim is well-stated and accurately reflects the evidence without exaggeration. The authors acknowledge limitations and provide extensive experimental validation.

<score>1</score>