Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Deep surrogate assisted generation of environments.

The review comment to be evaluated is:

### Strengths
- Open-endedness is an important, underexplored topic in ML research.
- The paper provides an interesting extension of EPOET that allows agent topologies to be co-evolved with the environment.
- The paper proposes two new transfer mechanisms for EPOET that are required for applying NEAT in updating the agent population, both of which serve as novel extensions to EPOET's standard transfer mechanism.

### Weaknesses
- It seems the comparison to EPOET is unfair because **NEAT requires ATEP to effectively maintain more agents than EPOET**. It is not clear if this paper controls for the larger population size of ATEP.
- The main experimental results (e.g. Figure 2, Figure 5) are only based on two training runs.
- The ANNECS values in Figure 2 for EPOET do not seem to match that from the original paper, where at 20k updates, EPOET reaches an ANNECS score of 100, thus matching the performance of SBT-ATEP in this work. This may be due to the authors reduction of the number of active environments from 40 to 20 with respect to the original study. However, the authors should run their implementation of EPOET on the original setting of 40 environments on 3 seeds to ensure their implementation is correct, or otherwise state that they reuse the official open source implementation of EPOET.
- The agent network size seems to play a significant role in improved ANNECS and robustness. Therefore, authors should also provide more detailed discussion on how the choice of 20x20 and 40x40 compares to the original EPOET implementation, which achieved much higher ANNECS scores in the same number of updates.
- The paper is unclear about several key details, described in the Clarity section.
- The paper claims the original POET work uses BipedalWalkerHardcore, but this is inaccurate. POET and EPOET used modifications of the BipedalWalker environment, not the specific BipedalWalkerHardcore environment, which refers to a specific setting of the BipedalWalker environment parameters.
- At the top of Page 6, the authors claim SBT-ATEP does not stagnate, but this does not follow from the results, because it is unknown if SBT-ATEP will stagnate beyond the number of training steps investigated.
- The paper does not provide the correct citation for the VAE. It should be Kingma, Diederik P., and Max Welling. "Auto-encoding variational bayes." 2013.
- The introduction mentions prior works are limited in open-endedness due to the finiteness of the environment. However, the authors do not state this shared weakness in their own experimental environment.
- The authors do not provide any measures of environment complexity over the course of training, e.g. as done in prior works. This would be useful for comparing the difference in kinds of environments produced by ATEP vs. EPOET.

The evidence to be evaluated is:Input : Candidate population's best individual \(I\), a function \(Score(.)\) that calculates the maximum of the target agent's 5 most recent fitness scores. Let \(M\) = All environments - {Candidate environment} foreach\(m\) in Mdo  Compute direct transfer \(I_{D}\); if\(I_{D}>Score(m)\)then  Compute fine-tuning transfer \(I_{P}\); if\(I_{P}>Score(m)\)then  Add m to \(T\_candidates\) else  Transfer not possible  end if else  Transfer not possible  end if  end foreach  Delete whole population of \(T\_candidates\)  Transfer whole candidate population to \(T\_candidates\)

_Performance of All Transferred Agents-Environment Characterization_ (PATA-EC) is introduced by EPOET as a domain-general environment characterization score. PATA-EC is a score that describes the environment behaviour through all active and archived agents behaviour on the environment. Four steps to calculate PATA-EC are: (1) **Evaluate**: Each created environment evaluates all agents and stores their raw scores in a vector.

Input: Figure 10: From left to right: NNs at iteration 4000 and 6000 of when it was created.

Input: Figure 1: A flowchart demonstrating the flow of the ATEP framework, with blocks in green being where ATEP differs from POET. For both EPOET and ATEP, each environment is associated with an agent, represented by an ES population for EPOET and a NEAT population for ATEP. Mechanisms used in NEAT are descrtibed in Section 3 while the hyperparameters are in Table 3 of Appendix C. PATA-EC and ANNECS are described in Section 4 with other components used in EPOET. Section 5.1 describes the transfer mechanisms and Appendix D illustrates pseudocodes for the transfer mechanisms used in ATEP. The environment images used in the chart were created by ATEP.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the provided evidence and review comments, I'll evaluate the claim "Deep surrogate assisted generation of environments."

This claim appears somewhat overstated based on several factors:

1. The evidence shows a framework (ATEP) that modifies EPOET with NEAT for environment generation, but doesn't clearly demonstrate "deep surrogate" aspects. The figures show neural network architectures and a flowchart, but don't specifically illustrate deep surrogate assistance.

2. The reviewer points out several limitations:
- Results are based on only two training runs
- Unclear comparison controls with EPOET
- Implementation details and environment complexity measures are missing
- Potential discrepancies with original EPOET results

The claim would be better stated by specifically referencing NEAT-based environment generation or evolutionary environment generation, rather than implying deep surrogate assistance.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Deep surrogate assisted generation of environments.

The review comment to be evaluated is:

1. the paper should be reorganized in the sake of readability.

For example "EPOET improves upon POET by adding in two algorithmic improvements" (1) ... (2).
What section does example (1) and (2)? If those the main contributions and improvements, one would expect to see them explicitly in the text.

2. a few relevant works are not mentioned. E.g., "Task-Agnostic Morphology Evolution" and similar.

3. I am not sure that a method can be called reproducible if it takes 200000 CPU hours. A simple examples clarifying the main points one by one will be helpful.

4. "Accumulated Number of Novel Environments Created and Solved (ANNECS), a metric for open-ended learning that, intuitively, describes the amount of interesting new...". What is the definition of "interesting"?
is Novel and interesting the same?

The evidence to be evaluated is:Input : Candidate population's best individual \(I\), a function \(Score(.)\) that calculates the maximum of the target agent's 5 most recent fitness scores. Let \(M\) = All environments - {Candidate environment} foreach\(m\) in Mdo  Compute direct transfer \(I_{D}\); if\(I_{D}>Score(m)\)then  Compute fine-tuning transfer \(I_{P}\); if\(I_{P}>Score(m)\)then  Add m to \(T\_candidates\) else  Transfer not possible  end if else  Transfer not possible  end if  end foreach  Delete whole population of \(T\_candidates\)  Transfer whole candidate population to \(T\_candidates\)

_Performance of All Transferred Agents-Environment Characterization_ (PATA-EC) is introduced by EPOET as a domain-general environment characterization score. PATA-EC is a score that describes the environment behaviour through all active and archived agents behaviour on the environment. Four steps to calculate PATA-EC are: (1) **Evaluate**: Each created environment evaluates all agents and stores their raw scores in a vector.

Input: Figure 10: From left to right: NNs at iteration 4000 and 6000 of when it was created.

Input: Figure 1: A flowchart demonstrating the flow of the ATEP framework, with blocks in green being where ATEP differs from POET. For both EPOET and ATEP, each environment is associated with an agent, represented by an ES population for EPOET and a NEAT population for ATEP. Mechanisms used in NEAT are descrtibed in Section 3 while the hyperparameters are in Table 3 of Appendix C. PATA-EC and ANNECS are described in Section 4 with other components used in EPOET. Section 5.1 describes the transfer mechanisms and Appendix D illustrates pseudocodes for the transfer mechanisms used in ATEP. The environment images used in the chart were created by ATEP.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and review comments, I'll evaluate the claim "Deep surrogate assisted generation of environments":

This is a quite overstated claim based on the available evidence. While the figures and pseudocode show a sophisticated environment generation system (ATEP framework and PATA-EC scoring), several key issues arise:

1. The claim is too broad and vague, not specifying what "deep surrogate" means in this context
2. The reviewer points out critical reproducibility issues (200,000 CPU hours)
3. Key terms like "interesting" and "novel" are not well-defined, as noted in the review
4. The evidence focuses more on the framework implementation than demonstrating surrogate assistance
5. Missing relevant work comparisons mentioned by reviewer

The claim would be better stated by specifically describing the actual mechanisms used rather than using broad terminology.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Deep surrogate assisted generation of environments.

The review comment to be evaluated is:

1. The paper lacks sufficient insights/results/illustrations to understand how the agent topologies are changing, most of the methodology and evaluation is borrowed from EPOET whose focus was more on novel environment generation rather than agent topology, so the results and discussion seem inadequate. Further EPOET already introduced the idea of using NEAT for environment generation which the authors have adapted for topology evolution.

2. Statistical significance of the results is poor, only 2 seeds have been used for the comparisons. Performance of RL based agents is typically high variance, so it is not justified to compare the agent policies/performance with so less data.

3. Currently the work comes short on novelty and lacks reusable insights that can actually be informed for progressing open ended learning. Are there any interesting patterns in the way the topologies evolve? What insights can be drawn about choosing the hyperparameters for fitness and mutation for open ended learning?

4. To me, the empirical evaluation seems lacking as no adequate baseline for topology evolution is used.

The evidence to be evaluated is:Input : Candidate population's best individual \(I\), a function \(Score(.)\) that calculates the maximum of the target agent's 5 most recent fitness scores. Let \(M\) = All environments - {Candidate environment} foreach\(m\) in Mdo  Compute direct transfer \(I_{D}\); if\(I_{D}>Score(m)\)then  Compute fine-tuning transfer \(I_{P}\); if\(I_{P}>Score(m)\)then  Add m to \(T\_candidates\) else  Transfer not possible  end if else  Transfer not possible  end if  end foreach  Delete whole population of \(T\_candidates\)  Transfer whole candidate population to \(T\_candidates\)

_Performance of All Transferred Agents-Environment Characterization_ (PATA-EC) is introduced by EPOET as a domain-general environment characterization score. PATA-EC is a score that describes the environment behaviour through all active and archived agents behaviour on the environment. Four steps to calculate PATA-EC are: (1) **Evaluate**: Each created environment evaluates all agents and stores their raw scores in a vector.

Input: Figure 10: From left to right: NNs at iteration 4000 and 6000 of when it was created.

Input: Figure 1: A flowchart demonstrating the flow of the ATEP framework, with blocks in green being where ATEP differs from POET. For both EPOET and ATEP, each environment is associated with an agent, represented by an ES population for EPOET and a NEAT population for ATEP. Mechanisms used in NEAT are descrtibed in Section 3 while the hyperparameters are in Table 3 of Appendix C. PATA-EC and ANNECS are described in Section 4 with other components used in EPOET. Section 5.1 describes the transfer mechanisms and Appendix D illustrates pseudocodes for the transfer mechanisms used in ATEP. The environment images used in the chart were created by ATEP.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence and review comments provided, I'll evaluate the claim "Deep surrogate assisted generation of environments":

The claim appears overstated given the available evidence and reviewer feedback. Key issues include:
- Limited statistical significance (only 2 seeds used)
- Lack of adequate baselines for topology evolution
- Insufficient analysis of agent topology changes
- Heavy reliance on EPOET's methodology without significant novel contributions
- Missing insights about topology evolution patterns and hyperparameter selection

While the paper presents some implementation details and neural network visualizations, it doesn't fully justify the broad claim about deep surrogate assisted generation, especially given the limitations pointed out in the reviews.

<score>4</score>

The high score (4/5) indicates significant overstatement, as the evidence and reviewer comments reveal substantial gaps between the broad claim and what's actually demonstrated through the research.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this study suggest the reversed proposal - that common NN models used can learn useful representation even without modifying the final output layer, which often holds a large number of parameters that grows linearly with number of classes.

The evidence to be evaluated is:Despite the enormous number of trainable parameters these layers added to the model, they are known to have a rather marginal impact on the final performance of the network (Zeiler & Fergus, 2014) and are easily compressed and reduced after a model was trained by simple means such as matrix decomposition and sparsification (Han et al., 2015). Further more, modern architecture choices are characterized with the removal of most of the fully connected layers (Lin et al., 2013; Szegedy et al., 2015; He et al., 2016), which was found to lead to better generalization and overall accuracy, together with a huge decrease in the number of trainable parameters. Additionally, numerous works showed that CNNs can be trained in a metric learning regime (Bromley et al., 1994; Schroff et al., 2015; Hoffer & Ailon, 2015), where no explicit classification layer was introduced and the objective regarded only distance measures between intermediate representations. Hardt & Ma (2017) suggested an all-convolutional network variant, where they kept the original initialization of the classification layer fixed with no negative impact on performance on the Cifar10 dataset. All of these properties provide evidence that fully-connected layers are in fact redundant and play a small role in learning and generalization.

In this work we claim that for common use-cases of convolutional network, the parameters used for the final classification transform are completely redundant, and can be replaced with a pre-determined linear transform.

We suggest that existing models can, with no other modification, devoid their classifier weights, which can help the deployment of those models in devices with low computation ability and smaller memory capacity.

The use of a fixed transform which does not depend on the number classes can allow models to scale to a large number of possible outputs, without a linear cost in the number of parameters. We also suggest that these finding might shed light on the importance of the preceding non-linear layers to learning and generalization. We focus our attention on the final representation obtained by the network (the last hidden layer), before the classifier. We denote these representation as \(x=F(z;\theta)\) where \(F\) is assumed to be a deep neural network with input \(z\) and parameters \(\theta\), e.g., a convolutional network, trained by back-propagation. In common NN models, this representation is followed by an additional affine transformation \[y=W^{T}x+b\] where \(W\) and \(b\) are also trained by back-propagation.

To evaluate our conjecture regarding the importance of the final classification transformation, we replaced the trainable parameter matrix \(W\) with a fixed orthonormal projection \(Q\in\mathbb{R}^{N\times C}\), such that \(\forall i\neq j:q_{i}\cdot q_{j}=0\) and \(\|q_{i}\|_{2}=1\), where \(q_{i}\) is the \(i\)th column of \(Q\). This can be ensured by a simple random sampling and singular-value decomposition

We keep the additional vector of bias parameters \(b\in\mathbb{R}^{C}\), and train using the same negative-log-likelihood criterion. More explicitly, our classifier output is now \[v_{i}=\frac{e^{\alpha q_{i}\cdot\hat{x}+b_{i}}}{\sum_{j}^{C}e^{\alpha q_{j} \cdot\hat{x}+b_{j}}},\,i\in\{1,\ldots,C\}\] and we minimize the loss: \[\mathcal{L}(x,t)=-\alpha q_{t}\cdot\frac{x}{\|x\|_{2}}+b_{t}+\log\left(\sum_{ i=1}^{C}\exp\left(\alpha q_{i}\cdot\frac{x}{\|x\|_{2}}+b_{i}\right)\right)\] where we recall \(x\) is the final representation obtained by the network for a specific sample, and \(t\in\{1,\ldots,C\}\) is the ground-truth label for that sample. Observing the behavior of the \(\alpha\) parameter over time revealed a logarithmic growth depicted in graph 1. Interestingly, this is the same behavior exhibited by the norm of a learned classifier, first described by Hoffer et al. (2017) and linked to the generalization of the network. This was recently explained by the under-review work of Soudry et al. (2018) as convergence to a max margin classifier. We suggest that using a single parameter will enable a simpler examination and possible further exploration of this phenomenon and its implications.

allowing to discard the softmax function and its scale altogether, but resulting in a slight decrease in final validation accuracy compared to original models. We further suggest the use of a Hadamard matrix (Hedayat et al., 1978) as the final classification transform.

We can use a truncated Hadamard matrix \(\hat{H}\in\{-1,1\}^{C\times N}\) where all \(C\) rows are orthogonal as our final classification layer such that

This usage allows two main benefits:* A deterministic, low-memory and easily generated matrix that can be used to classify. * as multiplying by a Hadamard matrix can be done by simple sign manipulation and addition.

We also note the similarity of using a Hadamard matrix as a final classifier to methods of weight binarization such as the one suggested by Courbariaux et al. (2015). As the classifier weights are fixed to need only 1-bit precision, it is now possible to focus our attention on the features preceding it.

We trained a residual network of He et al. (2016) on the Cifar10 dataset. We used a network of depth 56 and the same hyper-parameters used in the original work.

The results shown in figure 2 demonstrate that although the training error is considerably lower for the network with learned classifier, both models achieve the same classification accuracy on the validation set. Our conjecture is that with our new fixed parameterization, the network can no longer increase the norm of a given sample's representation - thus learning its label requires more effort.

As can be seen, similar validation accuracy can be obtained using a fixed scale value (in this case \(\alpha=1\) or \(10\) will suffice) at the expense of another hyper-parameter to seek.

Validation accuracy for the fixed-classifier model remained equally good as the original model, and we continued to observe the same training curve.

By using a fixed classifier, approximately 2-million parameters were removed from the model, accounting for about 8% of the model parameters. Following the same procedure, we trained a Densenet169 model (Huang et al., 2017) for which a fixed classifier reduced about \(12\%\) of the parameters.

Furthermore, we were interested in evaluating more challenging models where the classifier parameters constitutes the majority amount. For this reason we chose the Shufflenet architecture (Zhang et al., 2017), which was designed to be used in low memory and limited computing platforms. The Shufflenet network contains about 1.8 million parameters, out of which 0.96 million are part of the final classifier. Fixing the classifier resulted with a model with only 0.86 million parameters. This model was trained and found, again, to converge to similar validation accuracy as the original. Interestingly, this method allowed Imagenet training in an under-specified regime, where there are more training samples than number of parameters. This is an unconventional regime for modern deep networks, which are usually over-specified to have many more parameters than training samples (Zhang et al., 2017).

As language modeling requires classification of all possible tokens available in the task vocabulary, we were interested to see if a fixed classifier can be used, possible saving a very large number of trainable parameters (vocabulary size can have tens or even hundreds of thousands of different words). Recent works have already found empirically that using the same weights for both word embedding and classifier can yield equal or better results than using a separate pair of weights (Inan et al., 2016; Press & Wolf, 2017; Vaswani et al., 2017). This is compliant with our findings that the linear classifier is largely redundant. To examine further reduction in the number of parameters, we removed both classifier and embedding weights and replaced them with a fixed transform.

We find that using fixed word2vec embeddings, we achieve much better results. Specifically, we use \(89\%\) less parameters than the fully learned model, and obtain only somewhat worse perplexity. We argue that this implies a required structure in word embedding that stems from semantic relatedness between words and the natural imbalance between classes. However, we suggest that with a much more cost effective ways to train word embeddings (e.g., Mikolov et al. (2013)), we can narrow the gap and avoid their cost when training bigger models.

Therefore the computational demands of the final classifier will increase as well and should be considered no less than the architecture chosen.

Our work can help considerably in this kind of scenario - where using a fixed classifier removes the need to do any gradient synchronization for the final layer.

We argue that our method works due to the ability of preceding layers in the network to learn separable representations that are easily classified even when the classifier itself is fixed.

In both scenarios, our method converged similarly to a fully learned classifier reaching the same final validation accuracy.

Understanding that linear classifiers used in NN models are largely redundant allows us to consider new approaches in training and understanding these models.

In this work we suggested removing the parameters from the classification layer used in deep neural networks. We showed empirical results suggesting that keeping the classifier fixed cause little or no decline in classification performance for common balanced datasets such as Cifar and Imagenet, while allowing a noticeable reduction in trainable parameters. We argue that fixing the last layer can reduce the computational complexity for training as well as the communication cost in distributed learning.

As datasets tend to become more complex by time (e.g., Cifar100, ImageNet1K, ImageNet22k, JFT-300M, and language modeling) we believe that resource hungry affine transformation should remain fixed during training, at least partially.

Based on these findings, we recommend future research to focus on representations learned by the non-linear part of neural networks - up to the final classifier, as it seems to be highly redundant.

Input: Figure 2: Comparing training and validation error of fixed and learned classifier (ResNet56, Cifar10)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim to be well-stated and supported by extensive empirical evidence. The authors provide comprehensive experimental results showing that:
1. Fixed classifiers achieve similar validation accuracy as learned classifiers across multiple architectures (ResNet56, DenseNet169, ShuffleNet)
2. The graphs in Figure 2 clearly demonstrate equivalent validation performance between fixed and learned classifiers
3. Quantitative evidence shows significant parameter reduction (8-12% for large models, up to 89% for language models) without major performance degradation
4. The claim acknowledges limitations and is carefully scoped to "common NN models"

The authors support their claim with thorough experimentation and reasonable interpretation of results.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this study suggest the reversed proposal - that common NN models used can learn useful representation even without modifying the final output layer, which often holds a large number of parameters that grows linearly with number of classes.

The review comment to be evaluated is:

This paper proposes replacing the weights of the final classifier layer in a CNN with a fixed projection matrix.  In particular a Hadamard matrix can be used, which can be represented implicitly.

I'd have liked to see some discussion of how to efficiently implement the Hadamard transform when the number of penultimate features does not match the number of classes, since the provided code does not do this.

How does this approach scale as the number of classes grows very large (as it would in language modeling, for example)?

An interesting experiment to do here would be to look this technique interacts with distillation, when used in the teacher or student network or both.   Does fixing the features make it more difficult to place dog than on boat when classifying a cat?  Do networks with fixed classifier weights make worse teachers for distillation?

The evidence to be evaluated is:Despite the enormous number of trainable parameters these layers added to the model, they are known to have a rather marginal impact on the final performance of the network (Zeiler & Fergus, 2014) and are easily compressed and reduced after a model was trained by simple means such as matrix decomposition and sparsification (Han et al., 2015). Further more, modern architecture choices are characterized with the removal of most of the fully connected layers (Lin et al., 2013; Szegedy et al., 2015; He et al., 2016), which was found to lead to better generalization and overall accuracy, together with a huge decrease in the number of trainable parameters. Additionally, numerous works showed that CNNs can be trained in a metric learning regime (Bromley et al., 1994; Schroff et al., 2015; Hoffer & Ailon, 2015), where no explicit classification layer was introduced and the objective regarded only distance measures between intermediate representations. Hardt & Ma (2017) suggested an all-convolutional network variant, where they kept the original initialization of the classification layer fixed with no negative impact on performance on the Cifar10 dataset. All of these properties provide evidence that fully-connected layers are in fact redundant and play a small role in learning and generalization.

In this work we claim that for common use-cases of convolutional network, the parameters used for the final classification transform are completely redundant, and can be replaced with a pre-determined linear transform.

We suggest that existing models can, with no other modification, devoid their classifier weights, which can help the deployment of those models in devices with low computation ability and smaller memory capacity.

The use of a fixed transform which does not depend on the number classes can allow models to scale to a large number of possible outputs, without a linear cost in the number of parameters. We also suggest that these finding might shed light on the importance of the preceding non-linear layers to learning and generalization. We focus our attention on the final representation obtained by the network (the last hidden layer), before the classifier. We denote these representation as \(x=F(z;\theta)\) where \(F\) is assumed to be a deep neural network with input \(z\) and parameters \(\theta\), e.g., a convolutional network, trained by back-propagation. In common NN models, this representation is followed by an additional affine transformation \[y=W^{T}x+b\] where \(W\) and \(b\) are also trained by back-propagation.

To evaluate our conjecture regarding the importance of the final classification transformation, we replaced the trainable parameter matrix \(W\) with a fixed orthonormal projection \(Q\in\mathbb{R}^{N\times C}\), such that \(\forall i\neq j:q_{i}\cdot q_{j}=0\) and \(\|q_{i}\|_{2}=1\), where \(q_{i}\) is the \(i\)th column of \(Q\). This can be ensured by a simple random sampling and singular-value decomposition

We keep the additional vector of bias parameters \(b\in\mathbb{R}^{C}\), and train using the same negative-log-likelihood criterion. More explicitly, our classifier output is now \[v_{i}=\frac{e^{\alpha q_{i}\cdot\hat{x}+b_{i}}}{\sum_{j}^{C}e^{\alpha q_{j} \cdot\hat{x}+b_{j}}},\,i\in\{1,\ldots,C\}\] and we minimize the loss: \[\mathcal{L}(x,t)=-\alpha q_{t}\cdot\frac{x}{\|x\|_{2}}+b_{t}+\log\left(\sum_{ i=1}^{C}\exp\left(\alpha q_{i}\cdot\frac{x}{\|x\|_{2}}+b_{i}\right)\right)\] where we recall \(x\) is the final representation obtained by the network for a specific sample, and \(t\in\{1,\ldots,C\}\) is the ground-truth label for that sample. Observing the behavior of the \(\alpha\) parameter over time revealed a logarithmic growth depicted in graph 1. Interestingly, this is the same behavior exhibited by the norm of a learned classifier, first described by Hoffer et al. (2017) and linked to the generalization of the network. This was recently explained by the under-review work of Soudry et al. (2018) as convergence to a max margin classifier. We suggest that using a single parameter will enable a simpler examination and possible further exploration of this phenomenon and its implications.

allowing to discard the softmax function and its scale altogether, but resulting in a slight decrease in final validation accuracy compared to original models. We further suggest the use of a Hadamard matrix (Hedayat et al., 1978) as the final classification transform.

We can use a truncated Hadamard matrix \(\hat{H}\in\{-1,1\}^{C\times N}\) where all \(C\) rows are orthogonal as our final classification layer such that

This usage allows two main benefits:* A deterministic, low-memory and easily generated matrix that can be used to classify. * as multiplying by a Hadamard matrix can be done by simple sign manipulation and addition.

We also note the similarity of using a Hadamard matrix as a final classifier to methods of weight binarization such as the one suggested by Courbariaux et al. (2015). As the classifier weights are fixed to need only 1-bit precision, it is now possible to focus our attention on the features preceding it.

We trained a residual network of He et al. (2016) on the Cifar10 dataset. We used a network of depth 56 and the same hyper-parameters used in the original work.

The results shown in figure 2 demonstrate that although the training error is considerably lower for the network with learned classifier, both models achieve the same classification accuracy on the validation set. Our conjecture is that with our new fixed parameterization, the network can no longer increase the norm of a given sample's representation - thus learning its label requires more effort.

As can be seen, similar validation accuracy can be obtained using a fixed scale value (in this case \(\alpha=1\) or \(10\) will suffice) at the expense of another hyper-parameter to seek.

Validation accuracy for the fixed-classifier model remained equally good as the original model, and we continued to observe the same training curve.

By using a fixed classifier, approximately 2-million parameters were removed from the model, accounting for about 8% of the model parameters. Following the same procedure, we trained a Densenet169 model (Huang et al., 2017) for which a fixed classifier reduced about \(12\%\) of the parameters.

Furthermore, we were interested in evaluating more challenging models where the classifier parameters constitutes the majority amount. For this reason we chose the Shufflenet architecture (Zhang et al., 2017), which was designed to be used in low memory and limited computing platforms. The Shufflenet network contains about 1.8 million parameters, out of which 0.96 million are part of the final classifier. Fixing the classifier resulted with a model with only 0.86 million parameters. This model was trained and found, again, to converge to similar validation accuracy as the original. Interestingly, this method allowed Imagenet training in an under-specified regime, where there are more training samples than number of parameters. This is an unconventional regime for modern deep networks, which are usually over-specified to have many more parameters than training samples (Zhang et al., 2017).

As language modeling requires classification of all possible tokens available in the task vocabulary, we were interested to see if a fixed classifier can be used, possible saving a very large number of trainable parameters (vocabulary size can have tens or even hundreds of thousands of different words). Recent works have already found empirically that using the same weights for both word embedding and classifier can yield equal or better results than using a separate pair of weights (Inan et al., 2016; Press & Wolf, 2017; Vaswani et al., 2017). This is compliant with our findings that the linear classifier is largely redundant. To examine further reduction in the number of parameters, we removed both classifier and embedding weights and replaced them with a fixed transform.

We find that using fixed word2vec embeddings, we achieve much better results. Specifically, we use \(89\%\) less parameters than the fully learned model, and obtain only somewhat worse perplexity. We argue that this implies a required structure in word embedding that stems from semantic relatedness between words and the natural imbalance between classes. However, we suggest that with a much more cost effective ways to train word embeddings (e.g., Mikolov et al. (2013)), we can narrow the gap and avoid their cost when training bigger models.

Therefore the computational demands of the final classifier will increase as well and should be considered no less than the architecture chosen.

Our work can help considerably in this kind of scenario - where using a fixed classifier removes the need to do any gradient synchronization for the final layer.

We argue that our method works due to the ability of preceding layers in the network to learn separable representations that are easily classified even when the classifier itself is fixed.

In both scenarios, our method converged similarly to a fully learned classifier reaching the same final validation accuracy.

Understanding that linear classifiers used in NN models are largely redundant allows us to consider new approaches in training and understanding these models.

In this work we suggested removing the parameters from the classification layer used in deep neural networks. We showed empirical results suggesting that keeping the classifier fixed cause little or no decline in classification performance for common balanced datasets such as Cifar and Imagenet, while allowing a noticeable reduction in trainable parameters. We argue that fixing the last layer can reduce the computational complexity for training as well as the communication cost in distributed learning.

As datasets tend to become more complex by time (e.g., Cifar100, ImageNet1K, ImageNet22k, JFT-300M, and language modeling) we believe that resource hungry affine transformation should remain fixed during training, at least partially.

Based on these findings, we recommend future research to focus on representations learned by the non-linear part of neural networks - up to the final classifier, as it seems to be highly redundant.

Input: Figure 2: Comparing training and validation error of fixed and learned classifier (ResNet56, Cifar10)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim, evidence, and reviewer comments, I find that the claim is moderately well-stated with some limitations:

The evidence strongly supports that fixed classifier layers can achieve comparable performance to learned classifiers on balanced datasets like CIFAR and ImageNet, with empirical results showing similar validation accuracy. The paper demonstrates parameter reduction benefits (8-12% for ResNet/DenseNet, ~50% for ShuffleNet) without significant performance loss. However, the claim could better acknowledge limitations with language modeling tasks where performance was "somewhat worse" and scenarios with imbalanced datasets. The reviewer raises valid concerns about scaling to very large numbers of classes and implementation details that aren't fully addressed.

<score>2</score>

Justification:
1. Strong empirical evidence supports the core claim about fixed classifiers working well on common vision tasks
2. Results are clearly demonstrated in Figure 2 showing comparable validation accuracy
3. Parameter reduction benefits are quantified
4. Limitations are partially acknowledged but could be more explicitly stated regarding:
- Performance degradation on language modeling
- Scaling challenges with very large class numbers
- Implementation details for non-matching feature/class dimensions
- Impact on distillation and transfer learning scenarios

The claim earns a 2 rather than 1 due to these limitations not being fully acknowledged in the original statement.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this study suggest the reversed proposal - that common NN models used can learn useful representation even without modifying the final output layer, which often holds a large number of parameters that grows linearly with number of classes.

The review comment to be evaluated is:

The paper proposes to use a fixed weight matrix to replace the final linear projection in a deep neural network.
This fixed classifier is combined with a global scaling and per output shift that are learned.
The authors claim that this can be used as a drop in replacement for standard architectures and does not result in reduced performance.
The key advantage is that it generates a reduction in parameters (e.g. for resent 50 8% of parameters are eliminated).

The idea is extremely simple and I like it conceptually.
Currently it looks like my reimplementation on resent 50 is working.
I do lose a about 1% in accuracy compared to my baseline learned projection implementation.
Is the scale and bias regularized?

I have assigned a score of 6 now.  but I will wait for my final rating when I get the actual results.
Overall the evaluation is seems reasonably thorough many tasks were presented and the model was applied to different architectures.

I also think the manuscript could benefit from the following experiments:
- how does the chosen projection matrix affect performance.
- is the scale needed
I assume the authors did these experiments when they developed the method but it is unclear how important these choices are.
Including these experiments would make it a more scientific contribution.

The amount of computation saved seems rather limited? Especially since the gradient of the scale parameter has to go through the weight vector?
Therefore my assumption is that only the application of the gradients save a limited amount of time and memory?
At least in my experiments reproducing these results, the computational benefit is not there/obvious.

While I like the idea, the way the manuscript is written is a bit strange at times.
The introduction appears to be there to be because you need a introduction, not to explain the background.
For this reason some of the cited work seems a bit out of place.
Especially the universal approximation and data memorization references.
What I find interesting is that this work is the complement of the reservoir computing/extreme learning machines approach.
There the final output layer is trained but the network itself uses random weights.

It would be nice if Fig 2 had a better caption. Which dataset, model, ….
Is there an intuition why the training error remains higher but the validation error is identical? This is difficult to get my head round.
Also, it would be nice if an analysis was provided where the computational cost of not doing the gradient update was computed.

The evidence to be evaluated is:Despite the enormous number of trainable parameters these layers added to the model, they are known to have a rather marginal impact on the final performance of the network (Zeiler & Fergus, 2014) and are easily compressed and reduced after a model was trained by simple means such as matrix decomposition and sparsification (Han et al., 2015). Further more, modern architecture choices are characterized with the removal of most of the fully connected layers (Lin et al., 2013; Szegedy et al., 2015; He et al., 2016), which was found to lead to better generalization and overall accuracy, together with a huge decrease in the number of trainable parameters. Additionally, numerous works showed that CNNs can be trained in a metric learning regime (Bromley et al., 1994; Schroff et al., 2015; Hoffer & Ailon, 2015), where no explicit classification layer was introduced and the objective regarded only distance measures between intermediate representations. Hardt & Ma (2017) suggested an all-convolutional network variant, where they kept the original initialization of the classification layer fixed with no negative impact on performance on the Cifar10 dataset. All of these properties provide evidence that fully-connected layers are in fact redundant and play a small role in learning and generalization.

In this work we claim that for common use-cases of convolutional network, the parameters used for the final classification transform are completely redundant, and can be replaced with a pre-determined linear transform.

We suggest that existing models can, with no other modification, devoid their classifier weights, which can help the deployment of those models in devices with low computation ability and smaller memory capacity.

The use of a fixed transform which does not depend on the number classes can allow models to scale to a large number of possible outputs, without a linear cost in the number of parameters. We also suggest that these finding might shed light on the importance of the preceding non-linear layers to learning and generalization. We focus our attention on the final representation obtained by the network (the last hidden layer), before the classifier. We denote these representation as \(x=F(z;\theta)\) where \(F\) is assumed to be a deep neural network with input \(z\) and parameters \(\theta\), e.g., a convolutional network, trained by back-propagation. In common NN models, this representation is followed by an additional affine transformation \[y=W^{T}x+b\] where \(W\) and \(b\) are also trained by back-propagation.

To evaluate our conjecture regarding the importance of the final classification transformation, we replaced the trainable parameter matrix \(W\) with a fixed orthonormal projection \(Q\in\mathbb{R}^{N\times C}\), such that \(\forall i\neq j:q_{i}\cdot q_{j}=0\) and \(\|q_{i}\|_{2}=1\), where \(q_{i}\) is the \(i\)th column of \(Q\). This can be ensured by a simple random sampling and singular-value decomposition

We keep the additional vector of bias parameters \(b\in\mathbb{R}^{C}\), and train using the same negative-log-likelihood criterion. More explicitly, our classifier output is now \[v_{i}=\frac{e^{\alpha q_{i}\cdot\hat{x}+b_{i}}}{\sum_{j}^{C}e^{\alpha q_{j} \cdot\hat{x}+b_{j}}},\,i\in\{1,\ldots,C\}\] and we minimize the loss: \[\mathcal{L}(x,t)=-\alpha q_{t}\cdot\frac{x}{\|x\|_{2}}+b_{t}+\log\left(\sum_{ i=1}^{C}\exp\left(\alpha q_{i}\cdot\frac{x}{\|x\|_{2}}+b_{i}\right)\right)\] where we recall \(x\) is the final representation obtained by the network for a specific sample, and \(t\in\{1,\ldots,C\}\) is the ground-truth label for that sample. Observing the behavior of the \(\alpha\) parameter over time revealed a logarithmic growth depicted in graph 1. Interestingly, this is the same behavior exhibited by the norm of a learned classifier, first described by Hoffer et al. (2017) and linked to the generalization of the network. This was recently explained by the under-review work of Soudry et al. (2018) as convergence to a max margin classifier. We suggest that using a single parameter will enable a simpler examination and possible further exploration of this phenomenon and its implications.

allowing to discard the softmax function and its scale altogether, but resulting in a slight decrease in final validation accuracy compared to original models. We further suggest the use of a Hadamard matrix (Hedayat et al., 1978) as the final classification transform.

We can use a truncated Hadamard matrix \(\hat{H}\in\{-1,1\}^{C\times N}\) where all \(C\) rows are orthogonal as our final classification layer such that

This usage allows two main benefits:* A deterministic, low-memory and easily generated matrix that can be used to classify. * as multiplying by a Hadamard matrix can be done by simple sign manipulation and addition.

We also note the similarity of using a Hadamard matrix as a final classifier to methods of weight binarization such as the one suggested by Courbariaux et al. (2015). As the classifier weights are fixed to need only 1-bit precision, it is now possible to focus our attention on the features preceding it.

We trained a residual network of He et al. (2016) on the Cifar10 dataset. We used a network of depth 56 and the same hyper-parameters used in the original work.

The results shown in figure 2 demonstrate that although the training error is considerably lower for the network with learned classifier, both models achieve the same classification accuracy on the validation set. Our conjecture is that with our new fixed parameterization, the network can no longer increase the norm of a given sample's representation - thus learning its label requires more effort.

As can be seen, similar validation accuracy can be obtained using a fixed scale value (in this case \(\alpha=1\) or \(10\) will suffice) at the expense of another hyper-parameter to seek.

Validation accuracy for the fixed-classifier model remained equally good as the original model, and we continued to observe the same training curve.

By using a fixed classifier, approximately 2-million parameters were removed from the model, accounting for about 8% of the model parameters. Following the same procedure, we trained a Densenet169 model (Huang et al., 2017) for which a fixed classifier reduced about \(12\%\) of the parameters.

Furthermore, we were interested in evaluating more challenging models where the classifier parameters constitutes the majority amount. For this reason we chose the Shufflenet architecture (Zhang et al., 2017), which was designed to be used in low memory and limited computing platforms. The Shufflenet network contains about 1.8 million parameters, out of which 0.96 million are part of the final classifier. Fixing the classifier resulted with a model with only 0.86 million parameters. This model was trained and found, again, to converge to similar validation accuracy as the original. Interestingly, this method allowed Imagenet training in an under-specified regime, where there are more training samples than number of parameters. This is an unconventional regime for modern deep networks, which are usually over-specified to have many more parameters than training samples (Zhang et al., 2017).

As language modeling requires classification of all possible tokens available in the task vocabulary, we were interested to see if a fixed classifier can be used, possible saving a very large number of trainable parameters (vocabulary size can have tens or even hundreds of thousands of different words). Recent works have already found empirically that using the same weights for both word embedding and classifier can yield equal or better results than using a separate pair of weights (Inan et al., 2016; Press & Wolf, 2017; Vaswani et al., 2017). This is compliant with our findings that the linear classifier is largely redundant. To examine further reduction in the number of parameters, we removed both classifier and embedding weights and replaced them with a fixed transform.

We find that using fixed word2vec embeddings, we achieve much better results. Specifically, we use \(89\%\) less parameters than the fully learned model, and obtain only somewhat worse perplexity. We argue that this implies a required structure in word embedding that stems from semantic relatedness between words and the natural imbalance between classes. However, we suggest that with a much more cost effective ways to train word embeddings (e.g., Mikolov et al. (2013)), we can narrow the gap and avoid their cost when training bigger models.

Therefore the computational demands of the final classifier will increase as well and should be considered no less than the architecture chosen.

Our work can help considerably in this kind of scenario - where using a fixed classifier removes the need to do any gradient synchronization for the final layer.

We argue that our method works due to the ability of preceding layers in the network to learn separable representations that are easily classified even when the classifier itself is fixed.

In both scenarios, our method converged similarly to a fully learned classifier reaching the same final validation accuracy.

Understanding that linear classifiers used in NN models are largely redundant allows us to consider new approaches in training and understanding these models.

In this work we suggested removing the parameters from the classification layer used in deep neural networks. We showed empirical results suggesting that keeping the classifier fixed cause little or no decline in classification performance for common balanced datasets such as Cifar and Imagenet, while allowing a noticeable reduction in trainable parameters. We argue that fixing the last layer can reduce the computational complexity for training as well as the communication cost in distributed learning.

As datasets tend to become more complex by time (e.g., Cifar100, ImageNet1K, ImageNet22k, JFT-300M, and language modeling) we believe that resource hungry affine transformation should remain fixed during training, at least partially.

Based on these findings, we recommend future research to focus on representations learned by the non-linear part of neural networks - up to the final classifier, as it seems to be highly redundant.

Input: Figure 2: Comparing training and validation error of fixed and learned classifier (ResNet56, Cifar10)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review, the claim is reasonably well-stated but has some minor overstatements. The evidence shows that fixed classifiers can indeed achieve similar validation accuracy as learned classifiers across multiple architectures (ResNet56, DenseNet169, ShuffleNet) with minimal performance impact. However, the claim that this works "even without modifying the final output layer" is not entirely accurate, as the method still requires learning bias parameters and a scaling factor α. The review comments also point out some limitations regarding computational benefits and missing experimental details about projection matrix choices.

The evidence demonstrates clear parameter reduction benefits (8-12% for common architectures) while maintaining comparable performance, but does require some modifications to the output layer, making the claim slightly overstated.

<score>2</score>

Justification:
1. The evidence strongly supports that fixed classifiers can work well, with multiple experimental validations across different architectures
2. The claim needs minor qualification since some output layer modifications are still needed (bias, scaling)
3. The reviewer's comments about limited computational benefits and need for more experimental analysis of projection matrix choices suggest some overstating of benefits
4. The core premise is supported by empirical results showing similar validation accuracy
5. The evidence demonstrates this approach works across multiple datasets and architectures, lending credibility to the general claim


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this study suggest the reversed proposal - that common NN models used can learn useful representation even without modifying the final output layer, which often holds a large number of parameters that grows linearly with number of classes.

The review comment to be evaluated is:

Revised Review:

The authors have largely addressed my concerns with the revised manuscript. I still have some doubts about the C > N setting (the new settings of C / N of 4 and 2 aren't C >> N, and the associated results aren't detailed clearly in the paper), but I think the paper warrants acceptance.

Original Review:

The paper proposes fixing the classification layers of neural networks, replacing the traditional learned affine transformation with a fixed (e.g., Hadamard) matrix. This is motivated by the observation that classification layers frequently constitute a non-trivial fraction of a network's overall parameter count, compute requirements, and memory usage, and by the observation that removal of pre-classification fully-connected layers has often been found to have minimal impact on performance. Experiments are performed on a range of datasets and network architectures, in both image classification and NLP settings.

First, I'd like to note that the empirical component of this paper is strong: I was impressed by the breadth of architectures and settings covered, and the experiments left me reasonably convinced that the classification layer can often be fixed, at least for image classification tasks, without significant loss of accuracy.

I have two general concerns. For one, removing the fully connected classification layer is not a novel idea; All Convolutional Networks (https://arxiv.org/abs/1412.6806) reported excellent results without an additional fully connected affine transform (just a global average pooling after the last convolutional layer). I think it would be worth at least referencing/discussing differences with this and other all-convolutional architectures. Including a fixed Hadamard matrix for the classification layer is I believe new (although related to an existing literature on using structured matrices in neural networks).

However, I have doubts about the ability of the approach to scale to problems with a larger number of classes, which arguably is a primary motivation of the paper ("parameters ... grow linearly with the number of classes"). Specifically, the idea of using a fixed N x C matrix with C orthogonal columns (such as Hadamard) is only possible when N > C. This is a critical point: in the N > C regime, a final hidden representation with N dimensions can be chosen to achieve *any* C-dimensional output, regardless of the projection matrix used (so long as it is full rank). This makes it seem fairly reasonable to me that the network can (at least approximately, and complicated by the ReLU nonlinearities) fold the "desired" classification layer into the previous layer, especially with a learned scaling and bias term. In fact it's not clear to me that the fixed classification layer accomplishes anything here, beyond projecting from N -> C (i.e., if N = C, I'd guess it could be removed entirely similar to all convolutional nets, as long as the learned scaling and bias were retained).

On the other hand, when C > N, it is not possible to have mutually orthogonal columns, and in general the output is constrained to lie in an N-dimensional subspace of the overall C-dimensional output space. Picking somewhat randomly a *fixed* N-dimensional subspace seems like a bad idea when N << C, since it is unlikely to select a subspace in which it is possible to adequately capture correlations between the different classes. This makes the proposed technique much less appealing for precisely the family of problems where it would be most effective in reducing compute/memory requirements. It also provides (in my view) a clearer explanation for the failure of the approach in the NLP setting. These issues were not discussed anywhere in the text as far as I can tell, and I think it's necessary to at least acknowledge that mutually orthogonal columns can't be chosen when C > N in section 2.2 (and probably include a longer discussion on the probable implications).

Overall, I think the paper provides a useful observation that clearly isn't common knowledge, since classification layers persist in many popular recent architectures. But the notion of fixing or removing the classification layer isn't particularly novel, and I don't believe the proposed technique would scale well to settings with many classes. As is I think the paper falls slightly short.

The evidence to be evaluated is:Despite the enormous number of trainable parameters these layers added to the model, they are known to have a rather marginal impact on the final performance of the network (Zeiler & Fergus, 2014) and are easily compressed and reduced after a model was trained by simple means such as matrix decomposition and sparsification (Han et al., 2015). Further more, modern architecture choices are characterized with the removal of most of the fully connected layers (Lin et al., 2013; Szegedy et al., 2015; He et al., 2016), which was found to lead to better generalization and overall accuracy, together with a huge decrease in the number of trainable parameters. Additionally, numerous works showed that CNNs can be trained in a metric learning regime (Bromley et al., 1994; Schroff et al., 2015; Hoffer & Ailon, 2015), where no explicit classification layer was introduced and the objective regarded only distance measures between intermediate representations. Hardt & Ma (2017) suggested an all-convolutional network variant, where they kept the original initialization of the classification layer fixed with no negative impact on performance on the Cifar10 dataset. All of these properties provide evidence that fully-connected layers are in fact redundant and play a small role in learning and generalization.

In this work we claim that for common use-cases of convolutional network, the parameters used for the final classification transform are completely redundant, and can be replaced with a pre-determined linear transform.

We suggest that existing models can, with no other modification, devoid their classifier weights, which can help the deployment of those models in devices with low computation ability and smaller memory capacity.

The use of a fixed transform which does not depend on the number classes can allow models to scale to a large number of possible outputs, without a linear cost in the number of parameters. We also suggest that these finding might shed light on the importance of the preceding non-linear layers to learning and generalization. We focus our attention on the final representation obtained by the network (the last hidden layer), before the classifier. We denote these representation as \(x=F(z;\theta)\) where \(F\) is assumed to be a deep neural network with input \(z\) and parameters \(\theta\), e.g., a convolutional network, trained by back-propagation. In common NN models, this representation is followed by an additional affine transformation \[y=W^{T}x+b\] where \(W\) and \(b\) are also trained by back-propagation.

To evaluate our conjecture regarding the importance of the final classification transformation, we replaced the trainable parameter matrix \(W\) with a fixed orthonormal projection \(Q\in\mathbb{R}^{N\times C}\), such that \(\forall i\neq j:q_{i}\cdot q_{j}=0\) and \(\|q_{i}\|_{2}=1\), where \(q_{i}\) is the \(i\)th column of \(Q\). This can be ensured by a simple random sampling and singular-value decomposition

We keep the additional vector of bias parameters \(b\in\mathbb{R}^{C}\), and train using the same negative-log-likelihood criterion. More explicitly, our classifier output is now \[v_{i}=\frac{e^{\alpha q_{i}\cdot\hat{x}+b_{i}}}{\sum_{j}^{C}e^{\alpha q_{j} \cdot\hat{x}+b_{j}}},\,i\in\{1,\ldots,C\}\] and we minimize the loss: \[\mathcal{L}(x,t)=-\alpha q_{t}\cdot\frac{x}{\|x\|_{2}}+b_{t}+\log\left(\sum_{ i=1}^{C}\exp\left(\alpha q_{i}\cdot\frac{x}{\|x\|_{2}}+b_{i}\right)\right)\] where we recall \(x\) is the final representation obtained by the network for a specific sample, and \(t\in\{1,\ldots,C\}\) is the ground-truth label for that sample. Observing the behavior of the \(\alpha\) parameter over time revealed a logarithmic growth depicted in graph 1. Interestingly, this is the same behavior exhibited by the norm of a learned classifier, first described by Hoffer et al. (2017) and linked to the generalization of the network. This was recently explained by the under-review work of Soudry et al. (2018) as convergence to a max margin classifier. We suggest that using a single parameter will enable a simpler examination and possible further exploration of this phenomenon and its implications.

allowing to discard the softmax function and its scale altogether, but resulting in a slight decrease in final validation accuracy compared to original models. We further suggest the use of a Hadamard matrix (Hedayat et al., 1978) as the final classification transform.

We can use a truncated Hadamard matrix \(\hat{H}\in\{-1,1\}^{C\times N}\) where all \(C\) rows are orthogonal as our final classification layer such that

This usage allows two main benefits:* A deterministic, low-memory and easily generated matrix that can be used to classify. * as multiplying by a Hadamard matrix can be done by simple sign manipulation and addition.

We also note the similarity of using a Hadamard matrix as a final classifier to methods of weight binarization such as the one suggested by Courbariaux et al. (2015). As the classifier weights are fixed to need only 1-bit precision, it is now possible to focus our attention on the features preceding it.

We trained a residual network of He et al. (2016) on the Cifar10 dataset. We used a network of depth 56 and the same hyper-parameters used in the original work.

The results shown in figure 2 demonstrate that although the training error is considerably lower for the network with learned classifier, both models achieve the same classification accuracy on the validation set. Our conjecture is that with our new fixed parameterization, the network can no longer increase the norm of a given sample's representation - thus learning its label requires more effort.

As can be seen, similar validation accuracy can be obtained using a fixed scale value (in this case \(\alpha=1\) or \(10\) will suffice) at the expense of another hyper-parameter to seek.

Validation accuracy for the fixed-classifier model remained equally good as the original model, and we continued to observe the same training curve.

By using a fixed classifier, approximately 2-million parameters were removed from the model, accounting for about 8% of the model parameters. Following the same procedure, we trained a Densenet169 model (Huang et al., 2017) for which a fixed classifier reduced about \(12\%\) of the parameters.

Furthermore, we were interested in evaluating more challenging models where the classifier parameters constitutes the majority amount. For this reason we chose the Shufflenet architecture (Zhang et al., 2017), which was designed to be used in low memory and limited computing platforms. The Shufflenet network contains about 1.8 million parameters, out of which 0.96 million are part of the final classifier. Fixing the classifier resulted with a model with only 0.86 million parameters. This model was trained and found, again, to converge to similar validation accuracy as the original. Interestingly, this method allowed Imagenet training in an under-specified regime, where there are more training samples than number of parameters. This is an unconventional regime for modern deep networks, which are usually over-specified to have many more parameters than training samples (Zhang et al., 2017).

As language modeling requires classification of all possible tokens available in the task vocabulary, we were interested to see if a fixed classifier can be used, possible saving a very large number of trainable parameters (vocabulary size can have tens or even hundreds of thousands of different words). Recent works have already found empirically that using the same weights for both word embedding and classifier can yield equal or better results than using a separate pair of weights (Inan et al., 2016; Press & Wolf, 2017; Vaswani et al., 2017). This is compliant with our findings that the linear classifier is largely redundant. To examine further reduction in the number of parameters, we removed both classifier and embedding weights and replaced them with a fixed transform.

We find that using fixed word2vec embeddings, we achieve much better results. Specifically, we use \(89\%\) less parameters than the fully learned model, and obtain only somewhat worse perplexity. We argue that this implies a required structure in word embedding that stems from semantic relatedness between words and the natural imbalance between classes. However, we suggest that with a much more cost effective ways to train word embeddings (e.g., Mikolov et al. (2013)), we can narrow the gap and avoid their cost when training bigger models.

Therefore the computational demands of the final classifier will increase as well and should be considered no less than the architecture chosen.

Our work can help considerably in this kind of scenario - where using a fixed classifier removes the need to do any gradient synchronization for the final layer.

We argue that our method works due to the ability of preceding layers in the network to learn separable representations that are easily classified even when the classifier itself is fixed.

In both scenarios, our method converged similarly to a fully learned classifier reaching the same final validation accuracy.

Understanding that linear classifiers used in NN models are largely redundant allows us to consider new approaches in training and understanding these models.

In this work we suggested removing the parameters from the classification layer used in deep neural networks. We showed empirical results suggesting that keeping the classifier fixed cause little or no decline in classification performance for common balanced datasets such as Cifar and Imagenet, while allowing a noticeable reduction in trainable parameters. We argue that fixing the last layer can reduce the computational complexity for training as well as the communication cost in distributed learning.

As datasets tend to become more complex by time (e.g., Cifar100, ImageNet1K, ImageNet22k, JFT-300M, and language modeling) we believe that resource hungry affine transformation should remain fixed during training, at least partially.

Based on these findings, we recommend future research to focus on representations learned by the non-linear part of neural networks - up to the final classifier, as it seems to be highly redundant.

Input: Figure 2: Comparing training and validation error of fixed and learned classifier (ResNet56, Cifar10)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that it is moderately overstated. While the evidence shows that fixed classifiers can achieve comparable performance to learned classifiers in many cases (demonstrated by ResNet56 results on CIFAR-10 and other architectures), there are important limitations not fully acknowledged in the claim. The reviewer correctly points out that the approach faces challenges when C>N (number of classes exceeds representation dimensions). The paper's own results on language modeling show some performance degradation with fixed classifiers. The claim would be more accurate if it acknowledged these limitations and specified the conditions under which fixed classifiers work well.

<score>3</score>

Justification:
1. The evidence supports that fixed classifiers work well for balanced image datasets like CIFAR and ImageNet, with similar validation accuracy (Figure 2 demonstrates this for ResNet56).

2. However, the claim overstates by:
- Not acknowledging the C>N limitation highlighted in the review
- Not adequately addressing performance drops in language modeling tasks
- Using broad phrasing ("common NN models") without specifying limitations
- Not clearly qualifying the contexts where this approach works best

3. The review comment highlights valid technical limitations that should have been addressed in the claim's scope and formulation.