Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

This approach further improves on ImageNet performance over previous attempts in computer vision of minimizing trained components without substantially increasing the number of trained parameters.

The review comment to be evaluated is:

The authors present thorough analyses of standard feedforward networks that quantitatively demonstrate that such networks can model the visual stream well while requiring far fewer coordinated parameter changes than occur with traditional training methods.  In particular, they show that very little supervision is required for strong matches to neural responses, that plausible distributions of initial random weights can match neural responses very well, and that weights can be updated very sparsely while still achieving quite high neural response matches.

Such findings are highly relevant to allowing for such networks to be more plausibly accepted by the neuroscience community as valid models of cortical networks.  The striking finding that well-chosen random weights (Weight Compression) can predict neural responses with more than 50% accuracy is  especially interesting, as it suggests that indeed there may be less training required in mammalian brains compared to randomly initialized neuronal networks. This possibility opens up further avenues of study relating to the ability of real brains to learn with far fewer examples compared to ANNs.

Figure 4b further demonstrates this possibility, showing that, under the ethologically relevant condition of little supervision, WC + Critical Training, where only a small subset of weights in so-called critical layers are trained, greatly outperforms traditional training.  We note that, in the ideal case, a biologically plausible method might be found that performs well both under little supervision, as WC+CT does here, and under a large amount of supervision, as traditional backprop does, the demonstration of principle for the former case already represents appreciable progress.

However, little in the way of tying model predictivity to accuracy on another task was explored. While tying brain predictivity to network performance is of course a very active area in its own right, we feel that addressing this directly within the context of the present work would facilitate readers' abilities to contextualize the present study with respect to ANN performance.

Overall, we find the authors' presented work to be of great interest to those interested in applying ML models to theoretical questions in neuroscience and to have a thorough set of analyses that both illustrates progress in this area while suggesting future avenues of work.

Minor:
- opening quotes are backwards throughout paper

The evidence to be evaluated is:We evaluated the brain predictivity scores of CORnet-S variants that were trained with a combination of fewer epochs and images.

Similarly, when training until convergence with fractions of the 1.28M total images, 50,000 images are sufficient to obtain high neural scores (80% of full training in V1, V2, V4, IT).

For instance, the fully trained model's 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans (Yarbus, 1967; Gibaldi & Sabatini, 2020).

Sampling synaptic weights from reasonably compressed distributions on the other hand places little memory requirements on genetic encoding while potentially yielding useful initial weights. Current machine learning techniques for initializing weights, such as Kaiming Normal (He et al., 2015), sample from a Gaussian distribution centered around zero. To test the hypothesis that the genome might already encode more powerful initial representations with synaptic wiring sampled from distributions specified by only few bits, we explored multi-dimensional distributions as a more expressive alternative. These distributions only require a small number of parameters, but unlike current generic initializers, we explicitly specify them for each layer. To determine the right parameterization, we compress a trained model's weights into clusters which we then sample from ("_Weight Compression, WC_"). More specifically, for all convolutional layers except the first, we cluster the kernel weights and later sample from the clusters.

This results in 33 KB (4,166 parameters) to specify network initialization, compared to 423 MB for a trained model's weights (assuming 8 byte per parameter).

Applying _WC_ to CORnet-S, we first obtain a compressed and clustered set of parameters, from which we sample entirely new weights to yield a new model CORnet-S\({}_{\text{WC}}\). This model is _not trained at all_ and we only evaluate the goodness of its initial wiring on the suite of Brain-Score benchmarks.

Similarly, a trained linear decoder on CORnet-SWC's IT representations only reaches 5% of a trained model's ImageNet top-1 accuracy.

Training thin down-sampling layers reduces the number of updated synapses while maintaining high brain predictivity

As a proof-of-principle that training a reduced set of layers can retain high performance, we propose a novel thin training technique, which we term _Critical Training (CT: Figure 3A). CT_ updates only the weights in critical layers, instead of updating every single model synapse.

We explore successive variants of applying _CT_ up to a block in the architecture and then training the following blocks, e.g. freezing V1, V2, V4 with critical training of the respective down-sampling layers and additional IT training. The final _CT_ ventral stream model is almost completely frozen and only the synapses generating each cortical area's output are trained.

Compared to standard back-propagation training all the weights, all three approaches (_CT, DT, BN_) reduce the number of trained parameters (Figure 3C). However, while the average score with _DT_ (gray) already drops below 65% with over a quarter of trained parameters remaining and _BN_ drops to 62% with very few parameters, _CT_ (blue) maintains over 75% with only 1.4 out of 52.8 million parameters trained.

By reducing the number of trained parameters, _Critical Training_ also yields engineering benefits with more than 40% of the ImageNet score maintained at \(<3\%\) of parameters trained - a significant improvement over 22% accuracy with _BatchNormNorm_(Frankle et al., 2021) while adding only a small number of additional weights.

Testing synergies between these strategies, we combined all three methods to build novel models that only require a small number of supervised synaptic updates to reasonably capture the mechanisms of adult ventral visual stream processing and object recognition behavior.

_WC+CT_ yield strong benefits (green numbers) in a regime with few supervised updates, improving by up to 27 percent points when training for only 1 epoch on 1,000 images. With many updates on the other hand, _WC+CT_ is less advantageous than standard training (red numbers): with all 43 epochs and 1.28M images, the score reduces by 17 percent points.

_WC+CT_ maintain 91% of the score in ResNet despite an almost 80% reduction in parameters.

We developed a range of models with neural and behavioral scores approaching those of the current leading model of the adult ventral visual stream as quantified in Brain-Score, while requiring only a fraction of supervised synaptic updates.

We verified favorable transfer to models with similar architectures such as ResNet, but generalization to an already compressed MobileNet was limited (Figure 5A).

A critical component in more closely modeling primate development is to reduce the dependence on labels altogether. Recent unsupervised approaches are starting to rival the classification performance of supervised models (Caron et al., 2018; Henaff et al., 2019; Zhuang et al., 2020) and combining them with the advances presented here could further reduce the number of synaptic updates.

The changes to model initialization and training presented here serve as a proof-of-principle that models can be changed to more closely align with primate development by reducing training steps with labeled images and improving initialization.

Weight Compression (WC)'s cluster-based initialization generates an "at-birth" network that already captures some useful aspects of the visual inputs in a compressed manner and allows the system to learn faster.

However, independent of _how_ these clusters are determined, WC shows that it is possible to encode certain priors about a system's wiring diagram in a very compressed manner.

Our best model _WC+CT_ benefits from a combination of improved initialization through weight compression, and critical training.

The specific benchmark scores when either training with all labeled images for a varying number of epochs (Figure 9A) or when training with fewer labeled images until convergence (Figure 9B) show the benchmarks of early visual achieve the best results, relative to a fully trained model.

Notably, ImageNet performance of these networks seems to not be predictive of their brain predictivity, since even untrained networks with at-chance ImageNet performance correspond reasonably well to e.g. V1 (Figures 1 and 2).

To show the generalization of our approach we applied the weight compression methods to a ResNet-50 (He et al., 2016) and a MobileNet (Howard et al., 2017) (version 1, multiplier 1.0, image size 224) architecture.

This training version allows for more training but still reduces the amount of trained parameters by 43% while maintaining 78% of the original score.

* KN+CT The _KN_-initialized model trained with _Critical Training_ until convergence, i.e. three downstream layers and the decoder are trained and all other layers remain unchanged.

* **WC+CT** The _WC_-initialized model with _Critical Training_.

We used PyTorch 0.4.1 and trained the model using the ImageNet 2012 training set Deng et al. (2009). We used a batch size of 256 images and trained on a QuadroRTX6000 GPU until convergence.

Input: Figure 3: Training only critical layers reduces the number of updated synapses while maintaining high brain predictivity. A We could naively reduce the parameters of a fully-trained model by freezing layers from the bottom up, training only the top layers (“Downstream Training DT”; gray box). We instead propose Critical Training (CT) which only trains thin “critical” layers (e.g. downsampling layers, blue box). B CORnet-S circuitry. CT only trains critical layers, leaving the rest frozen. C Naively reducing parameters from standard training (black dot, top right) quickly deteriorates brain predictivity scores (DT, gray line) whereas Critical Training reduces parameters while retaining high scores (blue line, CT). D Like C, but measuring ImageNet score. CT retains nearly half the score with a fraction of parameters.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that "this approach further improves on ImageNet performance over previous attempts in computer vision of minimizing trained components" appears to be overstated. While the evidence shows that Critical Training (CT) maintains 40% ImageNet performance with <3% of parameters trained (an improvement over BatchNormNorm's 22%), this is a modest improvement in a specific context. The evidence demonstrates trade-offs, with WC+CT actually performing worse than standard training with full supervision. The performance improvements are mainly seen in limited-supervision scenarios, and generalization to some architectures like MobileNet was limited.

<score>4</score>

Justification:
1. The claim overstates the universal nature of the improvement, when the evidence shows context-dependent benefits:
- "WC+CT is less advantageous than standard training (red numbers): with all 43 epochs and 1.28M images, the score reduces by 17 percent points"
- "generalization to an already compressed MobileNet was limited"

2. While there are improvements over BatchNormNorm (40% vs 22% ImageNet accuracy), this is a narrow comparison point rather than "previous attempts" broadly.

3. The reviewer notes that "little in the way of tying model predictivity to accuracy on another task was explored," suggesting limitations in the broader evaluation of performance improvements.

4. The evidence focuses more on brain predictivity scores than ImageNet performance, making the claim's emphasis on ImageNet improvement particularly overreaching.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To reduce the varied redundancy across channel and temporal dimensions, we introduce an input-dependent redundancy reduction framework called VA-RED\({}^{2}\) (Video Adaptive REDundancy REDuction) for efficient video recognition (see Figure 1 for an illustrative example).

The evidence to be evaluated is:Despite promising results on common benchmarks, there is a significant interest in developing more efficient techniques and smaller models with reasonable performance. Previous works have shown reductions in computational complexity by using hybrid 2D-3D architectures (Xie et al., 2018; Zhou et al., 2018; Zolfaghari et al., 2018), group convolution (Tran et al., 2019) or selecting salient clips (Korbar et al., 2019). Feichtenhofer et al., (Feichtenhofer et al., 2018) propose a dedicated low-framerate pathway.

Fan et al. (Fan et al., 2020) incorporate a efficient learnable 3D-shift module into a 3D video network. Wang et al. (Wang et al., 2020) devise a correlation module to learn correlation along temporal dimension.

While these approaches bring considerable efficiency improvements, none of them dynamically calibrates the required feature map computations on a per-input basis. Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

Many adaptive computation methods have been recently proposed with the goal of improving efficiency (Bengio et al., 2015, 2013; Veit and Belongie, 2018; Wang et al., 2018; Graves, 2016; Meng et al., 2021). Several works have been proposed that add decision branches to different layers of CNNs to learn whether to exit the network for faster inference (Yu et al., 2018; Figurnov et al., 2017; McGill and Perona, 2017; Teerapittayanon et al., 2016) Wang et al. (Wang et al., 2018) propose to skip convolutional blocks on a per input basis using reinforcement learning and supervised pre-training. Veit et al. (Veit and Belongie, 2018) propose a block skipping method controlled by samples from a Gumbel softmax, while Wu et al. (Wu et al., 2018) develop a reinforcement learning approach to achieve this goal. Adaptive computation time for recurrent neural networks is also presented in (Graves, 2016). SpotTune (Guo et al., 2019) learns to adaptively route information through finetuned or pre-trained layers. A few works have been recently proposed for selecting salient frames conditioned on the input (Yeung et al., 2016; Wu et al., 2019; Korbar et al., 2019; Gao et al., 2019) while recognizing actions in long untrimmed videos. Different from adaptive data sampling (Yeung et al., 2016; Wu et al., 2019; Korbar et al., 2019; Gao et al., 2019), in this paper, our goal is to remove feature map redundancy by deciding how many features need to be computed for temporal and channel dimensions per input basis, for efficient video recognition.

In contrast, our method focuses on reducing the redundancy in both temporal and channel dimension, and is applicable to both 3D and 2D models, while AR-Net is only for 2D model and it is focused on spatial resolution.

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy. Our main goal is to automatically decide which feature maps to compute for each input video in order to classify it correctly with the minimum computation. The intuition behind our proposed method is that there are many similar feature maps along the temporal and channel dimensions. For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension. Then, for the other feature maps, we reconstruct them from those pre-computed feature maps using cheap linear operations.

Without loss of generality, we start from a 3D convolutional network \(\mathcal{G}\), and denote its \(l^{th}\) 3D convolution layer as \(f_{l}\), and the corresponding input and output feature maps as \(X_{l}\) and \(Y_{l}\) respectively. For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\). Thus the shape of output \(Y_{l}^{\prime}\) becomes \(C_{out}\times T_{o}/R\times H_{o}\times W_{o}\).

\[Y_{l}[j+iR]=\left\{\begin{array}{ll}\Phi_{i,j}^{t}(Y_{l}^{\prime}[i])&\text{ if }j\in\{1,...,R-1\}\\ Y_{l}^{\prime}[i]&\text{if }j=0\end{array}\right.,i\in\{0,1,...,T_{o}/R-1\}, \tag{1}\]where \(Y_{l}[j+iR]\) represents the \((j+iR)^{th}\) feature map of \(Y_{l}\) along the temporal dimension, \(Y_{l}^{\prime}[i]\) denotes the \(i^{th}\) feature map of \(Y_{l}^{\prime}\), and \(\Phi_{i,j}^{t}\) is the cheap linear operation along the temporal dimension. The total computational cost of this process can be written as: \[\mathcal{C}(f_{l}^{t})=\frac{1}{R}\cdot\mathcal{C}(f_{l})+\sum_{i,j}\mathcal{C }(\Phi_{i,j}^{t})\approx\frac{1}{R}\cdot\mathcal{C}(f_{l}), \tag{2}\] where the function \(\mathcal{C}(\cdot)\) returns the computation cost for a specific operation, and \(f_{l}^{t}\) represents our dynamic convolution process along temporal dimension. Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference. The total computation cost of joint temporal-wise and channel-wise dynamic inference is: \[\mathcal{C}(f_{l}^{t,c})\approx\frac{r}{R}\cdot\mathcal{C}(f_{l}), \tag{3}\]

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively.

\[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution.

\[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\).

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

We evaluate our method on three most widely-used model architectures: I3D (Carreira and Zisserman, 2017), R(2+1)D (Tran et al., 2018), and the recent efficient model X3D (Feichtenhofer, 2020).

We choose X3D-M (denote as X3D below) as our base model and demonstrate that our method is generally effective across datasets.

As PSPNet is devised for image semantic segmentation, we only apply the channel-wise redundancy reduction to the model and adopt synchronized SGD training for 100k iterations across 4GPUs with 2 images on each GPU.

We first evaluate our method by applying it to R(2+1)D-18 (Tran et al., 2018) with different number of input frames and different size of search space.

We test performance of or approach by turning of dynamic modeling along temporal and channel dimensions on Mini-Kinetics-200. Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

In X3D, point-wise convolutions which right after the depth-wise convolutions have more variation among classes and network tends to consume more temporal-wise features at the early stage and compute more channel-wise features at the late stage of the architecture. The channel-wise policy has also more variation than the temporal-wise policy among different categories.

Video clips which have more complicated scene configuration (e.g. cooking eggs and playing volleyball) and more violent camera motion (e.g. flipping pancake) tend to need more feature maps to do the correct predictions.

Our VA-RED\({}^{2}\) framework is also applicable for some dense visual tasks, like semantic segmentation, which requires the pixel-level prediction for the input content.

As can be seen from Table 11, our proposed VA-RED\({}^{2}\) has the absolute advantage in terms of efficiency while maintaining the precision of segmentation.

In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy. Extensive experimental results on video action recognition, spatio-temporal localization, and semantic segmentation validate the effectiveness of our framework in multiple standard benchmark datasets.

We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. For example, the original channel-wise dynamic convolution at ratio \(r=(\frac{1}{2})^{i(i-1)}\) can be analogized to \[\big{[}(f_{l}^{c}(X_{l},r=(\frac{1}{2})^{i_{s}^{c}-1})[0:(\frac{1}{2})^{(i-1)} C_{out}]),(\Phi^{c}(f_{l}^{c}(X_{l},r=(\frac{1}{2})^{i_{s}^{c}-1})[0:(\frac{1}{2})^ {(i-1)}\cdot C_{out}]))\big{]}, \tag{7}\] where \([\cdot\ \cdot]\) is the index operation along the channel dimension, and \(i_{s}^{c}\) is the index of the largest channel-wise filter, during training phase, we have \(i_{s}^{c}=1\), while during inference phase, \(i_{s}^{c}\) is the smallest index for \(V_{c}^{l}\), \(s.t.V_{c}^{l}[i_{s}^{c}]=0\). By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

To motivate our redundancy reduction approach, we measure and visualize the internal redundancy of well known pretrained networks. We analyze the internal feature maps of existing pre-trained 13D-InceptionV2 and R(2+1)D networks on Moments in Time and Kinetics. For each model-dataset pair, we extract feature maps for all examples in the validation sets, in both time and channel dimensions, and measure their similarity. In detail, our method consists of the following steps: (1) For a given input, we first extract the output feature maps from all convolutional layers in the network at hand. (2) In each layer, we measure the similarity of each feature map to each other with Person's correlation coefficient (CC) and root mean squared error (RMSE). We additionally flag feature maps that exhibit high similarity as redundant.

We can see that the examples consuming less computation tend to have less temporal motion, like the second example in Figure 11, or have a relatively simple scene configuration, like the first and second examples in Figure 12.

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Table 11: VA-RED2 on semantic segmentation. We choose dilated ResNet-18 as our backbone architecture and set the search space as 2. Models are trained for 100K iterations with batch size of 8.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 6: Visualization of temporal-wise feature maps. We plot the temporal feature maps which are fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps marked with red bounding boxes are cheaply generated. We do this analysis on 8-frame dynamic R(2+1)D-18 pretrained on Mini-Kinetics-200. These feature maps are the output of the first spatial convolution combined with ReLU non-linearity inside the ResBlock_1. We can see that most of the cheaply generated feature maps looks no difference from the original feature maps, which further support our approach. Best viewed in color.

Input: Figure 7: Visualization of channel-wise feature maps. We plot the feature maps across the channel dimension. We contrast two kinds of feature maps: fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps inside the red bounding boxes are cheaply generated. The analysis is performed on 8-frame dynamic R(2+1)D-18 model which is pretrained on Mini-Kinetics-200 dataset and we extract these feature maps which are output by the first spatial convolution layer inside the ResBlock_1. Best viewed in color.

Input: Figure 4: Validation video clips from Mini-Kinetics-200. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 8-frame dynamic R(2+1)D-18 model trained on Mini-Kinetics-200 and the percentage indicates the ratio of actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 5: Visualization of the first 9 filters of the first layer of I3D, on examples with most (top) and least (bottom) redundancy in the temporal dimension. We exemplify the results on frames 1, 2 and 3. As can be seen, the video with most redundancy consists of a relatively static video with little movement, and the sets of feature maps from frame to frame harbor heavy similarity. The video with least redundancy consists of a gift unwrapping with rapid movement (even in the first few frames) and the corresponding feature maps present visible structural differences from frame to frame. Although in both cases, redundancy is present, it is clear that some examples present much more redundancy than others, thus motivating our input-dependent redundancy reduction approach.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review, the claim appears well-stated and accurately reflects the evidence provided. The paper presents:

1. A clear framework (VA-RED²) that dynamically reduces redundancy in both temporal and channel dimensions
2. Comprehensive experimental results showing the effectiveness across multiple architectures (I3D, R(2+1)D, X3D) and tasks
3. Visual evidence demonstrating how the framework adapts computation based on video content (Figures 1, 11, 12)
4. Detailed technical explanations of the mechanism (Equations 1-5)
5. Ablation studies validating the benefits of both temporal and channel-wise redundancy reduction (Table 10)

The claim is specific, technically accurate, and supported by extensive evidence without exaggerating the capabilities or novelty of the approach.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To reduce the varied redundancy across channel and temporal dimensions, we introduce an input-dependent redundancy reduction framework called VA-RED\({}^{2}\) (Video Adaptive REDundancy REDuction) for efficient video recognition (see Figure 1 for an illustrative example).

The review comment to be evaluated is:

This paper proposes a novel framework called $\text{VA-RED}^2$ to reduce spatial and temporal features to be computed for video understanding, which can reduce FLOPs when inferencing the video but remains the performance.

The authors have done extensive experiments on video action recognition tasks and spatio-temporal action localization task in the area of video understanding. For the video action recognition task, experiments are carried out using Mini-Kinetics-200, Kinetics-400, and Moments-In-time datasets. For the action localization task, J-HMDB-21 dataset is used. Results show that this framework is promising, which reduces the computation but main the performance.

Question:
1. X3D-M in the original paper achieved at top-1 74.6 for Kinetics-400 dataset (but in table 4 reports clip-1: 61.8, video-1: 67.9) and FLOPs is 4.73 (6.20 reported in the paper). Can authors explain why there is a difference here?

Minor:
1. Reduce ratio fact for channel-wise dynamic convolution $r=\frac{1}{2}^{p_c}$ such as in Fig.2 on page 4 and equation 7, equation 8 in supplementary materials on page 13. I think it would make more sense representing it as $(\frac{1}{2})^{p_c}$.

The evidence to be evaluated is:Despite promising results on common benchmarks, there is a significant interest in developing more efficient techniques and smaller models with reasonable performance. Previous works have shown reductions in computational complexity by using hybrid 2D-3D architectures (Xie et al., 2018; Zhou et al., 2018; Zolfaghari et al., 2018), group convolution (Tran et al., 2019) or selecting salient clips (Korbar et al., 2019). Feichtenhofer et al., (Feichtenhofer et al., 2018) propose a dedicated low-framerate pathway.

Fan et al. (Fan et al., 2020) incorporate a efficient learnable 3D-shift module into a 3D video network. Wang et al. (Wang et al., 2020) devise a correlation module to learn correlation along temporal dimension.

While these approaches bring considerable efficiency improvements, none of them dynamically calibrates the required feature map computations on a per-input basis. Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

Many adaptive computation methods have been recently proposed with the goal of improving efficiency (Bengio et al., 2015, 2013; Veit and Belongie, 2018; Wang et al., 2018; Graves, 2016; Meng et al., 2021). Several works have been proposed that add decision branches to different layers of CNNs to learn whether to exit the network for faster inference (Yu et al., 2018; Figurnov et al., 2017; McGill and Perona, 2017; Teerapittayanon et al., 2016) Wang et al. (Wang et al., 2018) propose to skip convolutional blocks on a per input basis using reinforcement learning and supervised pre-training. Veit et al. (Veit and Belongie, 2018) propose a block skipping method controlled by samples from a Gumbel softmax, while Wu et al. (Wu et al., 2018) develop a reinforcement learning approach to achieve this goal. Adaptive computation time for recurrent neural networks is also presented in (Graves, 2016). SpotTune (Guo et al., 2019) learns to adaptively route information through finetuned or pre-trained layers. A few works have been recently proposed for selecting salient frames conditioned on the input (Yeung et al., 2016; Wu et al., 2019; Korbar et al., 2019; Gao et al., 2019) while recognizing actions in long untrimmed videos. Different from adaptive data sampling (Yeung et al., 2016; Wu et al., 2019; Korbar et al., 2019; Gao et al., 2019), in this paper, our goal is to remove feature map redundancy by deciding how many features need to be computed for temporal and channel dimensions per input basis, for efficient video recognition.

In contrast, our method focuses on reducing the redundancy in both temporal and channel dimension, and is applicable to both 3D and 2D models, while AR-Net is only for 2D model and it is focused on spatial resolution.

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy. Our main goal is to automatically decide which feature maps to compute for each input video in order to classify it correctly with the minimum computation. The intuition behind our proposed method is that there are many similar feature maps along the temporal and channel dimensions. For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension. Then, for the other feature maps, we reconstruct them from those pre-computed feature maps using cheap linear operations.

Without loss of generality, we start from a 3D convolutional network \(\mathcal{G}\), and denote its \(l^{th}\) 3D convolution layer as \(f_{l}\), and the corresponding input and output feature maps as \(X_{l}\) and \(Y_{l}\) respectively. For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\). Thus the shape of output \(Y_{l}^{\prime}\) becomes \(C_{out}\times T_{o}/R\times H_{o}\times W_{o}\).

\[Y_{l}[j+iR]=\left\{\begin{array}{ll}\Phi_{i,j}^{t}(Y_{l}^{\prime}[i])&\text{ if }j\in\{1,...,R-1\}\\ Y_{l}^{\prime}[i]&\text{if }j=0\end{array}\right.,i\in\{0,1,...,T_{o}/R-1\}, \tag{1}\]where \(Y_{l}[j+iR]\) represents the \((j+iR)^{th}\) feature map of \(Y_{l}\) along the temporal dimension, \(Y_{l}^{\prime}[i]\) denotes the \(i^{th}\) feature map of \(Y_{l}^{\prime}\), and \(\Phi_{i,j}^{t}\) is the cheap linear operation along the temporal dimension. The total computational cost of this process can be written as: \[\mathcal{C}(f_{l}^{t})=\frac{1}{R}\cdot\mathcal{C}(f_{l})+\sum_{i,j}\mathcal{C }(\Phi_{i,j}^{t})\approx\frac{1}{R}\cdot\mathcal{C}(f_{l}), \tag{2}\] where the function \(\mathcal{C}(\cdot)\) returns the computation cost for a specific operation, and \(f_{l}^{t}\) represents our dynamic convolution process along temporal dimension. Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference. The total computation cost of joint temporal-wise and channel-wise dynamic inference is: \[\mathcal{C}(f_{l}^{t,c})\approx\frac{r}{R}\cdot\mathcal{C}(f_{l}), \tag{3}\]

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively.

\[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution.

\[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\).

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

We evaluate our method on three most widely-used model architectures: I3D (Carreira and Zisserman, 2017), R(2+1)D (Tran et al., 2018), and the recent efficient model X3D (Feichtenhofer, 2020).

We choose X3D-M (denote as X3D below) as our base model and demonstrate that our method is generally effective across datasets.

As PSPNet is devised for image semantic segmentation, we only apply the channel-wise redundancy reduction to the model and adopt synchronized SGD training for 100k iterations across 4GPUs with 2 images on each GPU.

We first evaluate our method by applying it to R(2+1)D-18 (Tran et al., 2018) with different number of input frames and different size of search space.

We test performance of or approach by turning of dynamic modeling along temporal and channel dimensions on Mini-Kinetics-200. Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

In X3D, point-wise convolutions which right after the depth-wise convolutions have more variation among classes and network tends to consume more temporal-wise features at the early stage and compute more channel-wise features at the late stage of the architecture. The channel-wise policy has also more variation than the temporal-wise policy among different categories.

Video clips which have more complicated scene configuration (e.g. cooking eggs and playing volleyball) and more violent camera motion (e.g. flipping pancake) tend to need more feature maps to do the correct predictions.

Our VA-RED\({}^{2}\) framework is also applicable for some dense visual tasks, like semantic segmentation, which requires the pixel-level prediction for the input content.

As can be seen from Table 11, our proposed VA-RED\({}^{2}\) has the absolute advantage in terms of efficiency while maintaining the precision of segmentation.

In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy. Extensive experimental results on video action recognition, spatio-temporal localization, and semantic segmentation validate the effectiveness of our framework in multiple standard benchmark datasets.

We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. For example, the original channel-wise dynamic convolution at ratio \(r=(\frac{1}{2})^{i(i-1)}\) can be analogized to \[\big{[}(f_{l}^{c}(X_{l},r=(\frac{1}{2})^{i_{s}^{c}-1})[0:(\frac{1}{2})^{(i-1)} C_{out}]),(\Phi^{c}(f_{l}^{c}(X_{l},r=(\frac{1}{2})^{i_{s}^{c}-1})[0:(\frac{1}{2})^ {(i-1)}\cdot C_{out}]))\big{]}, \tag{7}\] where \([\cdot\ \cdot]\) is the index operation along the channel dimension, and \(i_{s}^{c}\) is the index of the largest channel-wise filter, during training phase, we have \(i_{s}^{c}=1\), while during inference phase, \(i_{s}^{c}\) is the smallest index for \(V_{c}^{l}\), \(s.t.V_{c}^{l}[i_{s}^{c}]=0\). By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

To motivate our redundancy reduction approach, we measure and visualize the internal redundancy of well known pretrained networks. We analyze the internal feature maps of existing pre-trained 13D-InceptionV2 and R(2+1)D networks on Moments in Time and Kinetics. For each model-dataset pair, we extract feature maps for all examples in the validation sets, in both time and channel dimensions, and measure their similarity. In detail, our method consists of the following steps: (1) For a given input, we first extract the output feature maps from all convolutional layers in the network at hand. (2) In each layer, we measure the similarity of each feature map to each other with Person's correlation coefficient (CC) and root mean squared error (RMSE). We additionally flag feature maps that exhibit high similarity as redundant.

We can see that the examples consuming less computation tend to have less temporal motion, like the second example in Figure 11, or have a relatively simple scene configuration, like the first and second examples in Figure 12.

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Table 11: VA-RED2 on semantic segmentation. We choose dilated ResNet-18 as our backbone architecture and set the search space as 2. Models are trained for 100K iterations with batch size of 8.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 6: Visualization of temporal-wise feature maps. We plot the temporal feature maps which are fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps marked with red bounding boxes are cheaply generated. We do this analysis on 8-frame dynamic R(2+1)D-18 pretrained on Mini-Kinetics-200. These feature maps are the output of the first spatial convolution combined with ReLU non-linearity inside the ResBlock_1. We can see that most of the cheaply generated feature maps looks no difference from the original feature maps, which further support our approach. Best viewed in color.

Input: Figure 7: Visualization of channel-wise feature maps. We plot the feature maps across the channel dimension. We contrast two kinds of feature maps: fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps inside the red bounding boxes are cheaply generated. The analysis is performed on 8-frame dynamic R(2+1)D-18 model which is pretrained on Mini-Kinetics-200 dataset and we extract these feature maps which are output by the first spatial convolution layer inside the ResBlock_1. Best viewed in color.

Input: Figure 4: Validation video clips from Mini-Kinetics-200. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 8-frame dynamic R(2+1)D-18 model trained on Mini-Kinetics-200 and the percentage indicates the ratio of actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 5: Visualization of the first 9 filters of the first layer of I3D, on examples with most (top) and least (bottom) redundancy in the temporal dimension. We exemplify the results on frames 1, 2 and 3. As can be seen, the video with most redundancy consists of a relatively static video with little movement, and the sets of feature maps from frame to frame harbor heavy similarity. The video with least redundancy consists of a gift unwrapping with rapid movement (even in the first few frames) and the corresponding feature maps present visible structural differences from frame to frame. Although in both cases, redundancy is present, it is clear that some examples present much more redundancy than others, thus motivating our input-dependent redundancy reduction approach.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim and evidence, this appears to be a well-stated claim (score of 1) for the following reasons:

The evidence strongly supports that VA-RED2 is indeed an input-dependent redundancy reduction framework for video recognition. The extensive technical details, equations, and experimental results demonstrate:
- Clear methodology for reducing redundancy in both temporal and channel dimensions
- Input-dependent policy learning through soft modulation gates
- Visual examples showing how the framework adapts computation based on video content
- Empirical validation across multiple datasets and architectures
- Transparent discussion of the implementation approach

The claim is precise and accurately reflects what the framework does without overstating its capabilities or impact. The visual examples and experimental results directly validate the core claim about input-dependent redundancy reduction.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To reduce the varied redundancy across channel and temporal dimensions, we introduce an input-dependent redundancy reduction framework called VA-RED\({}^{2}\) (Video Adaptive REDundancy REDuction) for efficient video recognition (see Figure 1 for an illustrative example).

The review comment to be evaluated is:

Pros:
1. The motivation of the submission is clear and the writing is easy to follow. The problem being addressed is a practical and important issue in deploying video models to real applications.

2. Experiments are thorough. This submission has done experiments on both action recognition and action localization task. The performance is promising on most video networks.

Cons:
1. This submission proposes to address the problem of slow video inference, but the only metric they report is the Gflops. We know that Gflops is not a good indicator for speed comparison. Sometimes, a small Gflops does not mean fast speed. So I recommend to add a "speed" column in every table of this submission, to compare the actual inference speed among the methods.

2. Regarding the searching method, can authors justify what are the differences to X3D? X3D also search the optimal combination of temporal dimension and channel dimension. Are the differences only on adding efficiency loss?

3. The efficiency loss is widely used in searching efficient image models, like in Proxylessnas. What is the contribution and motivation of using it here in searching video models?

The evidence to be evaluated is:Despite promising results on common benchmarks, there is a significant interest in developing more efficient techniques and smaller models with reasonable performance. Previous works have shown reductions in computational complexity by using hybrid 2D-3D architectures (Xie et al., 2018; Zhou et al., 2018; Zolfaghari et al., 2018), group convolution (Tran et al., 2019) or selecting salient clips (Korbar et al., 2019). Feichtenhofer et al., (Feichtenhofer et al., 2018) propose a dedicated low-framerate pathway.

Fan et al. (Fan et al., 2020) incorporate a efficient learnable 3D-shift module into a 3D video network. Wang et al. (Wang et al., 2020) devise a correlation module to learn correlation along temporal dimension.

While these approaches bring considerable efficiency improvements, none of them dynamically calibrates the required feature map computations on a per-input basis. Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

Many adaptive computation methods have been recently proposed with the goal of improving efficiency (Bengio et al., 2015, 2013; Veit and Belongie, 2018; Wang et al., 2018; Graves, 2016; Meng et al., 2021). Several works have been proposed that add decision branches to different layers of CNNs to learn whether to exit the network for faster inference (Yu et al., 2018; Figurnov et al., 2017; McGill and Perona, 2017; Teerapittayanon et al., 2016) Wang et al. (Wang et al., 2018) propose to skip convolutional blocks on a per input basis using reinforcement learning and supervised pre-training. Veit et al. (Veit and Belongie, 2018) propose a block skipping method controlled by samples from a Gumbel softmax, while Wu et al. (Wu et al., 2018) develop a reinforcement learning approach to achieve this goal. Adaptive computation time for recurrent neural networks is also presented in (Graves, 2016). SpotTune (Guo et al., 2019) learns to adaptively route information through finetuned or pre-trained layers. A few works have been recently proposed for selecting salient frames conditioned on the input (Yeung et al., 2016; Wu et al., 2019; Korbar et al., 2019; Gao et al., 2019) while recognizing actions in long untrimmed videos. Different from adaptive data sampling (Yeung et al., 2016; Wu et al., 2019; Korbar et al., 2019; Gao et al., 2019), in this paper, our goal is to remove feature map redundancy by deciding how many features need to be computed for temporal and channel dimensions per input basis, for efficient video recognition.

In contrast, our method focuses on reducing the redundancy in both temporal and channel dimension, and is applicable to both 3D and 2D models, while AR-Net is only for 2D model and it is focused on spatial resolution.

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy. Our main goal is to automatically decide which feature maps to compute for each input video in order to classify it correctly with the minimum computation. The intuition behind our proposed method is that there are many similar feature maps along the temporal and channel dimensions. For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension. Then, for the other feature maps, we reconstruct them from those pre-computed feature maps using cheap linear operations.

Without loss of generality, we start from a 3D convolutional network \(\mathcal{G}\), and denote its \(l^{th}\) 3D convolution layer as \(f_{l}\), and the corresponding input and output feature maps as \(X_{l}\) and \(Y_{l}\) respectively. For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\). Thus the shape of output \(Y_{l}^{\prime}\) becomes \(C_{out}\times T_{o}/R\times H_{o}\times W_{o}\).

\[Y_{l}[j+iR]=\left\{\begin{array}{ll}\Phi_{i,j}^{t}(Y_{l}^{\prime}[i])&\text{ if }j\in\{1,...,R-1\}\\ Y_{l}^{\prime}[i]&\text{if }j=0\end{array}\right.,i\in\{0,1,...,T_{o}/R-1\}, \tag{1}\]where \(Y_{l}[j+iR]\) represents the \((j+iR)^{th}\) feature map of \(Y_{l}\) along the temporal dimension, \(Y_{l}^{\prime}[i]\) denotes the \(i^{th}\) feature map of \(Y_{l}^{\prime}\), and \(\Phi_{i,j}^{t}\) is the cheap linear operation along the temporal dimension. The total computational cost of this process can be written as: \[\mathcal{C}(f_{l}^{t})=\frac{1}{R}\cdot\mathcal{C}(f_{l})+\sum_{i,j}\mathcal{C }(\Phi_{i,j}^{t})\approx\frac{1}{R}\cdot\mathcal{C}(f_{l}), \tag{2}\] where the function \(\mathcal{C}(\cdot)\) returns the computation cost for a specific operation, and \(f_{l}^{t}\) represents our dynamic convolution process along temporal dimension. Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference. The total computation cost of joint temporal-wise and channel-wise dynamic inference is: \[\mathcal{C}(f_{l}^{t,c})\approx\frac{r}{R}\cdot\mathcal{C}(f_{l}), \tag{3}\]

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively.

\[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution.

\[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\).

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

We evaluate our method on three most widely-used model architectures: I3D (Carreira and Zisserman, 2017), R(2+1)D (Tran et al., 2018), and the recent efficient model X3D (Feichtenhofer, 2020).

We choose X3D-M (denote as X3D below) as our base model and demonstrate that our method is generally effective across datasets.

As PSPNet is devised for image semantic segmentation, we only apply the channel-wise redundancy reduction to the model and adopt synchronized SGD training for 100k iterations across 4GPUs with 2 images on each GPU.

We first evaluate our method by applying it to R(2+1)D-18 (Tran et al., 2018) with different number of input frames and different size of search space.

We test performance of or approach by turning of dynamic modeling along temporal and channel dimensions on Mini-Kinetics-200. Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

In X3D, point-wise convolutions which right after the depth-wise convolutions have more variation among classes and network tends to consume more temporal-wise features at the early stage and compute more channel-wise features at the late stage of the architecture. The channel-wise policy has also more variation than the temporal-wise policy among different categories.

Video clips which have more complicated scene configuration (e.g. cooking eggs and playing volleyball) and more violent camera motion (e.g. flipping pancake) tend to need more feature maps to do the correct predictions.

Our VA-RED\({}^{2}\) framework is also applicable for some dense visual tasks, like semantic segmentation, which requires the pixel-level prediction for the input content.

As can be seen from Table 11, our proposed VA-RED\({}^{2}\) has the absolute advantage in terms of efficiency while maintaining the precision of segmentation.

In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy. Extensive experimental results on video action recognition, spatio-temporal localization, and semantic segmentation validate the effectiveness of our framework in multiple standard benchmark datasets.

We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. For example, the original channel-wise dynamic convolution at ratio \(r=(\frac{1}{2})^{i(i-1)}\) can be analogized to \[\big{[}(f_{l}^{c}(X_{l},r=(\frac{1}{2})^{i_{s}^{c}-1})[0:(\frac{1}{2})^{(i-1)} C_{out}]),(\Phi^{c}(f_{l}^{c}(X_{l},r=(\frac{1}{2})^{i_{s}^{c}-1})[0:(\frac{1}{2})^ {(i-1)}\cdot C_{out}]))\big{]}, \tag{7}\] where \([\cdot\ \cdot]\) is the index operation along the channel dimension, and \(i_{s}^{c}\) is the index of the largest channel-wise filter, during training phase, we have \(i_{s}^{c}=1\), while during inference phase, \(i_{s}^{c}\) is the smallest index for \(V_{c}^{l}\), \(s.t.V_{c}^{l}[i_{s}^{c}]=0\). By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

To motivate our redundancy reduction approach, we measure and visualize the internal redundancy of well known pretrained networks. We analyze the internal feature maps of existing pre-trained 13D-InceptionV2 and R(2+1)D networks on Moments in Time and Kinetics. For each model-dataset pair, we extract feature maps for all examples in the validation sets, in both time and channel dimensions, and measure their similarity. In detail, our method consists of the following steps: (1) For a given input, we first extract the output feature maps from all convolutional layers in the network at hand. (2) In each layer, we measure the similarity of each feature map to each other with Person's correlation coefficient (CC) and root mean squared error (RMSE). We additionally flag feature maps that exhibit high similarity as redundant.

We can see that the examples consuming less computation tend to have less temporal motion, like the second example in Figure 11, or have a relatively simple scene configuration, like the first and second examples in Figure 12.

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Table 11: VA-RED2 on semantic segmentation. We choose dilated ResNet-18 as our backbone architecture and set the search space as 2. Models are trained for 100K iterations with batch size of 8.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 6: Visualization of temporal-wise feature maps. We plot the temporal feature maps which are fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps marked with red bounding boxes are cheaply generated. We do this analysis on 8-frame dynamic R(2+1)D-18 pretrained on Mini-Kinetics-200. These feature maps are the output of the first spatial convolution combined with ReLU non-linearity inside the ResBlock_1. We can see that most of the cheaply generated feature maps looks no difference from the original feature maps, which further support our approach. Best viewed in color.

Input: Figure 7: Visualization of channel-wise feature maps. We plot the feature maps across the channel dimension. We contrast two kinds of feature maps: fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps inside the red bounding boxes are cheaply generated. The analysis is performed on 8-frame dynamic R(2+1)D-18 model which is pretrained on Mini-Kinetics-200 dataset and we extract these feature maps which are output by the first spatial convolution layer inside the ResBlock_1. Best viewed in color.

Input: Figure 4: Validation video clips from Mini-Kinetics-200. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 8-frame dynamic R(2+1)D-18 model trained on Mini-Kinetics-200 and the percentage indicates the ratio of actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 5: Visualization of the first 9 filters of the first layer of I3D, on examples with most (top) and least (bottom) redundancy in the temporal dimension. We exemplify the results on frames 1, 2 and 3. As can be seen, the video with most redundancy consists of a relatively static video with little movement, and the sets of feature maps from frame to frame harbor heavy similarity. The video with least redundancy consists of a gift unwrapping with rapid movement (even in the first few frames) and the corresponding feature maps present visible structural differences from frame to frame. Although in both cases, redundancy is present, it is clear that some examples present much more redundancy than others, thus motivating our input-dependent redundancy reduction approach.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the evidence, the claim appears to be well-stated (score of 1). The paper provides extensive experimental results, mathematical formulations, and visualizations that strongly support the proposed VA-RED2 framework for reducing redundancy across temporal and channel dimensions. The evidence demonstrates:

1. Clear motivation through redundancy analysis in pretrained networks
2. Detailed technical approach with mathematical formulations
3. Comprehensive experimental validation across multiple architectures and datasets
4. Visualization evidence showing how the framework adaptively reduces computation for different video types
5. Reviewer comments acknowledge clear motivation and thorough experiments

The claim accurately reflects what the evidence demonstrates without exaggeration, focusing on the core contribution of an input-dependent redundancy reduction framework.

<score>1</score>