Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

(2) An **adaptive policy** jointly learned with the network weights in a fully differentiable way with a shared-weight mechanism, that allows us to make decisions on how many feature maps to compute.

The evidence to be evaluated is:While these approaches bring considerable efficiency improvements, none of them dynamically calibrates the required feature map computations on a per-input basis. Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

Our network learns the best internal redundancy reduction scheme, which is similar to previous work on automatically searching architectures (Elsken et al., 2018).

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy.

For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension.

For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\).

Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference.

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively. The \(V_{t}^{l}\) and \(V_{c}^{t}\) are learned by: \[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution. During training, we obtain the final output of the dynamic convolution by weighted sum of all the feature maps which contains different ratio of fully-computed features as follows: \[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\). During the inference phase, only the dynamic convolutions whose weights are not zero will be computed.

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

To encourage our network to output a computational efficient subgraph, we introduce the efficiency loss \(\mathcal{L}_{c}\) during the training process, which can be formulated as \[\mathcal{L}_{e}=(\mu_{0}\sum_{l=1}^{L}\frac{\mathcal{C}(f_{l})}{\sum_{k=1}^{L }\mathcal{C}(f_{k})}\cdot\frac{r_{l}^{s}}{R_{l}^{s}})^{2},\mu_{0}=\left\{ \begin{array}{ll}1&\text{if correct}\\ 0&\text{otherwise}\end{array}\right., \tag{6}\]

Overall, the loss function of our whole framework can be written as \(\mathcal{L}=\mathcal{L}_{a}+\lambda_{e}\mathcal{L}_{e}\), where \(L_{a}\) is the accuracy loss of the whole network and \(\lambda_{e}\) is the weight of efficiency loss which can be used to balance the importance of the optimization of prediction accuracy and computational cost.

Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

To better understand the policy decision process, we dissect the network layers and count the ratio of feature maps that are being computed during each convolution layers for each category.

Video clips which have more complicated scene configuration (e.g. cooking eggs and playing volleyball) and more violent camera motion (e.g. flipping pancake) tend to need more feature maps to do the correct predictions.

In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy.

* Wang et al. (2018) Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. Skipnet: Learning dynamic routing in convolutional networks. In _Proceedings of the European Conference on Computer Vision (ECCV)_, pp. 409-424, 2018.

* Wu et al. (2018) Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S Davis, Kristen Grauman, and Rogerio Feris. Blockdrop: Dynamic inference paths in residual networks. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pp. 8817-8826, 2018.

In this section, we provide more details of the shared-weight mechanism presented in Section 3 of the main paper. We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output.

where \([\cdot\ \cdot]\) is the index operation along the channel dimension, and \(i_{s}^{c}\) is the index of the largest channel-wise filter, during training phase, we have \(i_{s}^{c}=1\), while during inference phase, \(i_{s}^{c}\) is the smallest index for \(V_{c}^{l}\), \(s.t.V_{c}^{l}[i_{s}^{c}]=0\). By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Table 5: Action recognition results on Kinetics-400. We set the search space as 2, meaning models can choose to compute all feature maps or 1 2 of them both on temporal and channel-wise convolutions.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 4: Validation video clips from Mini-Kinetics-200. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 8-frame dynamic R(2+1)D-18 model trained on Mini-Kinetics-200 and the percentage indicates the ratio of actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the evidence, the claim about an adaptive policy being jointly learned with network weights in a fully differentiable way with shared-weight mechanism is well-supported by substantial technical details and empirical validation. The evidence provides:

1. Detailed mathematical formulation of the policy learning mechanism through soft modulation gates
2. Clear explanation of the shared-weight mechanism implementation
3. Extensive experimental results showing the policy's effectiveness in reducing redundant computations while maintaining accuracy
4. Visualization evidence demonstrating how the policy adapts computation based on input complexity
5. Ablation studies validating the benefits of dynamic modeling along both temporal and channel dimensions

The claim appears to be precise and accurately reflects what the evidence demonstrates without exaggeration.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

(2) An **adaptive policy** jointly learned with the network weights in a fully differentiable way with a shared-weight mechanism, that allows us to make decisions on how many feature maps to compute.

The review comment to be evaluated is:

This paper proposes a novel framework called $\text{VA-RED}^2$ to reduce spatial and temporal features to be computed for video understanding, which can reduce FLOPs when inferencing the video but remains the performance.

The authors have done extensive experiments on video action recognition tasks and spatio-temporal action localization task in the area of video understanding. For the video action recognition task, experiments are carried out using Mini-Kinetics-200, Kinetics-400, and Moments-In-time datasets. For the action localization task, J-HMDB-21 dataset is used. Results show that this framework is promising, which reduces the computation but main the performance.

Question:
1. X3D-M in the original paper achieved at top-1 74.6 for Kinetics-400 dataset (but in table 4 reports clip-1: 61.8, video-1: 67.9) and FLOPs is 4.73 (6.20 reported in the paper). Can authors explain why there is a difference here?

Minor:
1. Reduce ratio fact for channel-wise dynamic convolution $r=\frac{1}{2}^{p_c}$ such as in Fig.2 on page 4 and equation 7, equation 8 in supplementary materials on page 13. I think it would make more sense representing it as $(\frac{1}{2})^{p_c}$.

The evidence to be evaluated is:While these approaches bring considerable efficiency improvements, none of them dynamically calibrates the required feature map computations on a per-input basis. Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

Our network learns the best internal redundancy reduction scheme, which is similar to previous work on automatically searching architectures (Elsken et al., 2018).

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy.

For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension.

For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\).

Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference.

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively. The \(V_{t}^{l}\) and \(V_{c}^{t}\) are learned by: \[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution. During training, we obtain the final output of the dynamic convolution by weighted sum of all the feature maps which contains different ratio of fully-computed features as follows: \[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\). During the inference phase, only the dynamic convolutions whose weights are not zero will be computed.

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

To encourage our network to output a computational efficient subgraph, we introduce the efficiency loss \(\mathcal{L}_{c}\) during the training process, which can be formulated as \[\mathcal{L}_{e}=(\mu_{0}\sum_{l=1}^{L}\frac{\mathcal{C}(f_{l})}{\sum_{k=1}^{L }\mathcal{C}(f_{k})}\cdot\frac{r_{l}^{s}}{R_{l}^{s}})^{2},\mu_{0}=\left\{ \begin{array}{ll}1&\text{if correct}\\ 0&\text{otherwise}\end{array}\right., \tag{6}\]

Overall, the loss function of our whole framework can be written as \(\mathcal{L}=\mathcal{L}_{a}+\lambda_{e}\mathcal{L}_{e}\), where \(L_{a}\) is the accuracy loss of the whole network and \(\lambda_{e}\) is the weight of efficiency loss which can be used to balance the importance of the optimization of prediction accuracy and computational cost.

Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

To better understand the policy decision process, we dissect the network layers and count the ratio of feature maps that are being computed during each convolution layers for each category.

Video clips which have more complicated scene configuration (e.g. cooking eggs and playing volleyball) and more violent camera motion (e.g. flipping pancake) tend to need more feature maps to do the correct predictions.

In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy.

* Wang et al. (2018) Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. Skipnet: Learning dynamic routing in convolutional networks. In _Proceedings of the European Conference on Computer Vision (ECCV)_, pp. 409-424, 2018.

* Wu et al. (2018) Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S Davis, Kristen Grauman, and Rogerio Feris. Blockdrop: Dynamic inference paths in residual networks. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pp. 8817-8826, 2018.

In this section, we provide more details of the shared-weight mechanism presented in Section 3 of the main paper. We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output.

where \([\cdot\ \cdot]\) is the index operation along the channel dimension, and \(i_{s}^{c}\) is the index of the largest channel-wise filter, during training phase, we have \(i_{s}^{c}=1\), while during inference phase, \(i_{s}^{c}\) is the smallest index for \(V_{c}^{l}\), \(s.t.V_{c}^{l}[i_{s}^{c}]=0\). By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Table 5: Action recognition results on Kinetics-400. We set the search space as 2, meaning models can choose to compute all feature maps or 1 2 of them both on temporal and channel-wise convolutions.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 4: Validation video clips from Mini-Kinetics-200. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 8-frame dynamic R(2+1)D-18 model trained on Mini-Kinetics-200 and the percentage indicates the ratio of actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim about the adaptive policy being jointly learned with network weights in a fully differentiable way with a shared-weight mechanism.

Summary:
The evidence strongly supports this claim through detailed mathematical formulations and architectural descriptions. The paper demonstrates:
1. A differentiable policy layer (soft modulation gate) that learns probability vectors for feature map selection
2. A weight-sharing mechanism that efficiently computes features by sampling and reconstructing
3. Joint optimization through combined loss function (accuracy + efficiency)
4. Clear implementation details and ablation studies showing the effectiveness

The claim appears to be well-stated and accurately reflects the technical approach without exaggeration. The evidence shows comprehensive mathematical foundations and empirical validation through experiments.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

(2) An **adaptive policy** jointly learned with the network weights in a fully differentiable way with a shared-weight mechanism, that allows us to make decisions on how many feature maps to compute.

The review comment to be evaluated is:

Pros:
1. The motivation of the submission is clear and the writing is easy to follow. The problem being addressed is a practical and important issue in deploying video models to real applications.

2. Experiments are thorough. This submission has done experiments on both action recognition and action localization task. The performance is promising on most video networks.

Cons:
1. This submission proposes to address the problem of slow video inference, but the only metric they report is the Gflops. We know that Gflops is not a good indicator for speed comparison. Sometimes, a small Gflops does not mean fast speed. So I recommend to add a "speed" column in every table of this submission, to compare the actual inference speed among the methods.

2. Regarding the searching method, can authors justify what are the differences to X3D? X3D also search the optimal combination of temporal dimension and channel dimension. Are the differences only on adding efficiency loss?

3. The efficiency loss is widely used in searching efficient image models, like in Proxylessnas. What is the contribution and motivation of using it here in searching video models?

The evidence to be evaluated is:While these approaches bring considerable efficiency improvements, none of them dynamically calibrates the required feature map computations on a per-input basis. Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

Our network learns the best internal redundancy reduction scheme, which is similar to previous work on automatically searching architectures (Elsken et al., 2018).

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy.

For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension.

For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\).

Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference.

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively. The \(V_{t}^{l}\) and \(V_{c}^{t}\) are learned by: \[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution. During training, we obtain the final output of the dynamic convolution by weighted sum of all the feature maps which contains different ratio of fully-computed features as follows: \[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\). During the inference phase, only the dynamic convolutions whose weights are not zero will be computed.

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

To encourage our network to output a computational efficient subgraph, we introduce the efficiency loss \(\mathcal{L}_{c}\) during the training process, which can be formulated as \[\mathcal{L}_{e}=(\mu_{0}\sum_{l=1}^{L}\frac{\mathcal{C}(f_{l})}{\sum_{k=1}^{L }\mathcal{C}(f_{k})}\cdot\frac{r_{l}^{s}}{R_{l}^{s}})^{2},\mu_{0}=\left\{ \begin{array}{ll}1&\text{if correct}\\ 0&\text{otherwise}\end{array}\right., \tag{6}\]

Overall, the loss function of our whole framework can be written as \(\mathcal{L}=\mathcal{L}_{a}+\lambda_{e}\mathcal{L}_{e}\), where \(L_{a}\) is the accuracy loss of the whole network and \(\lambda_{e}\) is the weight of efficiency loss which can be used to balance the importance of the optimization of prediction accuracy and computational cost.

Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

To better understand the policy decision process, we dissect the network layers and count the ratio of feature maps that are being computed during each convolution layers for each category.

Video clips which have more complicated scene configuration (e.g. cooking eggs and playing volleyball) and more violent camera motion (e.g. flipping pancake) tend to need more feature maps to do the correct predictions.

In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy.

* Wang et al. (2018) Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. Skipnet: Learning dynamic routing in convolutional networks. In _Proceedings of the European Conference on Computer Vision (ECCV)_, pp. 409-424, 2018.

* Wu et al. (2018) Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S Davis, Kristen Grauman, and Rogerio Feris. Blockdrop: Dynamic inference paths in residual networks. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pp. 8817-8826, 2018.

In this section, we provide more details of the shared-weight mechanism presented in Section 3 of the main paper. We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output.

where \([\cdot\ \cdot]\) is the index operation along the channel dimension, and \(i_{s}^{c}\) is the index of the largest channel-wise filter, during training phase, we have \(i_{s}^{c}=1\), while during inference phase, \(i_{s}^{c}\) is the smallest index for \(V_{c}^{l}\), \(s.t.V_{c}^{l}[i_{s}^{c}]=0\). By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Table 5: Action recognition results on Kinetics-400. We set the search space as 2, meaning models can choose to compute all feature maps or 1 2 of them both on temporal and channel-wise convolutions.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 4: Validation video clips from Mini-Kinetics-200. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 8-frame dynamic R(2+1)D-18 model trained on Mini-Kinetics-200 and the percentage indicates the ratio of actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim about an "adaptive policy learned jointly with network weights in a fully differentiable way with a shared-weight mechanism" is well-supported by substantial technical details. The evidence demonstrates:

- A detailed mathematical framework for learning policies that dynamically adjust feature map computations
- Implementation specifics of the shared-weight mechanism
- Experimental results showing the effectiveness across different architectures and datasets
- Visualization evidence of how the policy adapts to different video content
- Thorough ablation studies validating the joint learning approach

The claim accurately reflects what is demonstrated in the evidence without exaggeration of capabilities or results. The technical implementation and results align closely with what is claimed.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

(2) An **adaptive policy** jointly learned with the network weights in a fully differentiable way with a shared-weight mechanism, that allows us to make decisions on how many feature maps to compute.

The review comment to be evaluated is:

Summary
The paper presents a framework to reduce internal redundancy in the video recognition model. To do so, given the input frames, the framework predicts two scaling factors to conduct temporal and channel dimension reduction. The remaining part is reconstructed by cheap operations. The authors show that the framework achieves favorable results on several benchmarks.

Strengths
+ The paper is well written.
+ Strong quantitative and qualitative results (Consistent improvement over the state-of-the-art baselines).
+ Solid ablation studies and analysis.

Weakness
- The core idea of using a light-weight neural module to maximize the video model efficiency is not novel.
For example, AR-Net already has shown that a policy network can decide the video input resolution, i.e., spatial dimension, adaptively, leading to improve both efficiency and accuracy. In this work, the key concept is basically the same and only the primitive operation unit, i.e., temporal and channel dimensions, is different.

- Please provide experiment-level comparisons with AR-Net

- The following SOTA video models are missing in the reference.
DynamoNet: Dynamic action and motion network, ICCV 2019
Video Modeling with Correlation Networks, CVPR 2020
RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition, ECCV2020
Directional Temporal Modeling for Action Recognition, ECCV2020

Question

- Can the idea be applied to dense video understanding tasks, such as video semantic segmentation?

Rating

- Aiming at acquiring an efficient model in a data-driven manner is indeed important for video models.
While the key idea is not novel, the paper has obvious empirical contributions that may help communities to invigorate future researches in this direction.

The evidence to be evaluated is:While these approaches bring considerable efficiency improvements, none of them dynamically calibrates the required feature map computations on a per-input basis. Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

Our network learns the best internal redundancy reduction scheme, which is similar to previous work on automatically searching architectures (Elsken et al., 2018).

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy.

For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension.

For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\).

Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference.

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively. The \(V_{t}^{l}\) and \(V_{c}^{t}\) are learned by: \[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution. During training, we obtain the final output of the dynamic convolution by weighted sum of all the feature maps which contains different ratio of fully-computed features as follows: \[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\). During the inference phase, only the dynamic convolutions whose weights are not zero will be computed.

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

To encourage our network to output a computational efficient subgraph, we introduce the efficiency loss \(\mathcal{L}_{c}\) during the training process, which can be formulated as \[\mathcal{L}_{e}=(\mu_{0}\sum_{l=1}^{L}\frac{\mathcal{C}(f_{l})}{\sum_{k=1}^{L }\mathcal{C}(f_{k})}\cdot\frac{r_{l}^{s}}{R_{l}^{s}})^{2},\mu_{0}=\left\{ \begin{array}{ll}1&\text{if correct}\\ 0&\text{otherwise}\end{array}\right., \tag{6}\]

Overall, the loss function of our whole framework can be written as \(\mathcal{L}=\mathcal{L}_{a}+\lambda_{e}\mathcal{L}_{e}\), where \(L_{a}\) is the accuracy loss of the whole network and \(\lambda_{e}\) is the weight of efficiency loss which can be used to balance the importance of the optimization of prediction accuracy and computational cost.

Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

To better understand the policy decision process, we dissect the network layers and count the ratio of feature maps that are being computed during each convolution layers for each category.

Video clips which have more complicated scene configuration (e.g. cooking eggs and playing volleyball) and more violent camera motion (e.g. flipping pancake) tend to need more feature maps to do the correct predictions.

In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy.

* Wang et al. (2018) Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. Skipnet: Learning dynamic routing in convolutional networks. In _Proceedings of the European Conference on Computer Vision (ECCV)_, pp. 409-424, 2018.

* Wu et al. (2018) Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S Davis, Kristen Grauman, and Rogerio Feris. Blockdrop: Dynamic inference paths in residual networks. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pp. 8817-8826, 2018.

In this section, we provide more details of the shared-weight mechanism presented in Section 3 of the main paper. We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output.

where \([\cdot\ \cdot]\) is the index operation along the channel dimension, and \(i_{s}^{c}\) is the index of the largest channel-wise filter, during training phase, we have \(i_{s}^{c}=1\), while during inference phase, \(i_{s}^{c}\) is the smallest index for \(V_{c}^{l}\), \(s.t.V_{c}^{l}[i_{s}^{c}]=0\). By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Table 5: Action recognition results on Kinetics-400. We set the search space as 2, meaning models can choose to compute all feature maps or 1 2 of them both on temporal and channel-wise convolutions.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 4: Validation video clips from Mini-Kinetics-200. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 8-frame dynamic R(2+1)D-18 model trained on Mini-Kinetics-200 and the percentage indicates the ratio of actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find that the claim about an "adaptive policy jointly learned with network weights in a fully differentiable way with a shared-weight mechanism" is well-supported by the detailed technical evidence provided. The paper demonstrates:

1. A concrete implementation of the adaptive policy through soft modulation gates that output probability vectors to control feature computation
2. Clear mathematical formulation showing how the policy is learned jointly with network weights through backpropagation
3. Extensive experimental results showing the policy effectively adapts computation based on input complexity
4. Detailed explanation of the shared-weight mechanism for efficient training

While the reviewer notes the core idea isn't entirely novel, the specific technical implementation and empirical validation strongly support the claim's precise wording about the methodology. The claim accurately reflects what is demonstrated without overstating capabilities.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Our approach is model-agnostic and can be applied to any backbones to reduce feature redundancy in both time and channel domains.

The evidence to be evaluated is:Our framework achieves substantial improvements in average efficiency by avoiding redundant feature map computation depending on the input.

In contrast, our method focuses on reducing the redundancy in both temporal and channel dimension, and is applicable to both 3D and 2D models, while AR-Net is only for 2D model and it is focused on spatial resolution.

In contrast, our approach learns a policy that chooses over full or reduced convolutions at inference time, effectively switching between various discovered subnetworks to minimize redundant computations and deliver high accuracy.

For each video instance, we estimate the ratio of feature maps that need to be fully computed along the temporal dimension and channel dimension.

Without loss of generality, we start from a 3D convolutional network \(\mathcal{G}\), and denote its \(l^{th}\) 3D convolution layer as \(f_{l}\), and the corresponding input and output feature maps as \(X_{l}\) and \(Y_{l}\) respectively. For each 3D convolution layer, we use a very lightweight policy layer \(p_{l}\) denoted as _soft modulation gate_ to decide the ratio of feature maps along the temporal and channel dimensions which need to be computed. As shown in Figure 2, for temporal-wise dynamic inference, we reduce the computation of 3D convolution layer by dynamically scaling the temporal stride of the 3D filter with a factor \(R=2^{p_{l}(X_{l})[0]}\). Thus the shape of output \(Y_{l}^{\prime}\) becomes \(C_{out}\times T_{o}/R\times H_{o}\times W_{o}\).

Different from temporal-wise dynamic inference, we reduce the channel-wise computation by dynamically controlling the number of output channels. We scale the output channel number with a factor \(r=(\frac{1}{2})^{p_{l}(X_{l})[1]}\). In this case, the shape of output \(Y_{l}^{\prime}\) is \(rC_{out}\times T_{o}\times H_{o}\times W_{o}\). Same as before, we reconstruct the remaining features via cheap linear operations, which can be formulated as \(Y_{l}=[Y_{l}^{\prime},\Phi^{c}(Y_{l}^{\prime})]\), where \(\Phi^{c}(Y_{l}^{\prime})\in R^{(1-r)C_{out}\times T_{o}\times H_{o}\times W_{o}}\) represents the cheaply generated feature maps along the channel dimension, and \(Y_{l}\in R^{C_{out}\times T_{o}\times H_{o}\times W_{o}}\) is the output of the channel-wise dynamic inference.

We adopt an extremely lightweight policy layer \(p_{l}\) called soft modulation gate for each convolution layer \(f_{l}\) to modulate the ratio of features which need to be computed. Specifically, the soft modulation gate takes the input feature maps \(X_{l}\) as input and learns two probability vectors \(V_{t}^{l}\in R^{S_{t}}\) and \(V_{c}^{t}\in R^{S_{c}}\), where \(S_{t}\) and \(S_{c}\) are the temporal search space size and the channel search space size respectively. The \(V_{t}^{l}\) and \(V_{c}^{t}\) are learned by: \[[V_{t}^{l},V_{c}^{l}]=p_{l}(X_{l})=\phi(\mathcal{F}(\omega_{p,2},\delta( \mathcal{N}(\mathcal{F}(\omega_{p,1},G(X_{l})))))+\beta_{p}^{l}), \tag{4}\] where \(\mathcal{F}(\cdot,\cdot)\) denotes the fully-connected layer, \(\mathcal{N}\) is the batch normalization, \(\delta(\cdot)\) represents the \(\tanh(\cdot)\) function, \(G\) is the global pooling operation whose output shape is \(C_{in}\cdot T\times 1\times 1\), \(\phi(\cdot)\) is the output activation function, here we just use \(\max(\tanh(\cdot),0)\) whose output range is \([0,1)\), and \(\omega_{p,1}\in R^{(S_{t}+S_{c})\times D_{h}}\), \(\omega_{p,2}\in R^{D_{h}\times C_{in}\cdot T}\) are the weights of their corresponding layers, \(D_{h}\) is the hidden dimension number. \(V_{t}^{l}\) and \(V_{c}^{l}\) will then be used to modulate the ratio of the feature maps to be computed in temporal-wise dynamic convolution and channel-wise dynamic convolution. During training, we obtain the final output of the dynamic convolution by weighted sum of all the feature maps which contains different ratio of fully-computed features as follows: \[Y_{c}^{l}=\sum_{i=1}^{S_{c}}V_{c}^{l}[i]\cdot f_{l}^{c}(X_{l},r=(\frac{1}{2})^{(i -1)}),\quad Y_{l}=\sum_{j=1}^{S_{t}}V_{l}^{l}[j]\cdot f_{l}^{t}(Y_{c}^{l},R=2^{(j -1)}), \tag{5}\] where \(f_{l}^{c}(\cdot,r)\) is the channel-wise dynamic convolution with the channel scaling factor \(r\), and \(f_{l}^{t}(\cdot,R)\) it the temporal-wise dynamic convolution with the temporal stride scaling factor \(R\).

To train our model efficiently, we utilize a weight-sharing mechanism to reduce the computational cost and training memory. To be specific, we first compute all the possible necessary features using a big kernel. Then, for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output. Though this, we are able to keep the computational cost at a constant value invariant to the search space.

We evaluate our method on three most widely-used model architectures: I3D (Carreira and Zisserman, 2017), R(2+1)D (Tran et al., 2018), and the recent efficient model X3D (Feichtenhofer, 2020). We consider I3D-InceptionV2 (denoted as I3D below) and R(2+1)D-18 (denoted as R(2+1)D below) as our base model.

We choose X3D-M (denote as X3D below) as our base model and demonstrate that our method is generally effective across datasets.

For semantic segmentation, we conduct experiments using PSPNet (Zhao et al., 2017), with dilated ResNet-18 (Yu et al., 2017; He et al., 2016) as our backbone architecture.

We first evaluate our method by applying it to R(2+1)D-18 (Tran et al., 2018) with different number of input frames and different size of search space.

We test performance of or approach by turning of dynamic modeling along temporal and channel dimensions on Mini-Kinetics-200. Table 10 shows that dynamic modeling along both dimensions obtains the best performance while requiring the least computation. This shows importance of input-dependent policy for deciding how many features need to be computed for both temporal and channel dimensions.

Our VA-RED\({}^{2}\) framework is also applicable for some dense visual tasks, like semantic segmentation, which requires the pixel-level prediction for the input content. To prove this, we apply our method to a semantic segmentation model on the ADE-20K dataset (Zhou et al., 2017; 2018).

This experiment clearly shows that our method is not only effective on the recognition and detection tasks, but also applicable to the dense visual tasks like semantic segmentation. In this paper, we propose an input-dependent adaptive framework called VA-RED\({}^{2}\) for efficient inference which can be easily plugged into most of existing video understanding models to significantly reduce the model computation while maintaining the accuracy.

We first compute all the possible necessary features using a big kernel and then for each dynamic convolution with different scaling factor, we sample its corresponding ratio of necessary features and reconstruct the rest features by cheap operations to get the final output.

By utilizing such a share-weight mechanism, the computation of the total channel-wise dynamic convolution is reduce to \((\frac{1}{2})^{i_{s}^{c}-1}\cdot\mathcal{C}(f_{l})\).

We apply our method mainly to 2D convolutions in R(2+1)D since 2D convolution takes the most computational cost compared with 1D convolution.

To motivate our redundancy reduction approach, we measure and visualize the internal redundancy of well known pretrained networks. We analyze the internal feature maps of existing pre-trained 13D-InceptionV2 and R(2+1)D networks on Moments in Time and Kinetics. For each model-dataset pair, we extract feature maps for all examples in the validation sets, in both time and channel dimensions, and measure their similarity. In detail, our method consists of the following steps: (1) For a given input, we first extract the output feature maps from all convolutional layers in the network at hand. (2) In each layer, we measure the similarity of each feature map to each other with Person's correlation coefficient (CC) and root mean squared error (RMSE). We additionally flag feature maps that exhibit high similarity as redundant.

We demonstrate those in both temporal dimension (c.f. Figure 6) and channel dimension (c.f. Figure 7). In both cases we can see that the proposed cheap operation generates meaningful feature maps and some of them looks even no difference from the original feature maps.

Specifically, In X3D, point-wise convolutions which right after the depth-wise convolutions have more variation among classes and network tends to consume more temporal-wise features at the early stage and compute more channel-wise features at the late stage of the architecture.

Input: Table 10: Ablation experiments on dynamic modeling along temporal and channel dimensions. We choose R(2+1)D-18 on Mini-Kinetics-200 and set the search space to 2 in all the dynamic models.

Input: Figure 2: An illustration of dynamic convolution along temporal dimension (a) and channel dimension (b) respectively. Φt and Φs represent the temporal cheap operation and spatial cheap operation respectively. In (a), we multiply the temporal stride S with the factor R = 2pt to reduce computation, where pt is the temporal policy output by soft modulation gate. In (b), we compute part of output features with the ratio of r = ( 1 2 )pc , where pc is the channel policy. Best viewed in color.

Input: Figure 6: Visualization of temporal-wise feature maps. We plot the temporal feature maps which are fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps marked with red bounding boxes are cheaply generated. We do this analysis on 8-frame dynamic R(2+1)D-18 pretrained on Mini-Kinetics-200. These feature maps are the output of the first spatial convolution combined with ReLU non-linearity inside the ResBlock_1. We can see that most of the cheaply generated feature maps looks no difference from the original feature maps, which further support our approach. Best viewed in color.

Input: Figure 7: Visualization of channel-wise feature maps. We plot the feature maps across the channel dimension. We contrast two kinds of feature maps: fully computed by the original convolution and those mixed with cheaply generated feature maps. The feature maps inside the red bounding boxes are cheaply generated. The analysis is performed on 8-frame dynamic R(2+1)D-18 model which is pretrained on Mini-Kinetics-200 dataset and we extract these feature maps which are output by the first spatial convolution layer inside the ResBlock_1. Best viewed in color.

Input: Figure 1: Our VA-RED2 framework dynamically reduces the redundancy in two dimensions. Example 1 (left) shows a case where the input video has little movement. The features in the temporal dimension are highly redundant, so our framework fully computes a subset of features, and reconstructs the rest with cheap linear operations. In the second example, we show that our framework can reduce computational complexity by performing a similar operation over channels: only part of the features along the channel dimension are computed, and cheap operations are used to generate the rest.

Input: Figure 12: Validation video clips from Moments-In-Time. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Moments-In-Time. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 11: Validation video clips from Kinetics-400. For each category, we plot two input video clips which consume the most and the least computational cost respectively. We infer these video clips with 16-frame dynamic R(2+1)D-18 which is pre-trained on Kinetics-400. The percentage in the figure indicates the ratio of the actual computational cost of 2D convolution to that of the original fixed model. Best viewed in color.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Table 4: Comparison with CorrNet (Wang et al., 2020) and AR-Net (Meng et al., 2020) on Mini-Kinetics-200. We set the search space as 2 and train all the models with 16 frames.

Input: Table 5: Action recognition results on Kinetics-400. We set the search space as 2, meaning models can choose to compute all feature maps or 1 2 of them both on temporal and channel-wise convolutions.

Input: Figure 8: Ratio of computed feature per layer and class on Kinetics-400 dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 400 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed. While X3D-M tends to consume more temporalwise features at the early stage and compute more channel-wise features at the late stage, R(2+1)D choose to select fewer features at early stage by both temporal-wise and channel-wise policy. For both architectures, the channel-wise policy has more variation than the temporal-wise policy among different categories.

Input: Figure 9: Ratio of computed feature per layer and class on Moments-In-Time dataset. We visualize the per-block policy of X3D-M and R(2+1)D-18 on all 339 classes. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input: Figure 10: Computational cost distribution across different models on different datasets. We count the computation of each instance cost by different models on different datasets. For instance, for the upper-left one, we use the model backbone of R(2+1)D-18 on Mini-Kinetics-200. This sub-figure indicates that there are 87.7% of videos in Mini-Kinetics-200 (Dataset) consuming 38.6− 41.4 GFLOPs by using R(2+1)D-18 (Backbone), 8.8% of videos consuming 35.9− 38.6 GFLOPs, and 3.5% of videos consuming 41.4− 44.2 GFLOPs.

Input: Figure 3: Ratio of computed feature per layer and class on Mini-Kinetics-200 dataset. We pick the first 25 classes of Mini-Kinetics-200 and visualize the per-block policy of X3D-M on each class. Lighter color means fewer feature maps are computed while darker color represents more feature maps are computed.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that "Our approach is model-agnostic and can be applied to any backbones to reduce feature redundancy in both time and channel domains" appears partially overstated.

While the evidence shows successful application to several models (I3D, R(2+1)D, X3D, and PSPNet), indicating some level of generalizability, there are important limitations and caveats:

1. The authors mainly focus on 2D convolutions in R(2+1)D as stated: "We apply our method mainly to 2D convolutions in R(2+1)D"
2. The method requires specific architectural modifications (soft modulation gates, policy layers)
3. The testing is limited to a few specific architectures rather than "any backbones"
4. The term "model-agnostic" implies universal applicability, which is not fully demonstrated

The claim would be better stated by specifying the tested architectures and noting potential limitations.

<score>4</score>