\begin{table}[t]
    \centering
    
    \begin{tabular}{lccccccccccccc}
    \toprule
    
    Model & bad 1.0 & bad 2.0 & bad 4.0 \\
    
    \midrule
    GANet~\cite{zhang2019ga} & 6.56 & 1.10 & 0.54 \\
    AANet~\cite{xu2020aanet} & 5.01 & 1.66 & 0.75 \\
    CFNet~\cite{shen2021cfnet} & 3.31 & 0.77 & 0.31 \\
    RAFT-Stereo~\cite{lipson2021raft} & 2.44 & 0.44 & 0.15 \\
    CREStereo~\cite{li2022practical} & \textbf{0.98} & \textbf{0.22} & 0.10 \\
    GMStereo & 1.83 & 0.25 & \textbf{0.08} \\

    \bottomrule
    \end{tabular}
    \caption{\textbf{Stereo performance on ETH3D two-view stereo test set}. }
    \label{tab:stereo_eth3d_test}
\end{table}
\noindent \textbf{Training schedule.} For methodology comparison (Sec.~\ref{sec:method_comp}) and ablation experiments (Sec.~\ref{sec:ablation}), we first train our default $1/8$ feature resolution model on Chairs dataset for 100K iterations, with a batch size of 16 and a learning rate of 4e-4. We then finetune the model on Things dataset for 200K iterations, with a batch size of 8 and a learning rate of 2e-4. The models thus far all use bilinear upsamling to upsample to the full resolution flow prediction for simplicity. For later experiments, we use RAFT's convex upsampling~\cite{teed2020raft} and our models are trained on Things dataset for 800K iterations, which leads to improved performance~\cite{xu2022gmflow}. For the final fine-tuning process on Sintel and KITTI datasets for benchmark comparisons, we report details in Sec.~\ref{sec:flow_benchmark}.

\subsection{Stereo Matching}
\noindent \textbf{Datasets and evaluation setup.} We first train on the synthetic Scene Flow~\cite{mayer2016large} training set, and then evaluate on the Scene Flow test set and the KITTI 2015~\cite{menze2015object} training set for cross-dataset generalization. Unlike previous representative stereo networks~\cite{chang2018pyramid,zhang2019ga,xu2020aanet} that usually rely on a predefined disparity range (typically 192 pixels) to construct the local cost volume, our method is more flexible and can support unconstrained disparity prediction. To avoid extremely large disparity values in the data, we mask the pixels whose disparities exceed 400 pixels during both training and evaluation. Finally, we perform finetuning on KITTI 2015 Stereo, Middlebury Stereo, Argoverse Stereo and ETH3D Stereo datasets and report the performance on the online benchmarks.

\noindent \textbf{Training schedule.} For ablation experiments, we train our model on the ScanNet for 50K iterations, with a batch size of 80 and a learning rate of 4e-4. The depth range for training and testing is set to $[0.5, 10]$ meters, and the number of depth candidates in the matching layer (Eq.~\eqref{eq:depth_match}) is set to $64$. The training and testing process on SUN3D~\cite{xiao2013sun3d}, RGBD-SLAM~\cite{sturm2012benchmark} and Scenes11~\cite{ummenhofer2017demon} datasets will be elaborated in Sec.~\ref{sec:depth_benchmark}.

\noindent \textbf{Sintel.} Following RAFT~\cite{teed2020raft}, we further finetune our Things trained model on several mixed datasets that consist of KITTI 2015~\cite{menze2015object}, HD1K~\cite{kondermann2016hci}, FlyingThings3D~\cite{mayer2016large} and Sintel~\cite{butler2012naturalistic} training sets. We first finetune on the mixed dataset for 200K iterations with a batch size of 8 and a learning rate of 2e-4. Then we finetune on the Sintel training sets only with a larger crop size $416 \times 1024$ for 5K iterations. The batch size is 8 and the learning rate is 1e-4. To generate the flow prediction results on the Sintel test sets, we first resize the original images to $416 \times 1024$ and then resize the prediction back to the original image resolution for submission. The results on Sintel test set are shown in Table~\ref{tab:flow_sintel_test}. We achieve state-of-the-art results on the highly competitive Sintel (clean) dataset. On Sintel (final) dataset, our performance is only second to the recent FlowFormer~\cite{huang2022flowformer} model, which uses a Transformer model that is pretrained on the large scale ImageNet dataset and is more computationally expensive due to the large number of sequential refinements like RAFT. The visual comparisons with RAFT are shown in Fig.~\ref{fig:vis_sintel_compare}, our method can better capture the motion of fast-moving objects like the moving hand.

\noindent \textbf{KITTI.} We further finetune our VKITTI2 trained GMFlow+ model (Table~\ref{tab:flow_gen_kitti}) on the mixed KITTI 2012 and KITTI 2015 training sets for 30K iterations. The batch size is 8 and the learning rate is 2e-4. The comparison results with previous methods are shown in Table~\ref{tab:flow_kitti_test}. Again, we achieve better performance than RAFT.

\noindent \textbf{KITTI.} We first finetune our Scene Flow pretrained model on the Virtual KITTI 2~\cite{cabon2020virtual} dataset for 30K iterations, with a batch size of 16 and a learning rate of 4e-4. We then further finetune on the mixed KITTI 2012 and KITTI 2015 training sets for 10K iterations, with a batch size of 16 and a learning rate of 4e-4. The final model is used to generate the disparity prediction results on KITTI 2015 test set for submission to the online benchmark. The results are shown in Table~\ref{tab:stereo_kitti15_test}. We achieve competitive performance compared with the state-of-the-art methods LEAStereo~\cite{cheng2020hierarchical} and CREStereo~\cite{li2022practical}. Besides, our model runs about $2\times$ faster since we don't rely on any 3D convolutions (unlike LEAStereo) or a large number ($20+$) of sequential refinements (unlike CREStereo). Compared with previous lightweight stereo model AANet~\cite{xu2020aanet}, our method performs much better. Besides, our model can be implemented with pure PyTorch, without requiring to build additional CUDA ops like AANet, which demonstrates that our method achieves a better speed-accuracy trade-off and has more practical advantages. 

\noindent \textbf{Middlebury.} Following CREStereo~\cite{li2022practical}, we collect several public stereo datasets for training. More specifically, we first finetune the Scene Flow pretrained model on the mixed Scene Flow~\cite{mayer2016large}, Tartan Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, CARLA HR-VS~\cite{yang2019hierarchical}, CREStereo Dataset~\cite{li2022practical}, InStereo2K~\cite{bao2020instereo2k} and Middlebury~\cite{hirschmuller2007evaluation,scharstein2014high} datasets for 100K iterations. The batch size is 16 and the learning rate is 4e-4. We use a random crop size of $480\times 640$ for training at this stage. To adapt the model to high resolution (\emph{e.g}\onedot, $1536\times 2048$ for Middlebury), we perform another round of finetuning with a larger random crop size of $768\times 1024$. The training datasets include CARLA HR-VS~\cite{yang2019hierarchical}, CREStereo Dataset~\cite{li2022practical}, InStereo2K~\cite{bao2020instereo2k}, Falling Things~\cite{tremblay2018falling} and Middlebury~\cite{hirschmuller2007evaluation, scharstein2014high}. At inference, we resize all the Middlebury full resolution test images to $1536 \times 2048$ for prediction and finally resize the predicted disparities back to the original image resolution for evaluation. The results are shown in Table~\ref{tab:stereo_middlebury_test}. Our GMStereo achieves the first place in terms of the RMS (Root Mean Square) disparity error metric. Besides, our method shows much higher efficiency than CREStereo~\cite{li2022practical} ($5\times$ faster) and RAFT-Stereo~\cite{lipson2021raft} ($15 \times$ faster) on such a high-resolution dataset. We also show some visual comparisons in Fig.~\ref{fig:vis_middlebury_compare}, our method produces sharper object structures than CREStereo~\cite{li2022practical} and RAFT-Stereo~\cite{lipson2021raft}.

\noindent \textbf{ETH3D.} Similar to the training process on the Middlebury dataset, we use several public stereo datasets for training. More specifically, we first finetune the Scene Flow pretrained model on the mixed Scene Flow~\cite{mayer2016large}, Tartan Air~\cite{tartanair2020iros}, Sintel Stereo~\cite{butler2012naturalistic}, CREStereo Dataset~\cite{li2022practical}, InStereo2K~\cite{bao2020instereo2k} and ETH3D~\cite{schops2017multi} datasets for 100K iterations. The batch size is 24 and the learning rate is 4e-4. Then we perform another round of finetuning on the mixed CREStereo Dataset~\cite{li2022practical}, InStereo2K~\cite{bao2020instereo2k} and ETH3D~\cite{schops2017multi} datasets for 30K iterations. Again, the batch size is 24 and the learning rate is 4e-4. The final model is used to produce the prediction results for submission and the results are shown in Table~\ref{tab:stereo_eth3d_test}. We achieve the second place in terms of the `bad 1.0' and `bad 2.0' metrics and the first place in terms of the `bad 4.0' metric.
{"value": "6.56", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 1.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "GANet", "model settings": {"xx": "yy"}} {"value": "1.10", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 2.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "GANet", "model settings": {"xx": "yy"}} {"value": "0.54", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 3.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "GANet", "model settings": {"xx": "yy"}} {"value": "5.01", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 1.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "AANet", "model settings": {"xx": "yy"}} {"value": "1.66", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 2.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "AANet", "model settings": {"xx": "yy"}} {"value": "0.75", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 3.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "AANet", "model settings": {"xx": "yy"}} {"value": "3.31", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 1.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "CFNet", "model settings": {"xx": "yy"}} {"value": "0.77", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 2.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "CFNet", "model settings": {"xx": "yy"}} {"value": "0.31", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 3.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "CFNet", "model settings": {"xx": "yy"}} {"value": "2.44", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 1.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "RAFT-Stereo", "model settings": {"xx": "yy"}} {"value": "0.44", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 2.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "RAFT-Stereo", "model settings": {"xx": "yy"}} {"value": "0.15", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 3.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "RAFT-Stereo", "model settings": {"xx": "yy"}} {"value": "0.98", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 1.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "CREStereo", "model settings": {"xx": "yy"}} {"value": "0.22", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 2.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "CREStereo", "model settings": {"xx": "yy"}} {"value": "0.10", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 3.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "CREStereo", "model settings": {"xx": "yy"}} {"value": "1.83", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 1.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "GMStereo", "model settings": {"xx": "yy"}} {"value": "0.25", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 2.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "GMStereo", "model settings": {"xx": "yy"}} {"value": "0.08", "type": "Result", "training data/set": "Scene Flow Tartan Air Sintel Stereo CREStereo InStereo2K ETH3D", "test data/set": "ETH3D", "task": "stereo matching", "metric": "bad 3.0", "experimental settings": {"learning rate": "4e-4 4e-4", "batch size": "24 24", "iterations": "100K 30K"}, "model": "GMStereo", "model settings": {"xx": "yy"}}