\begin{table*}[t]
\caption{
Performance evaluations of the model on Cityscapes validation set.
}
\centerline{
\setlength{\tabcolsep}{3mm}{
\begin{tabular}{lccccc}
\toprule
    Methods   & \multicolumn{1}{l}{Backbone}  & \multicolumn{1}{l}{Coarse} & \multicolumn{1}{l}{ImageNet} & \multicolumn{1}{l}{mIOU(\%)} & \multicolumn{1}{l}{Params(M)} \\\hline
    
    PSPNet~\cite{PSANet:ECCV2018} & Dilated-ResNet-101  & $\times$  & $\checkmark$  &76.2  & $\times$   \\
    PSANet~\cite{PSANet:ECCV2018} & Dilated-ResNet-101   & $\times$  & $\checkmark$  & 77.3  & $\times$ \\
    PADNet~\cite{PADNet:CVPR2018} & Dilated-ResNet-101   & $\times$  & $\checkmark$  & 78.1  & $\times$  \\
    DenseASPP~\cite{DenseASPP:CVPR2018}  & WDenseNet-161    & $\times$  & $\checkmark$  & 77.8  & $\times$ \\
    DeepLabv3~\cite{DeepLabv3:2017}  & ResNet-101   & $\checkmark$  & $\checkmark$  & 79.5  & $\times$  \\\hline
    
    Auto-DeepLab~\cite{AutoDeepLab:CVPR2019}  & $\times$  & $\checkmark$  & $\times$   & 80.3  & 44.42 \\
    HRNetV2~\cite{HRNet:arxiv2019} & $\times$  & $\times$ & $\checkmark$  & 80.9  & 69.06 \\\hline
    Ours  & $\times$  & $\times$  & $\checkmark$  &  81.4  & 68.67 \\
    Ours + RSP  & $\times$  & $\times$  & $\checkmark$  & 81.0  & 17.20 \\
    
\bottomrule
\end{tabular}}
}    
\label{tbl:experiment}
\end{table*}
Based on Eq.(\ref{eq:LayerRelation}), we have $X_L = f_L$.  Then, Eq.(\ref{eq:GradientforLayer1}) can be rewritten as

For cell structure search, only one type of cell is used: the {\bf Normal Cell (Norm-Cell)} as in Fig.~\ref{fig:N:cell}. We use two intermediate nodes to construct each cell structure. Each cell keeps the number of channels and neither reduces the spatial dimension from $W \times H$ to $\frac{W}{2} \times \frac{H}{2} $ nor enlarges the spatial dimension from $W \times H$ to $2W \times 2H$.

Referring to Fig.~\ref{fig:overview} ({\em bottom left}) and Fig.~\ref{fig:N:cell}, for each Norm-Cell with one input, feature maps derived by previous layer with different shapes are reshaped by convolution.
The input maps are then fed into subsequent operations after adding together, which are searched by DARTS to generate a feature map with the same size $C \times {H} \times {W}$.  

Fig.~\ref{fig:PAM} depicts the details of this PAM.  Given a node, the idea of PAM aims at calculating the attention value of each path and choosing the number of outputs you want (denoted by red color in Fig.~\ref{fig:PAM}(a)) with higher attention to construct the final macrostructure. The PAM uses a $1\times1$ convolution and an interpolation operation to normalize all five inputs (denoted by four blue-arrows and one green-arrow in Fig.~\ref{fig:PAM}(b)) to the same size $C\times H\times W$. Then, they are sent to a {\bf Channel Attention Module (CAM)} to further calculate their channel attention.  Details of the CAM are depicted in Fig.~\ref{fig:CAM}.  For any pair of $i^{th}$ and $j^{th}$ inputs $F_{i}^c$ and $F_{j}^c$ on the $c^{th}$ channel, we perform an inner product between them to obtain channel attention $S^{c}_{i,j} \in R^{C \times C}$:
\vspace{-0.15cm}

where $S_{i,j}^c$ is the impact of the $i^{th}$ path on the $j^{th}$ path on the $c^{th}$ channel. Then, the attention $S_i^{Path}$ of the $i^{th}$ path is obtained as follows:
\vspace{-0.2cm}

The \textbf{Cityscapes} dataset~\cite{Cityscapes:CVPR2016} is a recent large-scale urban scene dataset containing a diverse set of stereo video sequences from 50 cities. Cityscapes dataset contains high quality pixel-level annotations of $5,000$ images with size $1,024 \times 2,048$. There are $2,975$, $500$, and $1,525$ for training, validation, and test images, respectively, and additional $20,000$ weakly annotated frames. It is an order of magnitude larger than similar previous datasets.

We consider 2 intermediate nodes in all cells with one input.
For each cell, we keep the channel numbers and the height and width of the feature tensor. Fig.~\ref{fig:N:cell_result} shows the searched cell architectures by NAS on the Cityscapes dataset. Fig.~\ref{fig:search_accuracy} shows that validation accuracy got more than 40\% during macro search. Compared to the accuracy in AutoDeepLab~\cite{AutoDeepLab:CVPR2019}, we got higher accuracy during search.
$512 \times 102$ random image crops are used. In DARTS search, batch size is 6 due to GPU memory limitation, architecture search optimization is conducted for $300$ epochs.
In the learning of network weight $w$, SGD optimizer with momentum $0.95$, and weight decay $0.0005$ are used. For learning the architecture, SGD optimizer with learning rate $0.005$ and weight decay $0.0001$ are used. The entire architecture search optimization takes about five days on two V100 GPUs.

Table~\ref{tbl:experiment} shows that our NAS model outperforms the SoTA on the Cityscapes.
Without any pretraining, our best model significantly outperforms all the SoTA method. Last row of Table~\ref{tbl:experiment} shows that the light-weigh RSP design uses only 1/4 parameter size of SoTA methods but still outperforms them.  Fig.~\ref{fig:cityscapes_results} shows that the visualization of our model on Cityscapes~\cite{Cityscapes:CVPR2016} validation and test set.
{"value": "76.2", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "PSPNet", "model settings": {"Backbone": "Dilated-ResNet-101", "Coarse": "false", "ImageNet pretraining": "true", "Params (M)": "unknown"}} {"value": "77.3", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "PSANet", "model settings": {"Backbone": "Dilated-ResNet-101", "Coarse": "false", "ImageNet pretraining": "true", "Params (M)": "unknown"}} {"value": "78.1", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "PADNet", "model settings": {"Backbone": "Dilated-ResNet-101", "Coarse": "false", "ImageNet pretraining": "true", "Params (M)": "unknown"}} {"value": "77.8", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "DenseASPP", "model settings": {"Backbone": "WDenseNet-161", "Coarse": "false", "ImageNet pretraining": "true", "Params (M)": "unknown"}} {"value": "79.5", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "DeepLabv3", "model settings": {"Backbone": "ResNet-101", "Coarse": "true", "ImageNet pretraining": "true", "Params (M)": "unknown"}} {"value": "80.3", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "Auto-DeepLab", "model settings": {"Backbone": "false", "Coarse": "true", "ImageNet pretraining": "false", "Params (M)": "44.42"}} {"value": "44.42", "type": "Hyper-parameter/Architecture", "model": "Auto-DeepLab", "parameter/architecture name": ["Params", "model size"], "dataset": "Cityscapes"} {"value": "80.9", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "HRNetV2", "model settings": {"Backbone": "false", "Coarse": "false", "ImageNet pretraining": "true", "Params (M)": "69.06"}} {"value": "69.06", "type": "Hyper-parameter/Architecture", "model": "HRNetV2", "parameter/architecture name": ["Params", "model size"], "dataset": "Cityscapes"} {"value": "81.4", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "Ours", "model settings": {"Backbone": "false", "Coarse": "false", "ImageNet pretraining": "true", "Params (M)": "68.67"}} {"value": "68.67", "type": "Hyper-parameter/Architecture", "model": "Ours", "parameter/architecture name": ["Params", "model size"], "dataset": "Cityscapes"} {"value": "81.0", "type": "Result", "training data/set": "Cityscapes", "test data/set": "Cityscapes validation", "task": ["semantic segmentation", "scene segmentation"], "metric": ["mIOU", "mean intersection-over-union"], "experimental settings": {"xx": "yy"}, "model": "Ours + RSP", "model settings": {"Backbone": "false", "Coarse": "false", "ImageNet pretraining": "true", "Params (M)": "17.20"}} {"value": "17.20", "type": "Hyper-parameter/Architecture", "model": "Ours + RSP", "parameter/architecture name": ["Params", "model size"], "dataset": "Cityscapes"}