Weakness
== Exposition ==
The exposition of the proposed method can be improved. For examples, - it’s unclear how the “Semantic Kernel Generation” is implemented. I can probably guess this step is essentially a 1 x 1 convolution, but it would be better to fill in the details. - In the introduction, the paper mentioned that the translated images often contain artifacts due to the nearest-neighbor upsampling. Yet, in the Feature Spatial Expansion step it also used nearest-neighbor interpolation. This needs some more justification. - For Figure 3, it’s unclear what the learned semantically adaptive kernel visualization means. At each upsampling layer, the kernel is only K x K. - At the end of Section 2, I do not understand what it means by “the semantics of the upsampled feature map can be stronger than the original one”. - The proposed upsampling layer is called “semantically aware”. However, I do not see anything that’s related to the semantics other than the fact that the input is a semantic map. I would suggest that this should be called “content aware” instead.
== Technical novelty == - My main concern about the paper lies in its technical novelty. There have been multiple papers that proposed content aware filter. As far as I know, the first one is [Xu et al. 2016]. Jia, Xu, et al. "Dynamic filter networks." Advances in neural information processing systems. 2016.
The work most relevant to this paper is the CARAFE [Wang et al. ICCV 2019], which can be viewed as a special case of the dynamic filter network for feature upsampling. By comparing Figure 2 in this paper and Figure 2 in the CARAFE paper [Wang et al. ICCV 2019], it seems to me that the two methods are * the same*. The only difference is the application of the layout-to-image task. The paper in the related work section suggests, “the settings of these tasks are significantly different from ours, making their methods cannot be used directly.” I respectfully disagree with this statement because both methods take in a feature tensor and produce an upsampled feature tensor. I believe that the application would be straightforward without any modification. Given the similarity to prior work, it would be great for the authors to (1) describe in detail the differences (if any) and (2) compare with prior work that also uses spatially varying upsampling kernel.
Minor: - CARAFE: Content-Aware ReAssembly of Features. The paper is in ICCV 2019, not CVPR 2019
In sum, I think the idea of the spatially adaptive upsampling kernel is technically sound. I also like the extensive evaluation in this paper. However, I have concerns about the high degree of similarity with the prior method and the lack of comparison with CARAFE.
** After discussions **
I have read other reviewers' comments. Many of the reviewers share similar concerns regarding the technical novelty of this work. I don't find sufficient ground to recommend acceptance of this paper.
- CARAFE: Content-Aware ReAssembly of Features. The paper is in ICCV 2019, not CVPR 2019 In sum, I think the idea of the spatially adaptive upsampling kernel is technically sound. I also like the extensive evaluation in this paper. However, I have concerns about the high degree of similarity with the prior method and the lack of comparison with CARAFE. ** After discussions ** I have read other reviewers' comments. Many of the reviewers share similar concerns regarding the technical novelty of this work. I don't find sufficient ground to recommend acceptance of this paper.