Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Since neural groundplans are 2D grids of features without perspective camera distortion, shift-equivariant processing using inexpensive 2D CNNs effectively completes occluded regions.

The review comment to be evaluated is:

Strength:

1. The authors tackle a challenging yet valuable problem, disentanglement of the movable and immovable components of the scene from a single image. They propose a neural scene representation called conditional neural groundplans and a self-supervised framework to disentangle static background and movable foreground objects given only a single image. The framework enables a variety of downstream tasks, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing. The direction would interest a wide range of communities, including 3D scene understanding, 3D neural representation, disentangled representation learning, and novel view synthesis.
2. The proposed 3D persistent scene representation learning framework elegantly leverages ideas from BEV representation, neural representation learning, volume rendering, and compositing operation (Yuan et al., 2021). Moreover, the proposed method utilizes multi-view videos (in a self-supervised manner) at training time to learn to separately reconstruct static and movable components of the scene from a single image at test time. The authors present a self-supervised neural scene representation framework for general scene understanding tasks. The reviewer found the work valuable to the community.
3. The proposed 3D scene representations enable novel view synthesis and disentangle movable and immovable components of the scene via diverse qualitative evaluation on two datasets, CLEVR and CoSY. In particular, the qualitative results shown on CoSY are impressive, as shown in the manuscript and supplementary website.

Weaknesses:

1. Persistent scene representation: By sufficient qualitative analysis, the proposed 3D scene representations enable novel view synthesis and disentangle movable and immovable elements for different scene understanding tasks. One failure case shown on the top row of p.17 is that the framework cannot effectively synthesize the viewing perspective behind the box with a green door. The framework seems unable to capture the background correctly, yielding incorrect synthesis. Moreover, the box's location seems incorrect from the images on the left. The reviewer wonder if the performance can be improved with additional observations.
2. The authors propose to have two separate groundplans neural representations for static and dynamic, respectively. As we know from the literature (e.g., Yuan et al., 2021), it is possible to keep separate 3D neural representations for novel view synthesis. It seems reasonable to tackle the goal of this paper, i.e., achieve decent performance on a variety of tasks from a single image, with separate 3D neural representations based on the proposed framework. It would be great for the authors to elaborate on the technical challenge of this direction. That could motivate the community to tackle the direction collectively.
3. Differentiable rendering based on neural groundplans feature: the reviewer found it less intuitive to understand why the differentiable rendering would work using the groundplan features g_x. Specifically, g_x is obtained based on the weighted sum of the features along the y-axis. The groundplan feature g_x contains information in a particular pillar. Why is it a better feature for volume rendering? Empirically, the design choice renders better quality than STOA methods (i.e., PixelNeRF and uORF). Could the authors please elaborate more on this design choice?
4. The design choice of BEV representation: The authors utilize a particular operation to form BEV representations, i.e., the weighted sum of the features. However, there are other approaches, e.g., Roddick and Cipolla, 2020, Philion and Fidler, 2020 and Saha et al., 2022. It would be valuable for the audience to learn the difference between these operations for representation learning.
5. The reviewer is interested in the qualitative performance of novel view synthesis without using the contracted coordinate. With the analysis, readers would be able to visualize the difference.

The evidence to be evaluated is:In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020). We similarly use axis-aligned 2D grids of features for self-supervised scene representation via neural rendering, but reconstruct them directly from few or a single 2D image observations.

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation.

Inferring a neural groundplan from one or several images proceeds in three steps: (1) feature extraction, (2) feature unprojection, (3) pillar aggregation. Given a single image \(\mathbf{I}\), we first extract per-pixel features via a CNN encoder to yield a feature tensor \(\mathbf{F}\).

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

Next, we aggregate each pillar into a point to create the 2D groundplan.

Associated to each 2D vertex of the groundplan is now a set of features \(\{\mathbf{f}_{i}\}_{i=1}^{N}\), where \(N\) is the number of samples along the \(y\)-dimension.

For each frame, we infer an _entangled_ neural groundplan as described in the previous section. Features in this entangled neural groundplan parameterize both static and dynamic features of the scene, for instance, a car as well as the road below it. We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

Note that our approach is compatible with prior work leveraging slot attention (Locatello et al., 2020; Kipf et al., 2021) and other inference modules, which can be run on the disentangled dynamic groundplan which, in contrast to image space, enables shift-equivariant processing free from perspective distortion and encodes 3D structure.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects. As expected from a non-generative method, regions that are entirely unconstrained such as occluded parts of the background (such as buildings) are blurry.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Following PixelNeRF, we use the first 4 convolutional blocks of the ResNet34 architecture to create a feature grid. For a \(128\times 128\) resolution image, this results in a feature grid of \(64\times 64\) resolution with \(512\) features per point. We further use a shallow CNN to reduce the number of feature channels, and to aggregate information along the y-axis.

This feature aggregation CNN has two convolutional hidden layers with 256 hidden units and a stride of 2 along the height of the feature tensor. The output is a \(16\times 64\) resolution feature grid with \(128\) features per point.

To construct an entangled groundplan, we aggregate the 3D feature volume into a 2D groundplan using a pillar aggregation MLP.

We use a shallow CNN with 4 hidden convolutional layers to disentangle the entangled groundplan of shape \(128\times 64\times 64\). The first two convolutional layers have 128 hidden units with kernel size of 3, stride of 1, and reflection padding of 1. The last two convolutional layers comprise of 256 hidden units with the same configuration for other parameters.

This shallow CNN outputs groundplan of shape \(256\times 256\times 256\), in which the first 128 channels of the feature tensor are attributed as the static groundplan and the rest of the 128 channels are used as the dynamic groundplan for that timestep.

Our method performs project of a 3D point on the image plane to get the corresponding feature tensor. This is performed using the intrinsic matrix of the camera. For a given 3D point in camera coordinates \(\mathbf{x_{c}}\) and intrinsic matrix \(\mathbf{K}\), the resulting 2D point on the image plane is computed as \(\mathbf{Kx}\). For projecting a 3D query point \(\mathbf{x}\) on the groundplan, we use grid sample using \((x,z)\)-coordinates of the 3D query points \(\mathbf{x}\).

For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view.

Since the groundplans have a spatial extent of \(256\times 256\), the phplic occupancy map is rendered at the same resolution.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim appears to be moderately overstated. While the neural groundplans do enable shift-equivariant processing using 2D CNNs and can complete some occluded regions, the evidence shows limitations:

1. The paper acknowledges that "regions that are entirely unconstrained such as occluded parts of the background (such as buildings) are blurry"
2. A reviewer points out a failure case where "the framework cannot effectively synthesize the viewing perspective behind the box"
3. The completion abilities are more modest than suggested - focusing mainly on predictable object completions rather than arbitrary occlusions

The evidence supports that the approach works, but not as "effectively" as claimed for all occluded regions.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Since neural groundplans are 2D grids of features without perspective camera distortion, shift-equivariant processing using inexpensive 2D CNNs effectively completes occluded regions.

The review comment to be evaluated is:

====> Strength
1. The authors introduced the bird’s eye-view (BEV) representation as a scene representation, which is novel in the area of neural rendering.
2. The disentanglement of the static and moving component of the scene adds more contribution to the proposed method. Although it is not new to automatically separate these components in the neural rendering community, using a vanilla 2D CNN to separate the feature representation is new to the reviewer.
3. The authors show that using a simple heuristic method, they can generate instance segmentation and bounding boxes for the dynamic components in the scene.
4. Good qualitative results are given.

====> Weakness
1. The details of the neural groundplans is confusing:

    i. In page 4, there are multiple $x$ such as $x'$, $x$, $x_c$, $x_i$. Their shape should be clarified. Whether they belong to 2D or 3D point, and also their coordinate system (i.e., the image frame coordinate or the world coordinate, and how to transform between these coordinates using the camera intrinsic)

    ii. For equation like $v(x ′ ) = F(π(C^{-1}(x ′ )))$, $f(x ′ ) = ...$, the parentheses here denote indexing the feature volume $v$ and feature tensor $F$. However, it is easy to confuse them with the functions like $\pi()$ and $D()$. The authors should consider using other formulation.

    iii.The $\pi(\cdot)$ is not clear. What is the meaning of it and how to learn $\pi(\cdot)$, is it learned per image I?

    iv. ``$g_x$ are the groundplan features for the point $x$ computed by projecting the coordinates onto the groundplane...'' How is the projection conducted? Is it projected by the camera extrinsic？

    v. In page 5, the authors say ``The per-frame static groundplans are pooled to obtain a single, time-independent static groundplan.'' How is the pooling process performed? The authors need to detail it.

2. The task setting is not clear. The authors should consider formulate the problem setting in a separated subsection (e.g., train one model for multiple scenes, all images are calibrated, the view density per scene, etc.)

3. It seems the compared baselines like pixelNerf and uORF are not proposed for the topic of separating dynamic and static representation. It would be fairer to compare with ``Neuraldiff: Segmenting 3d objects that move in egocentric videos.'' that used different components for the moving and static objects.

The evidence to be evaluated is:In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020). We similarly use axis-aligned 2D grids of features for self-supervised scene representation via neural rendering, but reconstruct them directly from few or a single 2D image observations.

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation.

Inferring a neural groundplan from one or several images proceeds in three steps: (1) feature extraction, (2) feature unprojection, (3) pillar aggregation. Given a single image \(\mathbf{I}\), we first extract per-pixel features via a CNN encoder to yield a feature tensor \(\mathbf{F}\).

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

Next, we aggregate each pillar into a point to create the 2D groundplan.

Associated to each 2D vertex of the groundplan is now a set of features \(\{\mathbf{f}_{i}\}_{i=1}^{N}\), where \(N\) is the number of samples along the \(y\)-dimension.

For each frame, we infer an _entangled_ neural groundplan as described in the previous section. Features in this entangled neural groundplan parameterize both static and dynamic features of the scene, for instance, a car as well as the road below it. We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

Note that our approach is compatible with prior work leveraging slot attention (Locatello et al., 2020; Kipf et al., 2021) and other inference modules, which can be run on the disentangled dynamic groundplan which, in contrast to image space, enables shift-equivariant processing free from perspective distortion and encodes 3D structure.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects. As expected from a non-generative method, regions that are entirely unconstrained such as occluded parts of the background (such as buildings) are blurry.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Following PixelNeRF, we use the first 4 convolutional blocks of the ResNet34 architecture to create a feature grid. For a \(128\times 128\) resolution image, this results in a feature grid of \(64\times 64\) resolution with \(512\) features per point. We further use a shallow CNN to reduce the number of feature channels, and to aggregate information along the y-axis.

This feature aggregation CNN has two convolutional hidden layers with 256 hidden units and a stride of 2 along the height of the feature tensor. The output is a \(16\times 64\) resolution feature grid with \(128\) features per point.

To construct an entangled groundplan, we aggregate the 3D feature volume into a 2D groundplan using a pillar aggregation MLP.

We use a shallow CNN with 4 hidden convolutional layers to disentangle the entangled groundplan of shape \(128\times 64\times 64\). The first two convolutional layers have 128 hidden units with kernel size of 3, stride of 1, and reflection padding of 1. The last two convolutional layers comprise of 256 hidden units with the same configuration for other parameters.

This shallow CNN outputs groundplan of shape \(256\times 256\times 256\), in which the first 128 channels of the feature tensor are attributed as the static groundplan and the rest of the 128 channels are used as the dynamic groundplan for that timestep.

Our method performs project of a 3D point on the image plane to get the corresponding feature tensor. This is performed using the intrinsic matrix of the camera. For a given 3D point in camera coordinates \(\mathbf{x_{c}}\) and intrinsic matrix \(\mathbf{K}\), the resulting 2D point on the image plane is computed as \(\mathbf{Kx}\). For projecting a 3D query point \(\mathbf{x}\) on the groundplan, we use grid sample using \((x,z)\)-coordinates of the 3D query points \(\mathbf{x}\).

For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view.

Since the groundplans have a spatial extent of \(256\times 256\), the phplic occupancy map is rendered at the same resolution.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the provided evidence, I find that the claim requires a nuanced evaluation:

The evidence shows that neural groundplans do enable completion of occluded regions using 2D CNNs, but with some important limitations. While the method successfully handles occlusions for dynamic objects like cars (as shown in the rotation examples), the paper explicitly states that "regions that are entirely unconstrained such as occluded parts of the background (such as buildings) are blurry." The claim somewhat overstates the effectiveness by not acknowledging these limitations and the fact that completion quality varies between static and dynamic elements.

Additionally, while the 2D CNN processing is relatively inexpensive compared to 3D alternatives, this comparative efficiency isn't directly demonstrated in the evidence.

<score>3</score>

Justification:
1. The claim correctly identifies the core capability but overstates its universal effectiveness
2. Evidence shows varying quality of completion (clear for cars, blurry for buildings)
3. The review points out several technical ambiguities in the groundplan implementation
4. Cost-effectiveness is implied but not explicitly proven in the evidence


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Since neural groundplans are 2D grids of features without perspective camera distortion, shift-equivariant processing using inexpensive 2D CNNs effectively completes occluded regions.

The review comment to be evaluated is:

Strengths include:
* The self-supervised training removes the need for GT annotation, e.g., object bounding boxes, for the supported downstream tasks.
* The decomposition of static and movable object feature spaces looks like quite an elegant solution to utilize the inductive bias in multi-view videos while enabling capabilities such as segmentation and object-level scene editing.
* The results overall look promising. Single-view/few-view novel view synthesis is a very challenging task, and the comparisons with PixelNeRF and uORF clearly demonstrated the advantages in synthesis quality. The analysis of adding more views (Fig.10) is strong evidence that the learned 3D representation can figure out how 2D views are related to, and hence help improve, 3D reconstruction.

Weaknesses include:
* There is no study of the generalization capability of the model. It seems likely that the learned models can overfit the limited appearances in the dataset (a few types of shapes/objects with mostly uniform colors).
* Related to the last point, the evaluation is completely done on synthesized datasets, and it's not clear how well the model can learn real-world data with much more diverse appearances. Since the method does not require GT annotations, some evaluation on real-world data does not seem too difficult.
* The are some quality issues with the presented view synthesis results. For example, while unobserved regions are understandably blurry, why are observed regions also not sharp? We can see from Fig.10 that adding more views does not help the quality of observed regions any further.
* Training requires multi-view videos. I can't find clarification on whether these need to be from static cameras. Nothing in the model appears to have such a requirement, but if so, that's a strong limitation. Moreover, why can't the model be trained on frames from a single moving camera?
* It's not clearly described how scene editing is done. It says that's enabled by "directly editing the neural groundplan". Does that mean editing the (movable object) feature space? The feature space is not sparse (i.e. locations without objects have non-zero features), so how does cropping in such space work, and why can that make sense?

The evidence to be evaluated is:In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020). We similarly use axis-aligned 2D grids of features for self-supervised scene representation via neural rendering, but reconstruct them directly from few or a single 2D image observations.

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation.

Inferring a neural groundplan from one or several images proceeds in three steps: (1) feature extraction, (2) feature unprojection, (3) pillar aggregation. Given a single image \(\mathbf{I}\), we first extract per-pixel features via a CNN encoder to yield a feature tensor \(\mathbf{F}\).

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

Next, we aggregate each pillar into a point to create the 2D groundplan.

Associated to each 2D vertex of the groundplan is now a set of features \(\{\mathbf{f}_{i}\}_{i=1}^{N}\), where \(N\) is the number of samples along the \(y\)-dimension.

For each frame, we infer an _entangled_ neural groundplan as described in the previous section. Features in this entangled neural groundplan parameterize both static and dynamic features of the scene, for instance, a car as well as the road below it. We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

Note that our approach is compatible with prior work leveraging slot attention (Locatello et al., 2020; Kipf et al., 2021) and other inference modules, which can be run on the disentangled dynamic groundplan which, in contrast to image space, enables shift-equivariant processing free from perspective distortion and encodes 3D structure.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects. As expected from a non-generative method, regions that are entirely unconstrained such as occluded parts of the background (such as buildings) are blurry.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Following PixelNeRF, we use the first 4 convolutional blocks of the ResNet34 architecture to create a feature grid. For a \(128\times 128\) resolution image, this results in a feature grid of \(64\times 64\) resolution with \(512\) features per point. We further use a shallow CNN to reduce the number of feature channels, and to aggregate information along the y-axis.

This feature aggregation CNN has two convolutional hidden layers with 256 hidden units and a stride of 2 along the height of the feature tensor. The output is a \(16\times 64\) resolution feature grid with \(128\) features per point.

To construct an entangled groundplan, we aggregate the 3D feature volume into a 2D groundplan using a pillar aggregation MLP.

We use a shallow CNN with 4 hidden convolutional layers to disentangle the entangled groundplan of shape \(128\times 64\times 64\). The first two convolutional layers have 128 hidden units with kernel size of 3, stride of 1, and reflection padding of 1. The last two convolutional layers comprise of 256 hidden units with the same configuration for other parameters.

This shallow CNN outputs groundplan of shape \(256\times 256\times 256\), in which the first 128 channels of the feature tensor are attributed as the static groundplan and the rest of the 128 channels are used as the dynamic groundplan for that timestep.

Our method performs project of a 3D point on the image plane to get the corresponding feature tensor. This is performed using the intrinsic matrix of the camera. For a given 3D point in camera coordinates \(\mathbf{x_{c}}\) and intrinsic matrix \(\mathbf{K}\), the resulting 2D point on the image plane is computed as \(\mathbf{Kx}\). For projecting a 3D query point \(\mathbf{x}\) on the groundplan, we use grid sample using \((x,z)\)-coordinates of the 3D query points \(\mathbf{x}\).

For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view.

Since the groundplans have a spatial extent of \(256\times 256\), the phplic occupancy map is rendered at the same resolution.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the evidence, the claim "Since neural groundplans are 2D grids of features without perspective camera distortion, shift-equivariant processing using inexpensive 2D CNNs effectively completes occluded regions" appears somewhat overstated (3.5/5). While the evidence shows that neural groundplans can help complete occluded regions through 2D CNN processing, both the paper and reviewer note quality limitations - particularly that unobserved regions are "blurry" and "not sharp." The claim overstates the effectiveness by not acknowledging these quality limitations in completion and lacks discussion of the method's constraints.

Detailed justification:

1. Supporting evidence:
- The method does enable completion of occluded regions as shown in the rotated car examples
- The 2D groundplan approach allows shift-equivariant processing free from perspective distortion
- The architecture uses relatively simple 2D CNNs

2. Limitations/Overstatements:
- The paper acknowledges "regions that are entirely unconstrained such as occluded parts of the background are blurry"
- The reviewer notes "unobserved regions are understandably blurry"
- The reviewer points out quality issues even in observed regions
- The evidence shows the method works mainly on synthetic data with limited object types/appearances
- The claim doesn't acknowledge these important quality limitations

The claim would be better stated if it acknowledged the quality limitations in the completion of occluded regions while still highlighting the architectural benefits of using 2D groundplans.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Since neural groundplans are 2D grids of features without perspective camera distortion, shift-equivariant processing using inexpensive 2D CNNs effectively completes occluded regions.

The review comment to be evaluated is:

Strength：
1. The conditional neural groundplans proposed are intriguing. This allows efficient processing of scene appearance and geometry directly in 3D, through the self-supervised learning of conditional neural groundplans, a hybrid discrete continuous representation of 3D neural scenes.
2. The proposed disentanglement of static background and movable foreground objects is novel. This approach uses object motion for training an encoder-based system that can reconstruct scenes from a single image instead of decomposing video frames based on scene specifics.
3. The proposed method is promising for understanding 3D from a single image. The dynamic groundplan encodes 3D geometry, allowing us to segment and animate objects in 3D using single images and 3D bounding boxes.

Weaknesses：
1. The biggest weakness is that only results on simple synthetic data are presented. Currently, there are a number of well-annotated 3D driving scenes available. An example is KITTI 360 (https://www.cvlibs.net/datasets/kitti-360/). Could the authors explain why results on the real datasets are not presented? Is it because of some specific annotation (such as BEV correspondence or the number of views) needed by the method? Otherwise, the generalizability on natural scenes and the practical value of the method are questioned.
2. I think there are some ambiguity behind the proposed Static-Dynamic Disentanglement. A definition of Static and Dynamic should be made clear in the proposed framework. A car remaining still in the sequence will be considered as Static. Considering a situation when two moving and static cars looks similar or exactly the same, how can we distinguish from a single image if a car is static or dynamic? The method should mention the above cases as an assumption, or making a clear definition of being Static and Dynamic.
3. Groundplans implicitly assume that objects are aligned on a plane. However, in real life, it is common that scenes/objects are not placed on a flat ground plane (e.g., a curved plane). The ability of persistent 3D scene representation is a little overclaimed, if we consider the implicit assumption behind groundplans.

The evidence to be evaluated is:In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020). We similarly use axis-aligned 2D grids of features for self-supervised scene representation via neural rendering, but reconstruct them directly from few or a single 2D image observations.

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation.

Inferring a neural groundplan from one or several images proceeds in three steps: (1) feature extraction, (2) feature unprojection, (3) pillar aggregation. Given a single image \(\mathbf{I}\), we first extract per-pixel features via a CNN encoder to yield a feature tensor \(\mathbf{F}\).

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

Next, we aggregate each pillar into a point to create the 2D groundplan.

Associated to each 2D vertex of the groundplan is now a set of features \(\{\mathbf{f}_{i}\}_{i=1}^{N}\), where \(N\) is the number of samples along the \(y\)-dimension.

For each frame, we infer an _entangled_ neural groundplan as described in the previous section. Features in this entangled neural groundplan parameterize both static and dynamic features of the scene, for instance, a car as well as the road below it. We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

Note that our approach is compatible with prior work leveraging slot attention (Locatello et al., 2020; Kipf et al., 2021) and other inference modules, which can be run on the disentangled dynamic groundplan which, in contrast to image space, enables shift-equivariant processing free from perspective distortion and encodes 3D structure.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects. As expected from a non-generative method, regions that are entirely unconstrained such as occluded parts of the background (such as buildings) are blurry.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Following PixelNeRF, we use the first 4 convolutional blocks of the ResNet34 architecture to create a feature grid. For a \(128\times 128\) resolution image, this results in a feature grid of \(64\times 64\) resolution with \(512\) features per point. We further use a shallow CNN to reduce the number of feature channels, and to aggregate information along the y-axis.

This feature aggregation CNN has two convolutional hidden layers with 256 hidden units and a stride of 2 along the height of the feature tensor. The output is a \(16\times 64\) resolution feature grid with \(128\) features per point.

To construct an entangled groundplan, we aggregate the 3D feature volume into a 2D groundplan using a pillar aggregation MLP.

We use a shallow CNN with 4 hidden convolutional layers to disentangle the entangled groundplan of shape \(128\times 64\times 64\). The first two convolutional layers have 128 hidden units with kernel size of 3, stride of 1, and reflection padding of 1. The last two convolutional layers comprise of 256 hidden units with the same configuration for other parameters.

This shallow CNN outputs groundplan of shape \(256\times 256\times 256\), in which the first 128 channels of the feature tensor are attributed as the static groundplan and the rest of the 128 channels are used as the dynamic groundplan for that timestep.

Our method performs project of a 3D point on the image plane to get the corresponding feature tensor. This is performed using the intrinsic matrix of the camera. For a given 3D point in camera coordinates \(\mathbf{x_{c}}\) and intrinsic matrix \(\mathbf{K}\), the resulting 2D point on the image plane is computed as \(\mathbf{Kx}\). For projecting a 3D query point \(\mathbf{x}\) on the groundplan, we use grid sample using \((x,z)\)-coordinates of the 3D query points \(\mathbf{x}\).

For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view.

Since the groundplans have a spatial extent of \(256\times 256\), the phplic occupancy map is rendered at the same resolution.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that "neural groundplans are 2D grids of features without perspective camera distortion, shift-equivariant processing using inexpensive 2D CNNs effectively completes occluded regions" appears somewhat overstated. While the evidence shows that neural groundplans do enable completion of occluded regions and allow for shift-equivariant processing, there are important limitations:

1. The evidence shows results primarily on synthetic data, as noted in the review comments
2. The completions of entirely unconstrained/occluded regions are described as "blurry" in the evidence
3. The method makes implicit assumptions about objects being aligned on a plane, which may not hold in real-world scenarios
4. The evidence doesn't explicitly demonstrate the comparative cost-effectiveness of the 2D CNN approach

These factors suggest the claim overstates the effectiveness and generalizability of the approach.

<score>4</score>

Justification:

1. The evidence shows the method works but with limitations that aren't acknowledged in the claim:
- "regions that are entirely unconstrained such as occluded parts of the background (such as buildings) are blurry"
- The reviewer notes "only results on simple synthetic data are presented"
- "Groundplans implicitly assume that objects are aligned on a plane" which limits real-world applicability

2. The term "effectively" in the claim implies a strong level of performance that isn't fully supported by the evidence presented. While the method does complete occluded regions, the quality and generalizability have notable limitations.

3. The claim presents the approach as a general solution but doesn't acknowledge the constraints and assumptions identified in both the paper and review comments.

4. The "inexpensive" aspect of using 2D CNNs isn't directly compared or substantiated with computational cost analyses in the evidence.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Our model thus outperforms prior pixel-aligned approaches in the synthesis of novel views that observe 3D regions that are occluded in the input view.

The evidence to be evaluated is:Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Pixel-aligned conditioning enables reconstruction of compositional scenes (Yu et al., 2020; Trevithick and Yang, 2021), but does not infer a compact 3D representation.

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects.

While PixelNeRF succeeds in novel view synthesis on CLEVR, renderings on the complex CoSY dataset show significant artifacts, possibly caused by the linear sampling employed by PixelNeRF.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements. By leveraging multi-view video at training time, we can reconstruct disentangled 3D representations from a single image observation at test time.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views.

We observe that as the target view shifts away from the context view, our method renders the geometry with appropriate texture of the car in the view, but outputs a blurry background as expected from a non-generative model.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

We outperform pixelNeRF, a strong baseline in this regime of 3D reconstruction from few images, both quantitatively and qualitatively.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 9: Visual quality of views for a given input image.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim "Our model thus outperforms prior pixel-aligned approaches in the synthesis of novel views that observe 3D regions that are occluded in the input view."

The evidence shows mixed support for this claim. While Figures 4, 9, and 10 demonstrate that the model can generate plausible novel views from occluded regions, particularly for cars and simple objects, the authors acknowledge limitations like "blurry backgrounds" and that reconstruction quality degrades as views move further from the input view. The comparison with PixelNeRF shows better performance on CoSY but comparable results on CLEVR. The claim could be more precise by specifying the types of scenes and conditions where their approach excels.

<score>3</score>

Justification:
1. The evidence shows improved performance in specific scenarios but not universal superiority
2. Visual comparisons in Figures 4 and 9 support better handling of occluded regions for vehicles/objects
3. Authors acknowledge limitations like blurry backgrounds and degrading quality
4. The claim would be more accurate if it specified:
   - The types of scenes where it performs better (e.g., automotive scenes)
   - The conditions under which improvements are observed
   - The specific aspects of novel view synthesis that are improved

The claim isn't entirely overstated but could benefit from more precise qualification of when and how it outperforms prior approaches.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Our model thus outperforms prior pixel-aligned approaches in the synthesis of novel views that observe 3D regions that are occluded in the input view.

The review comment to be evaluated is:

Strength:

1. The authors tackle a challenging yet valuable problem, disentanglement of the movable and immovable components of the scene from a single image. They propose a neural scene representation called conditional neural groundplans and a self-supervised framework to disentangle static background and movable foreground objects given only a single image. The framework enables a variety of downstream tasks, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing. The direction would interest a wide range of communities, including 3D scene understanding, 3D neural representation, disentangled representation learning, and novel view synthesis.
2. The proposed 3D persistent scene representation learning framework elegantly leverages ideas from BEV representation, neural representation learning, volume rendering, and compositing operation (Yuan et al., 2021). Moreover, the proposed method utilizes multi-view videos (in a self-supervised manner) at training time to learn to separately reconstruct static and movable components of the scene from a single image at test time. The authors present a self-supervised neural scene representation framework for general scene understanding tasks. The reviewer found the work valuable to the community.
3. The proposed 3D scene representations enable novel view synthesis and disentangle movable and immovable components of the scene via diverse qualitative evaluation on two datasets, CLEVR and CoSY. In particular, the qualitative results shown on CoSY are impressive, as shown in the manuscript and supplementary website.

Weaknesses:

1. Persistent scene representation: By sufficient qualitative analysis, the proposed 3D scene representations enable novel view synthesis and disentangle movable and immovable elements for different scene understanding tasks. One failure case shown on the top row of p.17 is that the framework cannot effectively synthesize the viewing perspective behind the box with a green door. The framework seems unable to capture the background correctly, yielding incorrect synthesis. Moreover, the box's location seems incorrect from the images on the left. The reviewer wonder if the performance can be improved with additional observations.
2. The authors propose to have two separate groundplans neural representations for static and dynamic, respectively. As we know from the literature (e.g., Yuan et al., 2021), it is possible to keep separate 3D neural representations for novel view synthesis. It seems reasonable to tackle the goal of this paper, i.e., achieve decent performance on a variety of tasks from a single image, with separate 3D neural representations based on the proposed framework. It would be great for the authors to elaborate on the technical challenge of this direction. That could motivate the community to tackle the direction collectively.
3. Differentiable rendering based on neural groundplans feature: the reviewer found it less intuitive to understand why the differentiable rendering would work using the groundplan features g_x. Specifically, g_x is obtained based on the weighted sum of the features along the y-axis. The groundplan feature g_x contains information in a particular pillar. Why is it a better feature for volume rendering? Empirically, the design choice renders better quality than STOA methods (i.e., PixelNeRF and uORF). Could the authors please elaborate more on this design choice?
4. The design choice of BEV representation: The authors utilize a particular operation to form BEV representations, i.e., the weighted sum of the features. However, there are other approaches, e.g., Roddick and Cipolla, 2020, Philion and Fidler, 2020 and Saha et al., 2022. It would be valuable for the audience to learn the difference between these operations for representation learning.
5. The reviewer is interested in the qualitative performance of novel view synthesis without using the contracted coordinate. With the analysis, readers would be able to visualize the difference.

The evidence to be evaluated is:Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Pixel-aligned conditioning enables reconstruction of compositional scenes (Yu et al., 2020; Trevithick and Yang, 2021), but does not infer a compact 3D representation.

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects.

While PixelNeRF succeeds in novel view synthesis on CLEVR, renderings on the complex CoSY dataset show significant artifacts, possibly caused by the linear sampling employed by PixelNeRF.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements. By leveraging multi-view video at training time, we can reconstruct disentangled 3D representations from a single image observation at test time.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views.

We observe that as the target view shifts away from the context view, our method renders the geometry with appropriate texture of the car in the view, but outputs a blurry background as expected from a non-generative model.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

We outperform pixelNeRF, a strong baseline in this regime of 3D reconstruction from few images, both quantitatively and qualitatively.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 9: Visual quality of views for a given input image.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find this claim requires careful evaluation. While the model does show impressive novel view synthesis capabilities for occluded regions (as demonstrated in Figures 4, 9, and 10), there are notable limitations acknowledged in both the paper and review comments. The qualitative results show that with single views, background reconstructions can be blurry and uncertain, improving only with additional views. The reviewer specifically points out a failure case where the framework "cannot effectively synthesize the viewing perspective behind the box." The claim would be more accurate if it acknowledged these limitations rather than making a broad superiority statement.

<score>4</score>

Justification:
1. The claim overstates performance by making an unqualified assertion of superiority over "prior pixel-aligned approaches" without acknowledging important limitations.

2. Evidence shows mixed results:
- Positive: Successfully renders novel views with "plausible completions" of unobserved parts
- Limitations:
  - "Outputs a blurry background as expected from a non-generative model"
  - Background reconstructions "improve with more input views"
  - Reviewer notes failure cases in synthesizing perspectives behind objects
  - Model "struggles to reconstruct high-quality details in the unobserved regions"

3. A more well-stated claim would qualify the performance advantages while acknowledging the specific conditions where the model excels vs where it has limitations.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Our model thus outperforms prior pixel-aligned approaches in the synthesis of novel views that observe 3D regions that are occluded in the input view.

The review comment to be evaluated is:

====> Strength
1. The authors introduced the bird’s eye-view (BEV) representation as a scene representation, which is novel in the area of neural rendering.
2. The disentanglement of the static and moving component of the scene adds more contribution to the proposed method. Although it is not new to automatically separate these components in the neural rendering community, using a vanilla 2D CNN to separate the feature representation is new to the reviewer.
3. The authors show that using a simple heuristic method, they can generate instance segmentation and bounding boxes for the dynamic components in the scene.
4. Good qualitative results are given.

====> Weakness
1. The details of the neural groundplans is confusing:

    i. In page 4, there are multiple $x$ such as $x'$, $x$, $x_c$, $x_i$. Their shape should be clarified. Whether they belong to 2D or 3D point, and also their coordinate system (i.e., the image frame coordinate or the world coordinate, and how to transform between these coordinates using the camera intrinsic)

    ii. For equation like $v(x ′ ) = F(π(C^{-1}(x ′ )))$, $f(x ′ ) = ...$, the parentheses here denote indexing the feature volume $v$ and feature tensor $F$. However, it is easy to confuse them with the functions like $\pi()$ and $D()$. The authors should consider using other formulation.

    iii.The $\pi(\cdot)$ is not clear. What is the meaning of it and how to learn $\pi(\cdot)$, is it learned per image I?

    iv. ``$g_x$ are the groundplan features for the point $x$ computed by projecting the coordinates onto the groundplane...'' How is the projection conducted? Is it projected by the camera extrinsic？

    v. In page 5, the authors say ``The per-frame static groundplans are pooled to obtain a single, time-independent static groundplan.'' How is the pooling process performed? The authors need to detail it.

2. The task setting is not clear. The authors should consider formulate the problem setting in a separated subsection (e.g., train one model for multiple scenes, all images are calibrated, the view density per scene, etc.)

3. It seems the compared baselines like pixelNerf and uORF are not proposed for the topic of separating dynamic and static representation. It would be fairer to compare with ``Neuraldiff: Segmenting 3d objects that move in egocentric videos.'' that used different components for the moving and static objects.

The evidence to be evaluated is:Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Pixel-aligned conditioning enables reconstruction of compositional scenes (Yu et al., 2020; Trevithick and Yang, 2021), but does not infer a compact 3D representation.

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects.

While PixelNeRF succeeds in novel view synthesis on CLEVR, renderings on the complex CoSY dataset show significant artifacts, possibly caused by the linear sampling employed by PixelNeRF.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements. By leveraging multi-view video at training time, we can reconstruct disentangled 3D representations from a single image observation at test time.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views.

We observe that as the target view shifts away from the context view, our method renders the geometry with appropriate texture of the car in the view, but outputs a blurry background as expected from a non-generative model.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

We outperform pixelNeRF, a strong baseline in this regime of 3D reconstruction from few images, both quantitatively and qualitatively.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 9: Visual quality of views for a given input image.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence provided, I'll evaluate the claim: "Our model thus outperforms prior pixel-aligned approaches in the synthesis of novel views that observe 3D regions that are occluded in the input view."

The evidence shows:
- Figures demonstrate the model's ability to render plausible completions of occluded regions, particularly for cars and objects
- Comparisons with PixelNeRF show the baseline having significant artifacts on complex datasets
- However, quantitative comparisons specifically focusing on occluded regions are not provided
- The reviewer notes "good qualitative results" but suggests fairer comparisons with other dynamic/static separation methods
- Multiple figures show reasonable novel view synthesis of occluded regions, especially with multiple input views

The claim appears somewhat overstated without comprehensive quantitative evidence specifically comparing occluded region reconstruction, though qualitative results support general improved performance.

<score>3</score>

The score of 3 reflects that while the model shows good performance on occluded regions through qualitative results, the claim would be better supported with:
1. Quantitative comparisons specifically for occluded regions
2. Comparisons with more directly relevant baselines as noted by the reviewer
3. More precise language acknowledging limitations and specific conditions where the improvement holds


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Our model thus outperforms prior pixel-aligned approaches in the synthesis of novel views that observe 3D regions that are occluded in the input view.

The review comment to be evaluated is:

Strengths include:
* The self-supervised training removes the need for GT annotation, e.g., object bounding boxes, for the supported downstream tasks.
* The decomposition of static and movable object feature spaces looks like quite an elegant solution to utilize the inductive bias in multi-view videos while enabling capabilities such as segmentation and object-level scene editing.
* The results overall look promising. Single-view/few-view novel view synthesis is a very challenging task, and the comparisons with PixelNeRF and uORF clearly demonstrated the advantages in synthesis quality. The analysis of adding more views (Fig.10) is strong evidence that the learned 3D representation can figure out how 2D views are related to, and hence help improve, 3D reconstruction.

Weaknesses include:
* There is no study of the generalization capability of the model. It seems likely that the learned models can overfit the limited appearances in the dataset (a few types of shapes/objects with mostly uniform colors).
* Related to the last point, the evaluation is completely done on synthesized datasets, and it's not clear how well the model can learn real-world data with much more diverse appearances. Since the method does not require GT annotations, some evaluation on real-world data does not seem too difficult.
* The are some quality issues with the presented view synthesis results. For example, while unobserved regions are understandably blurry, why are observed regions also not sharp? We can see from Fig.10 that adding more views does not help the quality of observed regions any further.
* Training requires multi-view videos. I can't find clarification on whether these need to be from static cameras. Nothing in the model appears to have such a requirement, but if so, that's a strong limitation. Moreover, why can't the model be trained on frames from a single moving camera?
* It's not clearly described how scene editing is done. It says that's enabled by "directly editing the neural groundplan". Does that mean editing the (movable object) feature space? The feature space is not sparse (i.e. locations without objects have non-zero features), so how does cropping in such space work, and why can that make sense?

The evidence to be evaluated is:Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Pixel-aligned conditioning enables reconstruction of compositional scenes (Yu et al., 2020; Trevithick and Yang, 2021), but does not infer a compact 3D representation.

The image features are unprojected to a 3D feature volume \(\mathbf{v}\) in contracted world space using the inverse of the contract function defined earlier. We extract the feature at a contracted 3D point \(\mathbf{x}^{\prime}\) in \(\mathbf{v}\) as \(\mathbf{v}(\mathbf{x}^{\prime})=\mathbf{F}[\pi(C^{-1}(\mathbf{x}^{\prime}))]\), where \(C^{-1}(\mathbf{x}^{\prime})\) first maps the contracted point to linear world space and \(\pi(\cdot)\) projects it onto the image plane of the context view using camera extrinsics and intrinsics.

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Note that our model produces novel views with plausible completions of parts of the scene that are unobserved in the context image such as the back-side of objects.

While PixelNeRF succeeds in novel view synthesis on CLEVR, renderings on the complex CoSY dataset show significant artifacts, possibly caused by the linear sampling employed by PixelNeRF.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively.

Note that cars are reliably encoded by the dynamic groundplan, and our method inpaints regions occluded in the input view.

Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements. By leveraging multi-view video at training time, we can reconstruct disentangled 3D representations from a single image observation at test time.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views.

We observe that as the target view shifts away from the context view, our method renders the geometry with appropriate texture of the car in the view, but outputs a blurry background as expected from a non-generative model.

Our method successfully integrates information across observations into a single, multi-view consistent representation. The background reconstructions improve with more input views, as a result of less uncertainty for the occluded regions.

We outperform pixelNeRF, a strong baseline in this regime of 3D reconstruction from few images, both quantitatively and qualitatively.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 9: Visual quality of views for a given input image.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim "Our model thus outperforms prior pixel-aligned approaches in the synthesis of novel views that observe 3D regions that are occluded in the input view" appears somewhat overstated. While Figures 4, 9, and 10 show the model can generate plausible completions for occluded regions and performs better than PixelNeRF and uORF in some cases, there are notable limitations:

1. The reviewer points out quality issues with blurry regions, both observed and unobserved
2. The evaluation is limited to synthetic datasets
3. The evidence shows reconstruction quality degrades significantly for occluded regions with just one input view
4. Results show uncertainty in reconstructing background/occluded regions that only improves with multiple views

<score>4</score>

Justification:
- While the model does show improvements over baselines, the claim overstates the degree of superiority by not acknowledging key limitations
- The evidence from Fig. 10 explicitly shows the model "struggles to reconstruct high-quality details in the unobserved regions"
- The reviewer notes "unobserved regions are understandably blurry"
- Limited evaluation on synthetic data raises questions about real-world generalization
- More measured wording like "shows improvements over" rather than "outperforms" would better reflect the evidence