Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

The ability to separately reconstruct movable objects enables a variety of downstream tasks using simple heuristics, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing.

The review comment to be evaluated is:

Strength:

1. The authors tackle a challenging yet valuable problem, disentanglement of the movable and immovable components of the scene from a single image. They propose a neural scene representation called conditional neural groundplans and a self-supervised framework to disentangle static background and movable foreground objects given only a single image. The framework enables a variety of downstream tasks, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing. The direction would interest a wide range of communities, including 3D scene understanding, 3D neural representation, disentangled representation learning, and novel view synthesis.
2. The proposed 3D persistent scene representation learning framework elegantly leverages ideas from BEV representation, neural representation learning, volume rendering, and compositing operation (Yuan et al., 2021). Moreover, the proposed method utilizes multi-view videos (in a self-supervised manner) at training time to learn to separately reconstruct static and movable components of the scene from a single image at test time. The authors present a self-supervised neural scene representation framework for general scene understanding tasks. The reviewer found the work valuable to the community.
3. The proposed 3D scene representations enable novel view synthesis and disentangle movable and immovable components of the scene via diverse qualitative evaluation on two datasets, CLEVR and CoSY. In particular, the qualitative results shown on CoSY are impressive, as shown in the manuscript and supplementary website.

Weaknesses:

1. Persistent scene representation: By sufficient qualitative analysis, the proposed 3D scene representations enable novel view synthesis and disentangle movable and immovable elements for different scene understanding tasks. One failure case shown on the top row of p.17 is that the framework cannot effectively synthesize the viewing perspective behind the box with a green door. The framework seems unable to capture the background correctly, yielding incorrect synthesis. Moreover, the box's location seems incorrect from the images on the left. The reviewer wonder if the performance can be improved with additional observations.
2. The authors propose to have two separate groundplans neural representations for static and dynamic, respectively. As we know from the literature (e.g., Yuan et al., 2021), it is possible to keep separate 3D neural representations for novel view synthesis. It seems reasonable to tackle the goal of this paper, i.e., achieve decent performance on a variety of tasks from a single image, with separate 3D neural representations based on the proposed framework. It would be great for the authors to elaborate on the technical challenge of this direction. That could motivate the community to tackle the direction collectively.
3. Differentiable rendering based on neural groundplans feature: the reviewer found it less intuitive to understand why the differentiable rendering would work using the groundplan features g_x. Specifically, g_x is obtained based on the weighted sum of the features along the y-axis. The groundplan feature g_x contains information in a particular pillar. Why is it a better feature for volume rendering? Empirically, the design choice renders better quality than STOA methods (i.e., PixelNeRF and uORF). Could the authors please elaborate more on this design choice?
4. The design choice of BEV representation: The authors utilize a particular operation to form BEV representations, i.e., the weighted sum of the features. However, there are other approaches, e.g., Roddick and Cipolla, 2020, Philion and Fidler, 2020 and Saha et al., 2022. It would be valuable for the audience to learn the difference between these operations for representation learning.
5. The reviewer is interested in the qualitative performance of novel view synthesis without using the contracted coordinate. With the analysis, readers would be able to visualize the difference.

The evidence to be evaluated is:Several works have explored learning neural scene representations for downstream tasks in 3D. Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Hybrid discrete-continuous neural scene representations offer a compromise by factorizing a dense 3D field into several lower-dimensional representations that are used to condition an MLP (Chan et al., 2022; Chen et al., 2022). In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020).

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

In contrast, we present a self-supervised conditional groundplan representation, learned only from images via neural rendering. While we show that our self-supervised representation can be used for rich inference tasks using simple heuristics, our method may be extended for more challenging tasks using the techniques developed in prior work.

Prior work has aimed to infer object-centric representations directly from images, with objects either represented as localized object-centric patches (Lin et al., 2020; Eslami et al., 2016; Crawford and Pineau, 2019; Kosiorek et al., 2018; Jiang et al., 2019) or scene mixture components (Engelcke et al., 2020; Burgess et al., 2019; Greff et al., 2019, 2016; 2017; Du et al., 2021), with the slot attention module (Locatello et al., 2020) increasingly driving object-centric inference. Resulting object representations may be decoded into object-centric 3D representations and composed for novel view synthesis (Yu et al., 2022; Smith et al., 2022; Elich et al., 2022; Chen et al., 2021; Bear et al., 2020; Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Du et al., 2021).

Several scene reconstruction methods (Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Nie et al., 2020) use direct supervision to train an object representation and detector to infer an editable 3D scene from a single frame observation.

In this work, we demonstrate that a representation factorized into static and movable 3D regions can serve as a powerful backbone for object discovery. While not explored in this work, slot attention and related object-centric algorithms could be run on our already sparse groundplan of movable 3D regions, faced with a dramatically easier task than when run on images directly.

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation. This feature is then concatenated with the vertical \(y\)-coordinate of the query point and decoded into radiance and density values via a fully connected network, enabling novel view synthesis using volume rendering (Mildenhall et al., 2020).

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Furthermore, we describe a method for performing self-supervised 3D object discovery and 3D bounding box prediction using the geometry encoded in the dynamic groundplan representation.

We leverage the fact that objects move in the given multi-view videos as the training signal.

We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

**Unsupervised object detection and extracting object-centric 3D representations**. Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example. Furthermore, we compute the smallest box that contains the connected component to get a 3D bounding box for each identified object. Finally, we crop tiles of the dynamic groundplan that belong to a given object instance to obtain object-centric 3D representations, enabling editing of 3D scenes such as deletion, insertion, and rigid-body transformation of objects.

We demonstrate that our method infers a 3D scene representation from a single image while disentangling static and dynamic components of the scene into static and dynamic groundplans respectively. We then show that connected components analysis suffices to leverage the densities in the dynamic groundplan for instance-level segmentation, bounding box prediction, and scene editing.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively. Fig. 5 shows results on single-image reconstruction of static and movable scene elements.

**Instance-level Segmentation and Bounding Box prediction**. The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction. Fig. 6 presents the instance-level segmentation and 3D bounding box prediction results of the proposed 3D object discovery via connected component discovery using the density inferred using the dynamic groundplan from the bird's-eye view.

**Scene Editing.** Instance-level segmentation, dynamic-static disentanglement, and 3D bounding boxes enable straight-forward 3D editing, such as translation, rotation, deletion, and insertion of individual objects in the scene. Objects can be rotated by arbitrary angles by simple bilinear interpolation of the groundplan features (refer to Appendix B.3). As the dynamic groundplan does not encode static scene regions such as the street below cars, cars can easily be moved from one scene to another. Fig. 8 provides scene editing results of our method. Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Future work may further explore the use of more sophisticated downstream processing of the groundplan to enable, for instance, prior-based inference of object-centric representations (Locatello et al., 2020).

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Learning object-compositional neural radiance field for editable scene rendering.

Neural rendering in a room: Amodal 3d understanding and free-viewpoint rendering for the closed scene composed of pre-captured objects.

Star: Self-supervised tracking and reconstruction of rigid objects in motion with neural rendering.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views. Thiscurriculum approach helps the network to first learn the relevant scene priors, before learning to complete the 3D structure of the scene. For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view. Along with the bird's-eye view RGB image, we also generate the occupancy map (per-pixel accumulated alpha) as shown in the manuscript in Fig. 5.

This information gives us a 3D bounding box around each of the localized object. The instance level segmentation is produced via volume rendering, by overriding the RGB values within the detected region to a chosen color for the object.

Scene editing is performed by editing the dynamic groundplan. Once the different objects have been localized in the dynamic groundplan, we can edit the dynamic groundplan to carry out object deletion, insertion, and rearrangement. Localizing an object gives us its features in the spatial region of the groundplan used for rendering. To delete the object, we replace the features in this regions with features from the dynamic groundplan that encode zero density. For inserting an object at a given location, we find the corresponding \((x,z)\) location in the dynamic groundplan and set the features at that location to the features corresponding to the object. Rearrangement of objects can be seen as a combination of deletion and insertion where we first delete the object from the existing location, followed by inserting the object at the new location in a possibly new orientation. To perform rotation of an object, we rotate the patch of features that correspond to the object to be rotated.

In addition to the rendering capabilities of CoSY, we add the ability to move cars over timesteps. Given the location and direction of the movement (direction where the car is facing), we change the location of the car over 10 timesteps to a sampled total translation from range 2-4 m.

Fig. 11 provides rendered images from edited groundplans where the cars are rotated at different angles. Note that the representation only allows for rotations in the \(xz\)-plane. The dynamic groundplan can be further used to perform motion interpolation in 3D, using optical flow for groundplan interpolation prediction using off-the-shelf, state-of-the-art optical flow and frame interpolation methods.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 5: Single-image reconstruction, disentanglement of static and dynamic objects, and novel view synthesis. Given a single input image, our method can disentangle the observed scene into static and object components based on what the model observed as not-moving and moving in the training data. In these examples, the cars are isolated in the object component as the model was training on video data of cars moving on the road.

Input: Figure 6: Object Localization. Given a single input image, we can use our inferred 3D representation of the scene to compute (a) bird’s-eye view rendering, (b) object localization in the bird’s-eye view space, (c) instance-level segmentation, and (d) 3D bounding box prediction.

Input: Figure 7: Qualitative Comparison for instance-level segmentation. Qualitative comparison of instance-level segmentation result with uORF (Yu et al., 2022).

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 8: Object-centric representations and scene editing. The proposed static-dynamic neural groundplans enables object discovery using connected components labeling on the density of objects encoded in the dynamic groundplan (left). This enables straight-forward scene editing such as deletion, addition, or rigid-body transformation via directly editing the neural groundplan (right).

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me evaluate the claim based on the provided evidence:

The claim that "the ability to separately reconstruct movable objects enables a variety of downstream tasks using simple heuristics" is well-supported by extensive experimental evidence. The figures and text demonstrate successful implementation of:
- Object-centric 3D representations (Figure 8)
- Novel view synthesis (Figures 4, 5, 10)
- Instance-level segmentation (Figures 1, 6, 7)
- 3D bounding box prediction (Figures 1, 6)
- Scene editing including deletion, insertion, rotation (Figures 8, 11)

The claim is careful to specify "simple heuristics" and the evidence shows these tasks are achieved through straightforward connected components analysis. The results are thoroughly documented across multiple datasets with clear comparisons to baselines.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

The ability to separately reconstruct movable objects enables a variety of downstream tasks using simple heuristics, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing.

The review comment to be evaluated is:

====> Strength
1. The authors introduced the bird’s eye-view (BEV) representation as a scene representation, which is novel in the area of neural rendering.
2. The disentanglement of the static and moving component of the scene adds more contribution to the proposed method. Although it is not new to automatically separate these components in the neural rendering community, using a vanilla 2D CNN to separate the feature representation is new to the reviewer.
3. The authors show that using a simple heuristic method, they can generate instance segmentation and bounding boxes for the dynamic components in the scene.
4. Good qualitative results are given.

====> Weakness
1. The details of the neural groundplans is confusing:

    i. In page 4, there are multiple $x$ such as $x'$, $x$, $x_c$, $x_i$. Their shape should be clarified. Whether they belong to 2D or 3D point, and also their coordinate system (i.e., the image frame coordinate or the world coordinate, and how to transform between these coordinates using the camera intrinsic)

    ii. For equation like $v(x ′ ) = F(π(C^{-1}(x ′ )))$, $f(x ′ ) = ...$, the parentheses here denote indexing the feature volume $v$ and feature tensor $F$. However, it is easy to confuse them with the functions like $\pi()$ and $D()$. The authors should consider using other formulation.

    iii.The $\pi(\cdot)$ is not clear. What is the meaning of it and how to learn $\pi(\cdot)$, is it learned per image I?

    iv. ``$g_x$ are the groundplan features for the point $x$ computed by projecting the coordinates onto the groundplane...'' How is the projection conducted? Is it projected by the camera extrinsic？

    v. In page 5, the authors say ``The per-frame static groundplans are pooled to obtain a single, time-independent static groundplan.'' How is the pooling process performed? The authors need to detail it.

2. The task setting is not clear. The authors should consider formulate the problem setting in a separated subsection (e.g., train one model for multiple scenes, all images are calibrated, the view density per scene, etc.)

3. It seems the compared baselines like pixelNerf and uORF are not proposed for the topic of separating dynamic and static representation. It would be fairer to compare with ``Neuraldiff: Segmenting 3d objects that move in egocentric videos.'' that used different components for the moving and static objects.

The evidence to be evaluated is:Several works have explored learning neural scene representations for downstream tasks in 3D. Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Hybrid discrete-continuous neural scene representations offer a compromise by factorizing a dense 3D field into several lower-dimensional representations that are used to condition an MLP (Chan et al., 2022; Chen et al., 2022). In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020).

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

In contrast, we present a self-supervised conditional groundplan representation, learned only from images via neural rendering. While we show that our self-supervised representation can be used for rich inference tasks using simple heuristics, our method may be extended for more challenging tasks using the techniques developed in prior work.

Prior work has aimed to infer object-centric representations directly from images, with objects either represented as localized object-centric patches (Lin et al., 2020; Eslami et al., 2016; Crawford and Pineau, 2019; Kosiorek et al., 2018; Jiang et al., 2019) or scene mixture components (Engelcke et al., 2020; Burgess et al., 2019; Greff et al., 2019, 2016; 2017; Du et al., 2021), with the slot attention module (Locatello et al., 2020) increasingly driving object-centric inference. Resulting object representations may be decoded into object-centric 3D representations and composed for novel view synthesis (Yu et al., 2022; Smith et al., 2022; Elich et al., 2022; Chen et al., 2021; Bear et al., 2020; Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Du et al., 2021).

Several scene reconstruction methods (Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Nie et al., 2020) use direct supervision to train an object representation and detector to infer an editable 3D scene from a single frame observation.

In this work, we demonstrate that a representation factorized into static and movable 3D regions can serve as a powerful backbone for object discovery. While not explored in this work, slot attention and related object-centric algorithms could be run on our already sparse groundplan of movable 3D regions, faced with a dramatically easier task than when run on images directly.

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation. This feature is then concatenated with the vertical \(y\)-coordinate of the query point and decoded into radiance and density values via a fully connected network, enabling novel view synthesis using volume rendering (Mildenhall et al., 2020).

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Furthermore, we describe a method for performing self-supervised 3D object discovery and 3D bounding box prediction using the geometry encoded in the dynamic groundplan representation.

We leverage the fact that objects move in the given multi-view videos as the training signal.

We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

**Unsupervised object detection and extracting object-centric 3D representations**. Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example. Furthermore, we compute the smallest box that contains the connected component to get a 3D bounding box for each identified object. Finally, we crop tiles of the dynamic groundplan that belong to a given object instance to obtain object-centric 3D representations, enabling editing of 3D scenes such as deletion, insertion, and rigid-body transformation of objects.

We demonstrate that our method infers a 3D scene representation from a single image while disentangling static and dynamic components of the scene into static and dynamic groundplans respectively. We then show that connected components analysis suffices to leverage the densities in the dynamic groundplan for instance-level segmentation, bounding box prediction, and scene editing.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively. Fig. 5 shows results on single-image reconstruction of static and movable scene elements.

**Instance-level Segmentation and Bounding Box prediction**. The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction. Fig. 6 presents the instance-level segmentation and 3D bounding box prediction results of the proposed 3D object discovery via connected component discovery using the density inferred using the dynamic groundplan from the bird's-eye view.

**Scene Editing.** Instance-level segmentation, dynamic-static disentanglement, and 3D bounding boxes enable straight-forward 3D editing, such as translation, rotation, deletion, and insertion of individual objects in the scene. Objects can be rotated by arbitrary angles by simple bilinear interpolation of the groundplan features (refer to Appendix B.3). As the dynamic groundplan does not encode static scene regions such as the street below cars, cars can easily be moved from one scene to another. Fig. 8 provides scene editing results of our method. Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Future work may further explore the use of more sophisticated downstream processing of the groundplan to enable, for instance, prior-based inference of object-centric representations (Locatello et al., 2020).

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Learning object-compositional neural radiance field for editable scene rendering.

Neural rendering in a room: Amodal 3d understanding and free-viewpoint rendering for the closed scene composed of pre-captured objects.

Star: Self-supervised tracking and reconstruction of rigid objects in motion with neural rendering.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views. Thiscurriculum approach helps the network to first learn the relevant scene priors, before learning to complete the 3D structure of the scene. For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view. Along with the bird's-eye view RGB image, we also generate the occupancy map (per-pixel accumulated alpha) as shown in the manuscript in Fig. 5.

This information gives us a 3D bounding box around each of the localized object. The instance level segmentation is produced via volume rendering, by overriding the RGB values within the detected region to a chosen color for the object.

Scene editing is performed by editing the dynamic groundplan. Once the different objects have been localized in the dynamic groundplan, we can edit the dynamic groundplan to carry out object deletion, insertion, and rearrangement. Localizing an object gives us its features in the spatial region of the groundplan used for rendering. To delete the object, we replace the features in this regions with features from the dynamic groundplan that encode zero density. For inserting an object at a given location, we find the corresponding \((x,z)\) location in the dynamic groundplan and set the features at that location to the features corresponding to the object. Rearrangement of objects can be seen as a combination of deletion and insertion where we first delete the object from the existing location, followed by inserting the object at the new location in a possibly new orientation. To perform rotation of an object, we rotate the patch of features that correspond to the object to be rotated.

In addition to the rendering capabilities of CoSY, we add the ability to move cars over timesteps. Given the location and direction of the movement (direction where the car is facing), we change the location of the car over 10 timesteps to a sampled total translation from range 2-4 m.

Fig. 11 provides rendered images from edited groundplans where the cars are rotated at different angles. Note that the representation only allows for rotations in the \(xz\)-plane. The dynamic groundplan can be further used to perform motion interpolation in 3D, using optical flow for groundplan interpolation prediction using off-the-shelf, state-of-the-art optical flow and frame interpolation methods.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 5: Single-image reconstruction, disentanglement of static and dynamic objects, and novel view synthesis. Given a single input image, our method can disentangle the observed scene into static and object components based on what the model observed as not-moving and moving in the training data. In these examples, the cars are isolated in the object component as the model was training on video data of cars moving on the road.

Input: Figure 6: Object Localization. Given a single input image, we can use our inferred 3D representation of the scene to compute (a) bird’s-eye view rendering, (b) object localization in the bird’s-eye view space, (c) instance-level segmentation, and (d) 3D bounding box prediction.

Input: Figure 7: Qualitative Comparison for instance-level segmentation. Qualitative comparison of instance-level segmentation result with uORF (Yu et al., 2022).

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 8: Object-centric representations and scene editing. The proposed static-dynamic neural groundplans enables object discovery using connected components labeling on the density of objects encoded in the dynamic groundplan (left). This enables straight-forward scene editing such as deletion, addition, or rigid-body transformation via directly editing the neural groundplan (right).

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim appears to be well-supported but slightly overstated. While the evidence demonstrates successful implementation of object detection, segmentation, 3D bounding boxes, and scene editing using the groundplan representation, some limitations exist. The claim implies these tasks are achieved through "simple heuristics," but the evidence shows more complex processing involving connected component analysis, density thresholds, and specialized training procedures. Additionally, novel view synthesis is demonstrated mainly for car scenes with relatively constrained scenarios.

<score>2</score>

Justification:
1. The evidence supports the model's ability to perform the claimed tasks through figures 1,5,6,8 showing successful segmentation, bounding box prediction, and scene editing

2. The methods, while effective, involve more sophisticated processing than just "simple heuristics":
- Connected component labeling on density values
- Multi-stage training curriculum (1-5 views)
- Specialized groundplan feature processing for rotations/edits
- Limited to specific object types (mainly cars) and ground plane rotations only

3. Relevant review comments highlight novelty but note some lack of clarity in technical details and fair comparisons

4. The score of 2 reflects that while the capabilities are demonstrated, characterizing them as achieved through "simple heuristics" somewhat understates the technical complexity involved.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

The ability to separately reconstruct movable objects enables a variety of downstream tasks using simple heuristics, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing.

The review comment to be evaluated is:

Strengths include:
* The self-supervised training removes the need for GT annotation, e.g., object bounding boxes, for the supported downstream tasks.
* The decomposition of static and movable object feature spaces looks like quite an elegant solution to utilize the inductive bias in multi-view videos while enabling capabilities such as segmentation and object-level scene editing.
* The results overall look promising. Single-view/few-view novel view synthesis is a very challenging task, and the comparisons with PixelNeRF and uORF clearly demonstrated the advantages in synthesis quality. The analysis of adding more views (Fig.10) is strong evidence that the learned 3D representation can figure out how 2D views are related to, and hence help improve, 3D reconstruction.

Weaknesses include:
* There is no study of the generalization capability of the model. It seems likely that the learned models can overfit the limited appearances in the dataset (a few types of shapes/objects with mostly uniform colors).
* Related to the last point, the evaluation is completely done on synthesized datasets, and it's not clear how well the model can learn real-world data with much more diverse appearances. Since the method does not require GT annotations, some evaluation on real-world data does not seem too difficult.
* The are some quality issues with the presented view synthesis results. For example, while unobserved regions are understandably blurry, why are observed regions also not sharp? We can see from Fig.10 that adding more views does not help the quality of observed regions any further.
* Training requires multi-view videos. I can't find clarification on whether these need to be from static cameras. Nothing in the model appears to have such a requirement, but if so, that's a strong limitation. Moreover, why can't the model be trained on frames from a single moving camera?
* It's not clearly described how scene editing is done. It says that's enabled by "directly editing the neural groundplan". Does that mean editing the (movable object) feature space? The feature space is not sparse (i.e. locations without objects have non-zero features), so how does cropping in such space work, and why can that make sense?

The evidence to be evaluated is:Several works have explored learning neural scene representations for downstream tasks in 3D. Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Hybrid discrete-continuous neural scene representations offer a compromise by factorizing a dense 3D field into several lower-dimensional representations that are used to condition an MLP (Chan et al., 2022; Chen et al., 2022). In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020).

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

In contrast, we present a self-supervised conditional groundplan representation, learned only from images via neural rendering. While we show that our self-supervised representation can be used for rich inference tasks using simple heuristics, our method may be extended for more challenging tasks using the techniques developed in prior work.

Prior work has aimed to infer object-centric representations directly from images, with objects either represented as localized object-centric patches (Lin et al., 2020; Eslami et al., 2016; Crawford and Pineau, 2019; Kosiorek et al., 2018; Jiang et al., 2019) or scene mixture components (Engelcke et al., 2020; Burgess et al., 2019; Greff et al., 2019, 2016; 2017; Du et al., 2021), with the slot attention module (Locatello et al., 2020) increasingly driving object-centric inference. Resulting object representations may be decoded into object-centric 3D representations and composed for novel view synthesis (Yu et al., 2022; Smith et al., 2022; Elich et al., 2022; Chen et al., 2021; Bear et al., 2020; Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Du et al., 2021).

Several scene reconstruction methods (Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Nie et al., 2020) use direct supervision to train an object representation and detector to infer an editable 3D scene from a single frame observation.

In this work, we demonstrate that a representation factorized into static and movable 3D regions can serve as a powerful backbone for object discovery. While not explored in this work, slot attention and related object-centric algorithms could be run on our already sparse groundplan of movable 3D regions, faced with a dramatically easier task than when run on images directly.

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation. This feature is then concatenated with the vertical \(y\)-coordinate of the query point and decoded into radiance and density values via a fully connected network, enabling novel view synthesis using volume rendering (Mildenhall et al., 2020).

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Furthermore, we describe a method for performing self-supervised 3D object discovery and 3D bounding box prediction using the geometry encoded in the dynamic groundplan representation.

We leverage the fact that objects move in the given multi-view videos as the training signal.

We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

**Unsupervised object detection and extracting object-centric 3D representations**. Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example. Furthermore, we compute the smallest box that contains the connected component to get a 3D bounding box for each identified object. Finally, we crop tiles of the dynamic groundplan that belong to a given object instance to obtain object-centric 3D representations, enabling editing of 3D scenes such as deletion, insertion, and rigid-body transformation of objects.

We demonstrate that our method infers a 3D scene representation from a single image while disentangling static and dynamic components of the scene into static and dynamic groundplans respectively. We then show that connected components analysis suffices to leverage the densities in the dynamic groundplan for instance-level segmentation, bounding box prediction, and scene editing.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively. Fig. 5 shows results on single-image reconstruction of static and movable scene elements.

**Instance-level Segmentation and Bounding Box prediction**. The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction. Fig. 6 presents the instance-level segmentation and 3D bounding box prediction results of the proposed 3D object discovery via connected component discovery using the density inferred using the dynamic groundplan from the bird's-eye view.

**Scene Editing.** Instance-level segmentation, dynamic-static disentanglement, and 3D bounding boxes enable straight-forward 3D editing, such as translation, rotation, deletion, and insertion of individual objects in the scene. Objects can be rotated by arbitrary angles by simple bilinear interpolation of the groundplan features (refer to Appendix B.3). As the dynamic groundplan does not encode static scene regions such as the street below cars, cars can easily be moved from one scene to another. Fig. 8 provides scene editing results of our method. Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Future work may further explore the use of more sophisticated downstream processing of the groundplan to enable, for instance, prior-based inference of object-centric representations (Locatello et al., 2020).

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Learning object-compositional neural radiance field for editable scene rendering.

Neural rendering in a room: Amodal 3d understanding and free-viewpoint rendering for the closed scene composed of pre-captured objects.

Star: Self-supervised tracking and reconstruction of rigid objects in motion with neural rendering.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views. Thiscurriculum approach helps the network to first learn the relevant scene priors, before learning to complete the 3D structure of the scene. For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view. Along with the bird's-eye view RGB image, we also generate the occupancy map (per-pixel accumulated alpha) as shown in the manuscript in Fig. 5.

This information gives us a 3D bounding box around each of the localized object. The instance level segmentation is produced via volume rendering, by overriding the RGB values within the detected region to a chosen color for the object.

Scene editing is performed by editing the dynamic groundplan. Once the different objects have been localized in the dynamic groundplan, we can edit the dynamic groundplan to carry out object deletion, insertion, and rearrangement. Localizing an object gives us its features in the spatial region of the groundplan used for rendering. To delete the object, we replace the features in this regions with features from the dynamic groundplan that encode zero density. For inserting an object at a given location, we find the corresponding \((x,z)\) location in the dynamic groundplan and set the features at that location to the features corresponding to the object. Rearrangement of objects can be seen as a combination of deletion and insertion where we first delete the object from the existing location, followed by inserting the object at the new location in a possibly new orientation. To perform rotation of an object, we rotate the patch of features that correspond to the object to be rotated.

In addition to the rendering capabilities of CoSY, we add the ability to move cars over timesteps. Given the location and direction of the movement (direction where the car is facing), we change the location of the car over 10 timesteps to a sampled total translation from range 2-4 m.

Fig. 11 provides rendered images from edited groundplans where the cars are rotated at different angles. Note that the representation only allows for rotations in the \(xz\)-plane. The dynamic groundplan can be further used to perform motion interpolation in 3D, using optical flow for groundplan interpolation prediction using off-the-shelf, state-of-the-art optical flow and frame interpolation methods.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 5: Single-image reconstruction, disentanglement of static and dynamic objects, and novel view synthesis. Given a single input image, our method can disentangle the observed scene into static and object components based on what the model observed as not-moving and moving in the training data. In these examples, the cars are isolated in the object component as the model was training on video data of cars moving on the road.

Input: Figure 6: Object Localization. Given a single input image, we can use our inferred 3D representation of the scene to compute (a) bird’s-eye view rendering, (b) object localization in the bird’s-eye view space, (c) instance-level segmentation, and (d) 3D bounding box prediction.

Input: Figure 7: Qualitative Comparison for instance-level segmentation. Qualitative comparison of instance-level segmentation result with uORF (Yu et al., 2022).

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 8: Object-centric representations and scene editing. The proposed static-dynamic neural groundplans enables object discovery using connected components labeling on the density of objects encoded in the dynamic groundplan (left). This enables straight-forward scene editing such as deletion, addition, or rigid-body transformation via directly editing the neural groundplan (right).

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and reviewer comments, I will evaluate the claim: "The ability to separately reconstruct movable objects enables a variety of downstream tasks using simple heuristics, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing."

The evidence demonstrates that the method successfully performs all the claimed downstream tasks through figures and detailed technical explanations. However, there are some limitations noted by reviewers: the evaluation is only on synthetic data, there are quality issues with view synthesis results, and scene editing processes aren't thoroughly explained. While the capabilities are real, describing them as enabled by "simple heuristics" somewhat understates the complexity of the technical approach required.

<score>2</score>

Justification:
1. The evidence shows clear demonstration of all claimed capabilities through multiple figures and experiments
2. The reviewer notes validate the core capabilities while highlighting some limitations
3. The term "simple heuristics" is slightly understated given the sophisticated neural groundplan architecture and processing required
4. The claim lists specific supported tasks rather than making broad generalizations
5. The limitations around synthetic-only evaluation and quality issues are real but don't invalidate the core claim about enabling these capabilities

While there is some minor understatement in characterizing the approach as "simple," the claim largely aligns with the demonstrated evidence and acknowledges the scope by listing specific supported tasks rather than making sweeping claims. Therefore, it leans toward being well-stated rather than overstated.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

The ability to separately reconstruct movable objects enables a variety of downstream tasks using simple heuristics, such as extraction of object-centric 3D representations, novel view synthesis, instance-level segmentation, 3D bounding box prediction, and scene editing.

The review comment to be evaluated is:

Strength：
1. The conditional neural groundplans proposed are intriguing. This allows efficient processing of scene appearance and geometry directly in 3D, through the self-supervised learning of conditional neural groundplans, a hybrid discrete continuous representation of 3D neural scenes.
2. The proposed disentanglement of static background and movable foreground objects is novel. This approach uses object motion for training an encoder-based system that can reconstruct scenes from a single image instead of decomposing video frames based on scene specifics.
3. The proposed method is promising for understanding 3D from a single image. The dynamic groundplan encodes 3D geometry, allowing us to segment and animate objects in 3D using single images and 3D bounding boxes.

Weaknesses：
1. The biggest weakness is that only results on simple synthetic data are presented. Currently, there are a number of well-annotated 3D driving scenes available. An example is KITTI 360 (https://www.cvlibs.net/datasets/kitti-360/). Could the authors explain why results on the real datasets are not presented? Is it because of some specific annotation (such as BEV correspondence or the number of views) needed by the method? Otherwise, the generalizability on natural scenes and the practical value of the method are questioned.
2. I think there are some ambiguity behind the proposed Static-Dynamic Disentanglement. A definition of Static and Dynamic should be made clear in the proposed framework. A car remaining still in the sequence will be considered as Static. Considering a situation when two moving and static cars looks similar or exactly the same, how can we distinguish from a single image if a car is static or dynamic? The method should mention the above cases as an assumption, or making a clear definition of being Static and Dynamic.
3. Groundplans implicitly assume that objects are aligned on a plane. However, in real life, it is common that scenes/objects are not placed on a flat ground plane (e.g., a curved plane). The ability of persistent 3D scene representation is a little overclaimed, if we consider the implicit assumption behind groundplans.

The evidence to be evaluated is:Several works have explored learning neural scene representations for downstream tasks in 3D. Emerging neural scene representations enable reconstruction of geometry and appearance from images as well as high-quality novel view synthesis via differentiable rendering.

Hybrid discrete-continuous neural scene representations offer a compromise by factorizing a dense 3D field into several lower-dimensional representations that are used to condition an MLP (Chan et al., 2022; Chen et al., 2022). In particular, neural groundplans and axis-aligned 2D grids enable high-quality unconditional generation of 3D scenes (DeVries et al., 2021; Chan et al., 2022) as well as reconstruction of 3D geometry from pointclouds (Peng et al., 2020).

Prior work uses ground-plane 2D grids as representations for object detection and segmentation (Saha et al., 2022; Harley et al., 2022; Philion and Fidler, 2020; Reiher et al., 2020; Roddick et al., 2019), layout generation and completion (Cao and de Charette, 2022; Jeong et al., 2022; Mani et al., 2020; Yang et al., 2021), and next-frame prediction (Hu et al., 2021; Zieba et al., 2020). The bird's-eye view is generated either directly without 3D inductive biases (Mani et al., 2020), or similar to our proposed approach, by using 3D geometry-driven inductive biases such as unprojection into a volume (Harley et al., 2022; Chen et al., 2022; Roddick et al., 2019), or by generating a 3D point cloud (Philion and Fidler, 2020; Hu et al., 2021).

In contrast, we present a self-supervised conditional groundplan representation, learned only from images via neural rendering. While we show that our self-supervised representation can be used for rich inference tasks using simple heuristics, our method may be extended for more challenging tasks using the techniques developed in prior work.

Prior work has aimed to infer object-centric representations directly from images, with objects either represented as localized object-centric patches (Lin et al., 2020; Eslami et al., 2016; Crawford and Pineau, 2019; Kosiorek et al., 2018; Jiang et al., 2019) or scene mixture components (Engelcke et al., 2020; Burgess et al., 2019; Greff et al., 2019, 2016; 2017; Du et al., 2021), with the slot attention module (Locatello et al., 2020) increasingly driving object-centric inference. Resulting object representations may be decoded into object-centric 3D representations and composed for novel view synthesis (Yu et al., 2022; Smith et al., 2022; Elich et al., 2022; Chen et al., 2021; Bear et al., 2020; Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Du et al., 2021).

Several scene reconstruction methods (Zakharov et al., 2020; Zekharov et al., 2020; Beker et al., 2020; Nie et al., 2020) use direct supervision to train an object representation and detector to infer an editable 3D scene from a single frame observation.

In this work, we demonstrate that a representation factorized into static and movable 3D regions can serve as a powerful backbone for object discovery. While not explored in this work, slot attention and related object-centric algorithms could be run on our already sparse groundplan of movable 3D regions, faced with a dramatically easier task than when run on images directly.

A neural groundplan is a 2D grid of features aligned with the ground plane of the 3D scene, which we define to be the \(xz-\)plane. A 3D point is decoded by projecting it onto the groundplan and retrieving the corresponding feature vector using bilinear interpolation. This feature is then concatenated with the vertical \(y\)-coordinate of the query point and decoded into radiance and density values via a fully connected network, enabling novel view synthesis using volume rendering (Mildenhall et al., 2020).

We can render images from novel camera views via differentiable volume rendering (DeVries et al., 2021; Lombardi et al., 2019; Mildenhall et al., 2020).

Furthermore, we describe a method for performing self-supervised 3D object discovery and 3D bounding box prediction using the geometry encoded in the dynamic groundplan representation.

We leverage the fact that objects move in the given multi-view videos as the training signal.

We feed this groundplan into a fully convolutional 2D network, which disentangles it into two separate groundplans containing _static_ and _dynamic_ features.

**Unsupervised object detection and extracting object-centric 3D representations**. Our formulation yields a model that maps a single image to two radiance fields, parameterizing static and dynamic 3D regions respectively.

We now perform a search for connected components in the dynamic neural groundplan to perform 3D instance-level segmentation, monocular 3D bounding box prediction, and the extraction of object-centric 3D representations. Specifically, given a dynamic groundplan, we first sample points in a 3D grid around the groundplan origin and decode their densities. We now perform conventional connected-component labeling in the groundplan space using accumulated density values, identifying the disconnected dynamic objects. We perform 2D instance-level segmentation for a queried viewpoint using volume rendering based on the densities expressed by the dynamic groundplan and assigning a color to the points corresponding to each of the identified objects, see Fig. 1 for an example. Furthermore, we compute the smallest box that contains the connected component to get a 3D bounding box for each identified object. Finally, we crop tiles of the dynamic groundplan that belong to a given object instance to obtain object-centric 3D representations, enabling editing of 3D scenes such as deletion, insertion, and rigid-body transformation of objects.

We demonstrate that our method infers a 3D scene representation from a single image while disentangling static and dynamic components of the scene into static and dynamic groundplans respectively. We then show that connected components analysis suffices to leverage the densities in the dynamic groundplan for instance-level segmentation, bounding box prediction, and scene editing.

We present novel views rendered from groundplans inferred from a single image from CLEVR and CoSY. For single-shot 3D reconstruction and novel view synthesis, we compare against PixelNeRF (Yu et al., 2020), a state-of-the-art single-image 3D reconstruction method, and uORF (Yu et al., 2022), state-of-the-art unsupervised object-centric 3D reconstruction method.

Given only a single image, our method computes separate static and dynamic groundplans that can be used to individually render the static and movable parts of the scene respectively. Fig. 5 shows results on single-image reconstruction of static and movable scene elements.

**Instance-level Segmentation and Bounding Box prediction**. The separate reconstruction of movable scene components in the dynamic groundplan enables object detection via instance-level segmentation and bounding box prediction. Fig. 6 presents the instance-level segmentation and 3D bounding box prediction results of the proposed 3D object discovery via connected component discovery using the density inferred using the dynamic groundplan from the bird's-eye view.

**Scene Editing.** Instance-level segmentation, dynamic-static disentanglement, and 3D bounding boxes enable straight-forward 3D editing, such as translation, rotation, deletion, and insertion of individual objects in the scene. Objects can be rotated by arbitrary angles by simple bilinear interpolation of the groundplan features (refer to Appendix B.3). As the dynamic groundplan does not encode static scene regions such as the street below cars, cars can easily be moved from one scene to another. Fig. 8 provides scene editing results of our method. Note that such editing is difficult with methods that lack a persistent 3D representation, such as PixelNeRF.

Future work may further explore the use of more sophisticated downstream processing of the groundplan to enable, for instance, prior-based inference of object-centric representations (Locatello et al., 2020).

Our paper demonstrates self-supervised learning of 3D scene representations that are disentangled into movable and immovable scene elements.

We show the potential of neural ground plans as a representation that may enable data-efficient solutions to downstream processing of the 3D scene, such as completion, instance-level segmentation, 3D bounding box prediction, and 3D scene editing.

Learning object-compositional neural radiance field for editable scene rendering.

Neural rendering in a room: Amodal 3d understanding and free-viewpoint rendering for the closed scene composed of pre-captured objects.

Star: Self-supervised tracking and reconstruction of rigid objects in motion with neural rendering.

Our model supports multi-view, as well as monocular reconstruction. We start our training only using multi-view reconstruction with 5 input views for the first 200 epochs, and then switch to a variable mode where the model is given a varying number of input views ranging between 1-5 views. Thiscurriculum approach helps the network to first learn the relevant scene priors, before learning to complete the 3D structure of the scene. For localization of objects, we consider the dynamic groundplan and render it from an orthographic bird's-eye view. Along with the bird's-eye view RGB image, we also generate the occupancy map (per-pixel accumulated alpha) as shown in the manuscript in Fig. 5.

This information gives us a 3D bounding box around each of the localized object. The instance level segmentation is produced via volume rendering, by overriding the RGB values within the detected region to a chosen color for the object.

Scene editing is performed by editing the dynamic groundplan. Once the different objects have been localized in the dynamic groundplan, we can edit the dynamic groundplan to carry out object deletion, insertion, and rearrangement. Localizing an object gives us its features in the spatial region of the groundplan used for rendering. To delete the object, we replace the features in this regions with features from the dynamic groundplan that encode zero density. For inserting an object at a given location, we find the corresponding \((x,z)\) location in the dynamic groundplan and set the features at that location to the features corresponding to the object. Rearrangement of objects can be seen as a combination of deletion and insertion where we first delete the object from the existing location, followed by inserting the object at the new location in a possibly new orientation. To perform rotation of an object, we rotate the patch of features that correspond to the object to be rotated.

In addition to the rendering capabilities of CoSY, we add the ability to move cars over timesteps. Given the location and direction of the movement (direction where the car is facing), we change the location of the car over 10 timesteps to a sampled total translation from range 2-4 m.

Fig. 11 provides rendered images from edited groundplans where the cars are rotated at different angles. Note that the representation only allows for rotations in the \(xz\)-plane. The dynamic groundplan can be further used to perform motion interpolation in 3D, using optical flow for groundplan interpolation prediction using off-the-shelf, state-of-the-art optical flow and frame interpolation methods.

The proposed method succeeds at inferring the correct object motion, and enables novel view synthesis through space and time.

Input: Figure 1: Given a single image, our model infers separate 3D representations for static and dynamic scene elements, enabling high-quality novel view synthesis with plausible completion, unsupervised instance-level segmentation, 3D bounding box prediction, 3D scene editing, and extraction of objectcentric 3D representations. Our model is trained self-supervised using unlabeled multi-view videos.

Input: Figure 10: Resolving uncertainty with increasing number of context views Our model can take a variable number of viewpoints of a scene as input. We find that the model gets a good prior on the object from a single view but struggles to reconstruct high-quality details in the unobserved regions of the background. This uncertainty gets resolved with more input views.

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input: Figure 4: Qualitative comparisons. Comparison for novel-view synthesis given a single context view with PixelNeRF (Yu et al., 2020) and uORF (Yu et al., 2022).

Input: Figure 5: Single-image reconstruction, disentanglement of static and dynamic objects, and novel view synthesis. Given a single input image, our method can disentangle the observed scene into static and object components based on what the model observed as not-moving and moving in the training data. In these examples, the cars are isolated in the object component as the model was training on video data of cars moving on the road.

Input: Figure 6: Object Localization. Given a single input image, we can use our inferred 3D representation of the scene to compute (a) bird’s-eye view rendering, (b) object localization in the bird’s-eye view space, (c) instance-level segmentation, and (d) 3D bounding box prediction.

Input: Figure 7: Qualitative Comparison for instance-level segmentation. Qualitative comparison of instance-level segmentation result with uORF (Yu et al., 2022).

Input: Figure 11: Scene editing with rotated objects. We present images of scenes with edited groundplans where cars are rotated at various angles in the ground plane.

Input: Figure 8: Object-centric representations and scene editing. The proposed static-dynamic neural groundplans enables object discovery using connected components labeling on the density of objects encoded in the dynamic groundplan (left). This enables straight-forward scene editing such as deletion, addition, or rigid-body transformation via directly editing the neural groundplan (right).

Input: Figure 12: Temporal 3D scene interpolation for novel view synthesis in space and time.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me evaluate the claim based on the provided evidence:

The evidence strongly supports that the method enables several downstream tasks through a static-dynamic disentanglement approach. The paper demonstrates:
- Instance-level segmentation via connected components analysis
- 3D bounding box prediction using density values
- Novel view synthesis through neural rendering
- Scene editing capabilities (deletion, insertion, rotation of objects)
- Extraction of object-centric representations

However, there are notable limitations highlighted in the review comments:
- Only tested on simple synthetic data, lacking real-world validation
- Assumes objects aligned on a plane, limiting real-world applicability
- Ambiguity in static-dynamic distinction for similar stationary/moving objects

While the capabilities are demonstrated, the claim somewhat overstates the simplicity ("simple heuristics") given these limitations and assumptions required.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In this paper, we propose THOMAS, a joint multi-agent trajectory prediction framework allowing for an efficient and consistent prediction of multi-agent multi-modal trajectories.

The evidence to be evaluated is:While very little work has directly tackled multi-agent prediction and evaluation so far, multiple methods hint at the ability to predict multiple agents at the same time (Liang et al., 2020; Ivanovic et al., 2020; Zeng et al., 2021) even if they then focus on a more single-agent oriented framework.

The approach most related to this paper is GOHOME (Gilles et al., 2021), which uses a similar graph encoder and then leverage lane rasters to generate a probability heatmap in a sparse manner.

Finally, none of these previous methods considers the problems of scene consistency for multi-agent prediction. Our goal is to predict the future \(T\) timesteps of \(A\) agents using their past history made of \(H\) timesteps and the HD-Map context. Similar to recent works (Zhao et al., 2020; Zeng et al., 2021; Gu et al., 2021), we will divide the problem into goal-based prediction followed by full trajectory reconstruction. Our prediction pipeline is displayed in Fig. 1. We first encode each agent trajectory and the HD-Map context graph into a common representation. We then decode a future probability heatmap for each agent in the scene, which we sample heuristically to maximize coverage. Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent. Our pipeline shares the same graph encoder, sampling algorithm and full trajectory generation as GOHOME (Gilles et al., 2021), but uses a novel efficient hierarchical heatmap process that enables to scale to simultaneous multi-agent prediction. Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

The final result is an encoding \(F_{a}\) for each agent, where history, context and interactions have been summarized. This encoding \(F_{a}\) is used in the next decoder operations and is also stored to be potentially used in modality recombination described in Sec. 3.2.

Our aim here is to decode each agent encoding into a heatmap representing its future probability distribution at prediction horizon \(T\). Since we create this heatmap for each agent in the scene, the decoding process has to be fast so that it can be applied to a great number of agents in parallel. We use hierarchical predictions at various levels of resolutions so that the decoder has the possibility of predicting over the full surroundings of the agent but learns to refine with more precision only in places where the agent will end up with high probability.

Starting from an initial full dense grid probability at low resolution \(R_{0}\times R_{0}\) by pixels, we iteratively refine the resolution by a fixed factor \(f\) until we reach the desired final resolution \(R_{final}\times R_{final}\). At each iteration \(i\), we select only the \(N_{i}\) highest ranking grid points from the previous iteration, and upsample only these points to the \(R_{i}\times R_{i}=\nicefrac{{R_{i-1}}}{{f}}\times\nicefrac{{R_{i-1}}}{{f}}\). At each step, the grid points features are computed by a 2-layer MLP applied on the point coordinates, these features are then concatenated to the agent encoding and passed through a linear layer, and finally enriched by a 2-layer cross-attention on the graph lane features, before applying a linear layer with sigmoid to get the probability.

Having a heatmap as output grants us the good property of letting us decide to privilege coverage in order to give a wide spectrum of candidates to the scene recombination process. From each heatmap, we therefore decode \(K\) end points using the same MissRate optimization algorithm as Gilles et al. (2021). We then generate the full trajectories for each end point using also the same model, a fully-connected MLP. The MLP takes the car history and predicted endpoint as input and produces \(T\) 2D-coordinates representing the full future trajectory.

The difficulty in multimodal multi-agent prediction resides in having coherent modalities between each agent.

Initially, the prediction output of the model can be defined as marginal, as all \(A\) agents have been predicted and sampled independently in order to get \(K\) possible endpoint each. Our goal is to output a set of scene predictions \(\mathcal{J}=(L,A)\) from the marginal prediction \(\mathcal{M}=(A,K)\), where each scene modality \(l\) belonging to \(\mathcal{J}\) is an association of endpoints \(p_{l}^{a}\) for each agent \(a\). To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions. For a given agent \(a\), the modality selected for the scene \(l\) would be a combination of this agent available marginal modalities \(p_{k}^{a}\in\mathcal{M}\). Each scene modality \(l\) would select a different association between the agents.

The model learns \(L\) scene modality embeddings \(S_{l}\) of \(D\) features each. \(K\times A\) agent modality vectors \(A_{k}^{a}\) are also derived from each agent modality endpoint. These vectors are obtained through a 2-layer MLP applied on the agent modality coordinates \(p_{k}^{a}\), to which the stored agent encoding \(F_{a}\) (previously described in Sec. 3.1.1) is concatenated in order to help the model recognise modalities from the same agent. The scene modality vectors \(S_{l}\) are'specialized' by querying the available modality proposals \(A_{k}^{a}\) of each agent through cross-attention layers. Subsequently, a matching score \(c_{l}^{k,a}\) between each scene modality \(S_{l}\) and each agent modality \(A_{k}^{a}\) is computed. This matching score can be intuitively interpreted as the selection of the role (maneuver \(k\)) that the agent \(a\) would play in the overall traffic scene \(l\): \[c_{l}^{k,a}=S_{l}.{A_{k}^{a}}^{T}\] Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\] The recombination module is trained with the scene-level minimum displacement error, where for each modality inside a single scene all predicted agent displacement losses are averaged before taking the minimum (winner-takes-all loss) over the \(L\) scene modalities. A formal definiton of this loss, also used as a metric, is given in Eq. 2 in Sec. 4.2 (minSFDE\({}_{k}\)).

We compare our THOMAS model performance with other joint predictions methods ILVM (Casas et al., 2020) and SceneTransformer (Ngiam et al., 2021).

While having comparable marginal distance performance (demonstrating that our model is not inherently more powerful or leveraging more information), THOMAS significantly outperforms other methods on every joint metric. SMR is improved by about 25\(\%\) and SCR by almost 30\(\%\), leading to a combined cSMR decreased by also more than 25\(\%\).

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We have presented THOMAS, a recombination module that can be added after any trajectory prediction module outputing multi-modal predictions. By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it. We show significant performance increase when adding the THOMAS module compared to the vanilla model and achieve state-of-the-art results compared to already existing methods tackling scene-consistent predictions.

The initial encoded \([A,D]\) tensor is obtained with a GOHOME encoder, multiplied across the \(F\) futures and concatenated with a modality-specific one-hot encoding as in Ngiam et al. (2021) to obtain the \([F,A,D]\) tensor. We then apply two layers of agent self-attention similar to the original paper, before decoding the endpoints through a MLP. We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Table 6: Argoverse Leaderboard (Chang et al.)

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 2: Model architecture for multi-agent prediction with shared backbone

Input: Figure 9: Qualitative examples of heatmap output from our multi-agent model. All the heatmaps from one scene are the results from one single forward pass in our model predicting all agents at once. We use matching colors for the agent history, current location and their future predicted heatmap (best viewed in color).

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 3: Hierarchical iterative refinement of the grid probabilities. First, the full grid is evaluated at a very low resolution, then the highest cells are up-sampled and evaluated at a higher resolution, until final resolution is reached. We highlight in grey the restricted area considered for refinement at each step.

Input: Figure 8: Detailed illustration of our heatmap generator model

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the proposed THOMAS framework is well-stated and supported by the detailed technical evidence provided. The paper thoroughly describes the novel hierarchical heatmap process and scene-consistency module, demonstrating significant performance improvements over baseline methods (e.g., 25% improvement in SMR and 30% in SCR). The evidence includes comprehensive architectural details, qualitative examples showing the model's effectiveness, and quantitative results validating the claims. The authors acknowledge building upon existing work (GOHOME) while clearly articulating their novel contributions. The claim accurately reflects the scope and achievements of the work without exaggeration.

<score>1</score>

Justification:
- The claim is precise and specific about what the framework does (joint multi-agent trajectory prediction)
- Evidence provides detailed technical implementation and clear performance metrics
- Results demonstrate concrete improvements over baselines while acknowledging limitations and prior work
- Claim language is measured and avoids overstating capabilities
- Experimental results directly support the stated contributions