Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We show that the representations learned by the model are interpretable and amenable to manipulations.

The review comment to be evaluated is:

Strengths:

+ The paper is clearly written and easy to follow.
+ It is novel and interesting to see object-centric learning models to infer 3D poses.
+ Validating the usefulness in downstream visual reasoning task is a plus.

Weakness:

My main conern is in the insufficient experiments that do not support the main contributions, i.e., the experiments do not demonstrate the benefits of the proposed 3D, structured object representations.

As for representations being structured (i.e., disentangled shape, appearance and poses):
- For disentanglement of shape and appearance, there is no experiments demonstrating this point. For example, this can be easily shown by swapping appearances of two objects in the scene while keeping their shapes unchanged, and vice versa.
- For disentanglement of 3D pose, the experiments include scaling and translation. However, there is no comparison to 2D baseline methods, which are able to do scaling and translation (at least the experiments on moving to the "right").

As for representations being 3D, there is no comparisons to show the benefits of 3D representation over 2D. In segmentation experiments, the results are worse than 2D baselines. In the visual reasoning task, the performance is worse than Aloe. Perhaps Aloe is better because it is designed for that task, but there is neither comparisons to 2D general object-centric baselines (such as original slot attention) or unstructured baselines.

Other minor weaknesses include:
- No ablation study on the model components to show the benefits of proposed technical contributions, such as the effect of regressing pose vs. not regressing pose (I suppose this should affect background decomposition because that is the only inductive bias that separates background slot from foreground slots).
- It would be good to demonstrate on datasets other than CLEVR.

The evidence to be evaluated is:Each object is composed of three tensors \(o_{i}=(o_{i}^{shape},o_{i}^{app},o_{i}^{pose})\). The object shape \(o_{i}^{shape}\in\mathbb{R}^{D_{shape}}\) and object appearance \(o^{app}\in\mathbb{R}^{D_{shape}}\) are tensors that respectively describe the 3D shape occupancy and color of an object with an implicit function. The object location \(o_{i}^{pose}\in\mathbb{R}^{4\times 4}\) is an affine matrix that describes the object 3D pose (i.e. scale, translation and rotation) in the scene.

To represent multiple 3D shapes with the same NeRF function, we can augment it with latent variables that determine which shape is being modeled (Schwarz et al., 2020). NeRFs are usually augmented with two random variables: one random variable \(\mu\in\mathbb{R}^{D_{shape}}\) defines the shape of the entity being modeled, while \(\upsilon\in\mathbb{R}^{D_{app}}\) models its appearance. In practice, this specialization is enforced by making the occupancy output a function of only the shape latent, while the color output is conditioned on the appearance latent.

In INFERNO, we share a single parametrization of a NeRF function across all objects. Each object specific shape and appearance is defined by the shape and appearance latent variables, which correspond to the object attributes \(o^{shape},o^{app}\).

Our inference mechanism computes image features through a neural network encoder and then extracts \(K\) object and a background slots. These slots are then mapped to our structured scene representation through learned neural networks. To extract image features for each object and background slots, we use Slot Attention (Locatello et al., 2020). Slot Attention is a mechanism that maps a set of \(K\) entities, called slots, to image features without annotations. It extracts image features \(I\in\mathbb{R}^{H\times W\times D}\) from a given input using a resolution-preserving convolutional encoder. These features are then attended to by a set of \(K\) randomly sampled slots \(\pi_{j}\sim\mathcal{N}(\mu,\sigma)\) of dimension \(D\), where \(\mu\) and \(\sigma\) are learnable parameters.

Object slots are unstructured tensors that result from aggregating image features. We map these slots to our structured scene representation through small fully-connected networks that operate on individual objects. More concretely, we map object slots to their 3D pose in the scene through a 2-layer MLP. To infer the object shape and appearance tensors we also use 2-layer MLPs, but we make them conditional on the predicted object pose through conditional normalization (Dumoulin et al., 2018).

Their factored representations are amenable to object manipulations.

When equipped with an inference mechanism, these models learn to segment objects in a scene without annotations, driven by their compositional generative process. MONet (B Burgess et al., 2019) implements a multi-object VAE that segments object sequentially by infering latents corresponding over parts of the scene not yet attended to iteratively. IODINE (Greff et al., 2019) uses a similar multi-object VAE and performs multiple rounds of inference to settle on a scene decomposition. Slot-Attention (Locatello et al., 2020) maps a set of entities, called slots, to image features through multiple rounds of soft attention. The slots compete among themselves to attend to features, making each slot attend to a region of the input image. When driven with alpha-compositing decoder, slot attention learns to segment objects in a scene. Our model uses a variant of Slot Attention to decide on which part of a 2D view should each object in our scene attend to, but we infer 3D-aware representations for each object.

Contrary to most previous approaches which segment 2D shapes, our method infers object-centric scene representations in 3D space.

First, we demonstrate the interpretability of the scene representations through manipulating scenes and verifying the corresponding effects in the rendered outputs.

Our model infers 3D object-centric scene representations from single 2D views. These representations disentangle the 3D appearance and pose of objects, which allows for semantic manipulations of the scene not possible otherwise. These manipulations can be validated by rendering the modified scene representations.

In INFERNO, we generate novel scenes by inferring representations for ground-truth images and then manipulating them. To compare to GIRAFFE, we manipulate scenes by adding additional objects and swapping object shapes and appearances across scenes.

We also investigate different interpretable manipulations of scene representations and visualize the effects in the corresponding output renderings.

We validate that our model is able to render out-of-distribution scenes not corresponding to training examples, such as scenes having 1 or 6 objects, and verify that the pose manipulations have semantically coherent effects.

Additionally, our model can perform inference and manipulations on existing scenes. We use that capability to generate novel scenes by manipulating existing ones. Our model is able to generate novel scenes with additional objects or with altered object shapes and appearances, with better FID than GIRAFFE. In Figure 2 we show some examples of the scene manipulations possible with our model. Given a scene representation, we can remove or add objects, rearrange object poses, translate the objects to new locations or change the object scales.

Figure 3 shows some examples of the discovered object masks. We can see that the model learns to segment different objects and properly discards object slots when the number of objects in the scene is lower than needed. We also include an example that our model fails to segment properly - multiple objects are segmented by the same object slot, and a single object is represented in multiple parts by different slots.

Once INFERNO is trained, we discard the rendering pipeline of our model, and instead feed the scene representations to a 12-layer transformer model to predict the final snitch position. Each object in our representation is given as an input element to the transformer. We add a learned positional encoding to the object representations based on their frame index.

The transformer last output is given to a MLP head that predicts the logits for the 36 possible output positions.

Following (Ding et al., 2020), we also consider the use of auxiliary SSL loss after pretraining. The SSL loss randomly masks one object per-frame and tries to predict its representation at the corresponding output. The model minimizes the L2 distance between the predicted object representation and the observed one.

This result suggests that the object-centric representation learned by INFERNO is useful for the visual reasoning task.

We observe that our model significantly outperforms the slot-attention model which focus on the 2D geometry of the scene.

Our model is able to discover objects in a scene without annotations, and the inferred scene representations are interpretable and amenable to manipulations.

The background and object slots are mapped to scene parameters using a series of MLP. For each object slot, we first map the object to its pose parameters. We use a 2-layer MLP with LayerNorm to map a slot to its pose.

Once we have inferred the object poses, we infer object shapes and appearances. These are inferred individually for each object using a common 2-layer MLP. The MLPs are conditional on the object pose using Conditional Layer Normalization, that makes the learned parameters of LayerNorm be a function of a condition. Specifically, we map the 7-dimensional pose tensor to LayerNorm parameters with a single linear layer with no activation or normalization. Shapes and appearances are defined by the output of the MLPs, which produce two 128-dimensional tensors.

Manipulations are done as follows:* _Substraction:_ We randomly delete up to two of the object slots.

* _Addition:_ We randomly add an object slot from another scene.

* _Scale:_ We reduce the scale (in all XYZ axis) of one of the objects in the substraction scene.

* _Forward:_ We manipulate the pose vector of one object in the substraction scene and move it forward on the Z axis.

* _Right:_ We manipulate the pose vector of one object in the substraction scene and move it forward on the X axis.

This transformations modifies a scene by replacing the object shape and appearance vectors with those of an object from another scene.

We discard the rendering pipeline of our model, and instead feed the inferred object slots representations to a transformer that aims at predicting the final position of the snitch. To predict the snitch, we consider a 12 layers transformer with the hidden dimension of 128 which takes the slot representation as input. The transformer treats each object as input element. A learned positional embedding is added each slots representation based on their frames index, i.e. the position of the objects is the same within a frame. The final output to the transformer is given to a 1 layer MLP head with an hidden dimension of 128.

We have included additional manipulations of one scene in GIF format in the supplementary material. We show: i) each object rendered individually, ii) object identity swaps with other scenes, iii) object translations along one axis, iv) translations in diagonal (two axis), and v) objects moving in a circle. In this section we synthesize novel views of a scene by modifying the camera pose. For each example we show the input image, our reconstruction, then two images as a result of moving the camera \(\pm 15^{\circ}\) in the azimuth axis, and two images as a result of zooming in the scene. While our model is trained without ground-truth camera poses and with single views of scenes, it is able to generalize to small camera pose modifications and render novel views of a scene

In this section we show the occupancy maps for each rendered object. The occupancy maps are obtained by integrating rays going from the image plane to the scene according to the NeRF density, but without taking into account the RGB output. We observe that each object is rendering a part of the scene corresponding to a single object instance.

To perform unsupervised object discovery we use the outputs of slot attention, since they do not require any upsampling and thus can provide more accurate segmentations.

Input: Figure 2: Manipulations on CLEVR: we show some examples of the manipulations we perform to CLEVR2345 images, including object removal and addition, changing the scale of an object, and object translation. Our model can perform these transformations because it disentangles object pose and appearance.

Input: Figure 3: Object Discovery on CLEVR6: INFERNO identifies the different objects in a scene without supervision. For each input image, we show which regions of the input are attended by each object as well as the background. We include an example of a failed segmentation in the last row, where one object slot (4) is trying to represent multiple objects at the same time.

Input: Figure 9: Object occupancy maps on CLEVR2345. For each input scene we show the reconstructed image as well as each individual object occupancy map. Occupancy maps are obtained by integrating the density outputs of the NeRF function along rays going from the image plane to the scene. We observe that each object is rendering a part of the scene corresponding to a single object instance. Outputs of the NeRF function have low resolution and are upscaled by a neural network. The last row shows an example of a bad grouping of two objects in the scene.

Input: Figure 1: Model Overview: We propose INFERNO, a model that infers and renders object-centric 3D scene representations. 1 Our model first decomposes an input observation into multiple object slots. 2 For each slot we infer a structured 3D representation. 3 The shape and appearance determine canonical objects rendered through NeRFs. 4 Objects are transformed and located in the overall scene according to their pose. 5 We combine objects and background and render a low-resolution scene given a camera location. 6 The input is reconstructed by upscaling the low resolution scene.

Input: Figure 6: Novel view synthesis on CLEVR2345. For each scene, we move the camera ±15◦ on the azimuth axis. Additionally, we zoom in the scene twice. While our model is trained with a fixed default camera pose and single scene views, it is able to generalize to small camera pose modifications and render novel views of a scene.

Input: Figure 7: NeRF outputs on CLEVR2345. For each input scene we show the reconstructed image as well as each the low resolution output of the NeRF function. This output is then upscaled with a neural network to obtain the reconstructed scene. While NeRF output lack full detail, they correctly depict each individual object and their position in the scene.

Input: Figure 5: Additional object additions and removals on CLEVR2345. For each scene, we show images with one randomly added object, and with 1-3 random objects removed. Some of these images show out-of-distribution samples with a number of objects not seen during training (2-5 objects).

Input: Figure 8: Shape and appearance disentanglement on CLEVR2345. For each input scene we alter the appearance vector of an object in the scene. First, we show the reconstructed scene. Then, for one of the objects in the scene, we replace its appearance vector by that of another object in the scene, while keeping its shape vector fixed. We perform this operation two times. We observe that, while the shading and color of the object changes as we change the appearance vector, its shape remains constant.

Input: Figure 4: Additional reconstructions on CLEVR2345.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the evidence, the claim that "the representations learned by the model are interpretable and amenable to manipulations" is well-supported by extensive experimental demonstrations. The evidence shows:

- Clear visualization of object segmentation (Fig 3)
- Successful object manipulations including addition, removal, scaling, and translation (Fig 2)
- Disentanglement of shape and appearance (Fig 8)
- Novel view synthesis capabilities (Fig 6)
- Individual object occupancy maps (Fig 9)

However, the reviewer points out valid limitations:
- Lack of experiments demonstrating shape/appearance disentanglement
- No comparisons with 2D baselines for pose manipulations
- Limited dataset testing (CLEVR only)

While these limitations suggest some overstatement, the core claim about interpretability and manipulation capabilities is substantially demonstrated through multiple visualization examples and manipulations.

<score>2</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks.

The evidence to be evaluated is:Given the rendering pipeline, the goal of our method is to infer representations that reconstruct a given scene. Our inference mechanism computes image features through a neural network encoder and then extracts \(K\) object and a background slots. These slots are then mapped to our structured scene representation through learned neural networks. To extract image features for each object and background slots, we use Slot Attention (Locatello et al., 2020).

When equipped with an inference mechanism, these models learn to segment objects in a scene without annotations, driven by their compositional generative process.

Slot-Attention (Locatello et al., 2020) maps a set of entities, called slots, to image features through multiple rounds of soft attention. The slots compete among themselves to attend to features, making each slot attend to a region of the input image. When driven with alpha-compositing decoder, slot attention learns to segment objects in a scene.

Contrary to most previous approaches which segment 2D shapes, our method infers object-centric scene representations in 3D space.

Then we show that it learns to identify and segment the objects in a scene without supervision. Finally, we highlight the usefulness of such representations for downstream tasks by applying our model on the CATER snitch localization task.

We also investigate different interpretable manipulations of scene representations and visualize the effects in the corresponding output renderings.

We validate that our model is able to render out-of-distribution scenes not corresponding to training examples, such as scenes having 1 or 6 objects, and verify that the pose manipulations have semantically coherent effects.

Additionally, our model can perform inference and manipulations on existing scenes. We use that capability to generate novel scenes by manipulating existing ones. Our model is able to generate novel scenes with additional objects or with altered object shapes and appearances, with better FID than GIRAFFE. In Figure 2 we show some examples of the scene manipulations possible with our model. Given a scene representation, we can remove or add objects, rearrange object poses, translate the objects to new locations or change the object scales. While some of these manipulations can be performed with regular object-centric models, modifications to the scale and location of the objects are hard to implement without explicitly modeling 3D object pose. Unsupervised object discovery consists in learning to segment the objects in a scene without using annotations.

Table 3 reports the ARI metric of our model and different baselines in this task. We observe that INFERNO is competitive with state-of-the-art methods, surpassing MONet and having slightly lower ARI than Slot-Attention and IDDINE. However, our model learns to segment objects in 3D, while the other baselines extract 2D segmentation masks. Figure 3 shows some examples of the discovered object masks. We can see that the model learns to segment different objects and properly discards object slots when the number of objects in the scene is lower than needed.

The goal of this experiment is to show that the representation learned by our model is useful for the snitch localization task from the CATER dataset (Girdhar and Ramanan, 2019). We focus on the CATER task involving videos with a static camera. The objective is to predict the final position of an object (the snitch) in a video. The scenes show multiple objects that move over time, one being the snitch object. The snitch can be occluded and moved around simultaneously by other objects, requiring object tracking and reasoning about dynamics to solve the task.

Once INFERNO is trained, we discard the rendering pipeline of our model, and instead feed the scene representations to a 12-layer transformer model to predict the final snitch position. Each object in our representation is given as an input element to the transformer. We add a learned positional encoding to the object representations based on their frame index. Objects in the same frame have the same positional encoding. The transformer last output is given to a MLP head that predicts the logits for the 36 possible output positions.

Following (Ding et al., 2020), we also consider the use of auxiliary SSL loss after pretraining. The SSL loss randomly masks one object per-frame and tries to predict its representation at the corresponding output. The model minimizes the L2 distance between the predicted object representation and the observed one.

We first compare our model that is pretrained to reconstruct image on CATER with a randomly initialized encoder. Using a randomly initialized encoder does not perform well. It indicates that INFERNO pretraining, and not the encoder architecture, is key to learn useful representation for the task.

We next compare our approach with Aloe (Ding et al., 2020), an object-centric baseline, R3D and R3D NL, 3D convolutional models proposed by (Girdhar and Ramanan, 2019) and architectures using strong inductive biases toward object tracking such as OPNet (Shamsian et al., 2020) or Hopper (Zhou et al., 2021). Our model outperforms the R3D and R3D NL models that rely on unstructured representation. This result suggests that the object-centric representation learned by INFERNO is useful for the visual reasoning task. Our model also outperforms Aloe, an object-centric method using 2D object representation, when both methods do not use additional SSL loss.

We observe that our model significantly outperforms the slot-attention model which focus on the 2D geometry of the scene. This result supports the advantage of 3D-aware representation for solving the CATER task.

Our model is able to discover objects in a scene without annotations, and the inferred scene representations are interpretable and amenable to manipulations. Further, the scene representation is useful for visual reasoning downstream tasks such as the snitch localization task in CATER.

For the visual reasoning task, we consider the CATER dataset and uses 5K videos that do not have camera motion.

We first pretrain our INFERNO to reconstruct individual frames from the CATER dataset. We bootstrap a model trained on CLEVR6 for object discovery for 400k iterations and train it on the CATER dataset for an additional 100k iterations to speed-up training, as the iteration time of a model with 10 objects is larger.

After pretraining, we finetune the INFERNO encoder to the supervised task of snitch localization. We discard the rendering pipeline of our model, and instead feed the inferred object slots representations to a transformer that aims at predicting the final position of the snitch. To predict the snitch, we consider a 12 layers transformer with the hidden dimension of 128 which takes the slot representation as input. The transformer treats each object as input element. A learned positional embedding is added each slots representation based on their frames index, i.e. the position of the objects is the same within a frame. The final output to the transformer is given to a 1 layer MLP head with an hidden dimension of 128. It outputs 36 logits that correspond to possible snitch location. We minimize the sum of the cross-entropy between the predicted position and the true target and a the l1 loss between the prediction and target.

To perform unsupervised object discovery we use the outputs of slot attention, since they do not require any upsampling and thus can provide more accurate segmentations.

Input: Figure 2: Manipulations on CLEVR: we show some examples of the manipulations we perform to CLEVR2345 images, including object removal and addition, changing the scale of an object, and object translation. Our model can perform these transformations because it disentangles object pose and appearance.

Input: Table 3: Object Discovery on CLEVR6 INFERNO, despite inferring more complex 3D object segmentations without annotations, is competitive with the current state-of-the-art 2D object discovery methods on CLEVR6.

Input: Figure 3: Object Discovery on CLEVR6: INFERNO identifies the different objects in a scene without supervision. For each input image, we show which regions of the input are attended by each object as well as the background. We include an example of a failed segmentation in the last row, where one object slot (4) is trying to represent multiple objects at the same time.

Input: Figure 9: Object occupancy maps on CLEVR2345. For each input scene we show the reconstructed image as well as each individual object occupancy map. Occupancy maps are obtained by integrating the density outputs of the NeRF function along rays going from the image plane to the scene. We observe that each object is rendering a part of the scene corresponding to a single object instance. Outputs of the NeRF function have low resolution and are upscaled by a neural network. The last row shows an example of a bad grouping of two objects in the scene.

Input: Figure 5: Additional object additions and removals on CLEVR2345. For each scene, we show images with one randomly added object, and with 1-3 random objects removed. Some of these images show out-of-distribution samples with a number of objects not seen during training (2-5 objects).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, the authors demonstrate competitive performance in object discovery tasks through quantitative metrics (Table 3 shows ARI scores close to SOTA) and qualitative results (Figures 2, 3, 5, 9 showing successful object segmentation and manipulation). For reasoning tasks, they show improved performance on the CATER snitch localization task compared to baseline models, particularly outperforming R3D, R3D NL and Aloe. The evidence substantiates both aspects of the claim with concrete experimental results and comparisons, making it well-stated without exaggeration.

<score>1</score>

Justification:
1. Object Discovery:
- Quantitative evidence: ARI of 96.7% vs SOTA 98.8% (Table 3)
- Qualitative evidence: Multiple figures demonstrating successful segmentation and manipulation
- Acknowledges limitations (occasional failed segmentations shown in figures)

2. Reasoning Tasks:
- Clear experimental setup for CATER task
- Direct performance comparisons with multiple baselines
- Outperforms relevant baselines (R3D, R3D NL, Aloe)
- Specific task details and methodology provided
- Acknowledges role of pretraining vs random initialization

The claim accurately reflects these demonstrated capabilities without overstating the results.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks.

The review comment to be evaluated is:

Strengths:

1. It is interesting to use NeRFs to represent objects in a scene, which encourages the understanding of 2D visual scene taking account into its 3D structure.
2. Explicitly modeling the 3D structure of 2D scene may help better deal with occlusion, ambiguity from similar appearance which can be a big challenge in 2D scene understanding.
3. It looks great to show how learned object features can be used for downstream tasks.

Weaknesses:
1. The technical novelty is somewhat limited. The paper mainly combines the Slot-attention and NeRF. Personally, it is hard to identify the exciting point(s).

2. The motivation of this work seems not well justified. It is claimed that the model is proposed to infer 3D representation for objects that are interpretable and amenable. But the experiments presented with such 3D representation cannot show its superiority and it is
hard to see the necessity of 3D object representation in this case considering the task presented can also be fulfilled with 2D object representation. In terms of the interpretability and manipulability of latent code, this paper presents objects removal/addition, object scaling and object translation. All of the above operations can actually be done just by explicitly modeling 2D object pose such as AIR (NIPS'16 paper). In terms of scene understanding in the form of object segmentation, this model cannot beat Slot Attention. In terms of downstream tasks, this model is not compared with other object-centric models. Thus, based on your current experiments, the motivation for learning such 3D representation for objects is not well justified.

3. Is it appropriate to use NeRF? If I understand correctly, NeRF learns the radiance field of a scene with 2D images rendered from different viewpoints. This model tries to recover the object radiance field from a single view of the objects. But the model training
objective can actually be fulfilled by just learning a single-view of the scene since there is only one image for each scene to generate reconstruction loss.

4. From my understanding, the 3D rendering model in this paper seems more similar to a decoder in VAE-wise architecture. It could be more convincing to provide some novel view synthesis (not novel scene) to claim it as a NeRF. For example, you may consider
object rotation.

5. All experiment datasets are a bit too simple. It is unclear how good/bad the proposed model will perform on realistic 3D scenes such as LLFF dataset or ScanNet. Otherwise, it is hard to see the real impact of this proposed method.

The evidence to be evaluated is:Given the rendering pipeline, the goal of our method is to infer representations that reconstruct a given scene. Our inference mechanism computes image features through a neural network encoder and then extracts \(K\) object and a background slots. These slots are then mapped to our structured scene representation through learned neural networks. To extract image features for each object and background slots, we use Slot Attention (Locatello et al., 2020).

When equipped with an inference mechanism, these models learn to segment objects in a scene without annotations, driven by their compositional generative process.

Slot-Attention (Locatello et al., 2020) maps a set of entities, called slots, to image features through multiple rounds of soft attention. The slots compete among themselves to attend to features, making each slot attend to a region of the input image. When driven with alpha-compositing decoder, slot attention learns to segment objects in a scene.

Contrary to most previous approaches which segment 2D shapes, our method infers object-centric scene representations in 3D space.

Then we show that it learns to identify and segment the objects in a scene without supervision. Finally, we highlight the usefulness of such representations for downstream tasks by applying our model on the CATER snitch localization task.

We also investigate different interpretable manipulations of scene representations and visualize the effects in the corresponding output renderings.

We validate that our model is able to render out-of-distribution scenes not corresponding to training examples, such as scenes having 1 or 6 objects, and verify that the pose manipulations have semantically coherent effects.

Additionally, our model can perform inference and manipulations on existing scenes. We use that capability to generate novel scenes by manipulating existing ones. Our model is able to generate novel scenes with additional objects or with altered object shapes and appearances, with better FID than GIRAFFE. In Figure 2 we show some examples of the scene manipulations possible with our model. Given a scene representation, we can remove or add objects, rearrange object poses, translate the objects to new locations or change the object scales. While some of these manipulations can be performed with regular object-centric models, modifications to the scale and location of the objects are hard to implement without explicitly modeling 3D object pose. Unsupervised object discovery consists in learning to segment the objects in a scene without using annotations.

Table 3 reports the ARI metric of our model and different baselines in this task. We observe that INFERNO is competitive with state-of-the-art methods, surpassing MONet and having slightly lower ARI than Slot-Attention and IDDINE. However, our model learns to segment objects in 3D, while the other baselines extract 2D segmentation masks. Figure 3 shows some examples of the discovered object masks. We can see that the model learns to segment different objects and properly discards object slots when the number of objects in the scene is lower than needed.

The goal of this experiment is to show that the representation learned by our model is useful for the snitch localization task from the CATER dataset (Girdhar and Ramanan, 2019). We focus on the CATER task involving videos with a static camera. The objective is to predict the final position of an object (the snitch) in a video. The scenes show multiple objects that move over time, one being the snitch object. The snitch can be occluded and moved around simultaneously by other objects, requiring object tracking and reasoning about dynamics to solve the task.

Once INFERNO is trained, we discard the rendering pipeline of our model, and instead feed the scene representations to a 12-layer transformer model to predict the final snitch position. Each object in our representation is given as an input element to the transformer. We add a learned positional encoding to the object representations based on their frame index. Objects in the same frame have the same positional encoding. The transformer last output is given to a MLP head that predicts the logits for the 36 possible output positions.

Following (Ding et al., 2020), we also consider the use of auxiliary SSL loss after pretraining. The SSL loss randomly masks one object per-frame and tries to predict its representation at the corresponding output. The model minimizes the L2 distance between the predicted object representation and the observed one.

We first compare our model that is pretrained to reconstruct image on CATER with a randomly initialized encoder. Using a randomly initialized encoder does not perform well. It indicates that INFERNO pretraining, and not the encoder architecture, is key to learn useful representation for the task.

We next compare our approach with Aloe (Ding et al., 2020), an object-centric baseline, R3D and R3D NL, 3D convolutional models proposed by (Girdhar and Ramanan, 2019) and architectures using strong inductive biases toward object tracking such as OPNet (Shamsian et al., 2020) or Hopper (Zhou et al., 2021). Our model outperforms the R3D and R3D NL models that rely on unstructured representation. This result suggests that the object-centric representation learned by INFERNO is useful for the visual reasoning task. Our model also outperforms Aloe, an object-centric method using 2D object representation, when both methods do not use additional SSL loss.

We observe that our model significantly outperforms the slot-attention model which focus on the 2D geometry of the scene. This result supports the advantage of 3D-aware representation for solving the CATER task.

Our model is able to discover objects in a scene without annotations, and the inferred scene representations are interpretable and amenable to manipulations. Further, the scene representation is useful for visual reasoning downstream tasks such as the snitch localization task in CATER.

For the visual reasoning task, we consider the CATER dataset and uses 5K videos that do not have camera motion.

We first pretrain our INFERNO to reconstruct individual frames from the CATER dataset. We bootstrap a model trained on CLEVR6 for object discovery for 400k iterations and train it on the CATER dataset for an additional 100k iterations to speed-up training, as the iteration time of a model with 10 objects is larger.

After pretraining, we finetune the INFERNO encoder to the supervised task of snitch localization. We discard the rendering pipeline of our model, and instead feed the inferred object slots representations to a transformer that aims at predicting the final position of the snitch. To predict the snitch, we consider a 12 layers transformer with the hidden dimension of 128 which takes the slot representation as input. The transformer treats each object as input element. A learned positional embedding is added each slots representation based on their frames index, i.e. the position of the objects is the same within a frame. The final output to the transformer is given to a 1 layer MLP head with an hidden dimension of 128. It outputs 36 logits that correspond to possible snitch location. We minimize the sum of the cross-entropy between the predicted position and the true target and a the l1 loss between the prediction and target.

To perform unsupervised object discovery we use the outputs of slot attention, since they do not require any upsampling and thus can provide more accurate segmentations.

Input: Figure 2: Manipulations on CLEVR: we show some examples of the manipulations we perform to CLEVR2345 images, including object removal and addition, changing the scale of an object, and object translation. Our model can perform these transformations because it disentangles object pose and appearance.

Input: Table 3: Object Discovery on CLEVR6 INFERNO, despite inferring more complex 3D object segmentations without annotations, is competitive with the current state-of-the-art 2D object discovery methods on CLEVR6.

Input: Figure 3: Object Discovery on CLEVR6: INFERNO identifies the different objects in a scene without supervision. For each input image, we show which regions of the input are attended by each object as well as the background. We include an example of a failed segmentation in the last row, where one object slot (4) is trying to represent multiple objects at the same time.

Input: Figure 9: Object occupancy maps on CLEVR2345. For each input scene we show the reconstructed image as well as each individual object occupancy map. Occupancy maps are obtained by integrating the density outputs of the NeRF function along rays going from the image plane to the scene. We observe that each object is rendering a part of the scene corresponding to a single object instance. Outputs of the NeRF function have low resolution and are upscaled by a neural network. The last row shows an example of a bad grouping of two objects in the scene.

Input: Figure 5: Additional object additions and removals on CLEVR2345. For each scene, we show images with one randomly added object, and with 1-3 random objects removed. Some of these images show out-of-distribution samples with a number of objects not seen during training (2-5 objects).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the statement to be somewhat overstated, though not severely. While the evidence shows the model performs competitively on object discovery (96.7% ARI vs 98.8% for state-of-the-art) and demonstrates reasonable performance on the CATER task, the claim that the representations are "useful for downstream tasks" is broad and only supported by limited experimental evidence on two specific tasks. The reviewers also point out valid concerns about the necessity of 3D representations and lack of comparisons with other object-centric models for downstream tasks.

<score>3</score>

Justification:
1. The claim is partially supported by:
- Competitive object discovery results (Table 3)
- Success on CATER snitch localization task
- Ability to perform meaningful object manipulations (Figures 2,5)

2. Key limitations:
- Only two downstream tasks evaluated
- No direct comparisons with 2D object-centric models on downstream tasks
- Reviewers note the tasks presented could be accomplished with 2D representations
- Limited testing on realistic/complex datasets

3. The phrasing "useful for downstream tasks" implies broader applicability than what's demonstrated in the evidence, making it somewhat overstated relative to the actual findings.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks.

The review comment to be evaluated is:

Strengths:

* an interesting combination of slot attention and NeRF mixture models.
* paper is nicely presented
* results are nice, though see below

Weaknesses:

While I like the paper in general I feel there are several points missing which are probably worth addressing to:

* The objects are transformed using the pose - but that means the appearance is transformed as well - this could be a problem if some of the appearance is pose dependent, for example shading will change, or reflections. Can the authors comment how this is handled here? my guess is that the post rendering network (the upscaling one) takes some of this burden but that's probably not ideal in terms of representation.

* How does the model perform when scenes are rendered from other viewpoints? if the resulting representation is indeed 3D it should be no problem to render the scene from other viewpoints - is that the case here? if not, why? (again, my guess is the upscaling network may be problematic here as well)

* I see that the masks used for estimating segmentation performance (Figure 3) are from the slot attention mechanism - what happens if you use the masks as decoded from the slots (i.e. render the densitiy of each slot separately). This would be more convincing to demonstrate these are "3D" segmentation.

* I feel the experimental validation is quite limited - using only a single dataset (even if it's two variants) is not very convincing as to the generality of the method. How does it work on other datasets? more/less complex.

* I would be happy to see more discussion of the post rendering / upscaling network - this seems like a cruicial part and is not discussed enough.

The evidence to be evaluated is:Given the rendering pipeline, the goal of our method is to infer representations that reconstruct a given scene. Our inference mechanism computes image features through a neural network encoder and then extracts \(K\) object and a background slots. These slots are then mapped to our structured scene representation through learned neural networks. To extract image features for each object and background slots, we use Slot Attention (Locatello et al., 2020).

When equipped with an inference mechanism, these models learn to segment objects in a scene without annotations, driven by their compositional generative process.

Slot-Attention (Locatello et al., 2020) maps a set of entities, called slots, to image features through multiple rounds of soft attention. The slots compete among themselves to attend to features, making each slot attend to a region of the input image. When driven with alpha-compositing decoder, slot attention learns to segment objects in a scene.

Contrary to most previous approaches which segment 2D shapes, our method infers object-centric scene representations in 3D space.

Then we show that it learns to identify and segment the objects in a scene without supervision. Finally, we highlight the usefulness of such representations for downstream tasks by applying our model on the CATER snitch localization task.

We also investigate different interpretable manipulations of scene representations and visualize the effects in the corresponding output renderings.

We validate that our model is able to render out-of-distribution scenes not corresponding to training examples, such as scenes having 1 or 6 objects, and verify that the pose manipulations have semantically coherent effects.

Additionally, our model can perform inference and manipulations on existing scenes. We use that capability to generate novel scenes by manipulating existing ones. Our model is able to generate novel scenes with additional objects or with altered object shapes and appearances, with better FID than GIRAFFE. In Figure 2 we show some examples of the scene manipulations possible with our model. Given a scene representation, we can remove or add objects, rearrange object poses, translate the objects to new locations or change the object scales. While some of these manipulations can be performed with regular object-centric models, modifications to the scale and location of the objects are hard to implement without explicitly modeling 3D object pose. Unsupervised object discovery consists in learning to segment the objects in a scene without using annotations.

Table 3 reports the ARI metric of our model and different baselines in this task. We observe that INFERNO is competitive with state-of-the-art methods, surpassing MONet and having slightly lower ARI than Slot-Attention and IDDINE. However, our model learns to segment objects in 3D, while the other baselines extract 2D segmentation masks. Figure 3 shows some examples of the discovered object masks. We can see that the model learns to segment different objects and properly discards object slots when the number of objects in the scene is lower than needed.

The goal of this experiment is to show that the representation learned by our model is useful for the snitch localization task from the CATER dataset (Girdhar and Ramanan, 2019). We focus on the CATER task involving videos with a static camera. The objective is to predict the final position of an object (the snitch) in a video. The scenes show multiple objects that move over time, one being the snitch object. The snitch can be occluded and moved around simultaneously by other objects, requiring object tracking and reasoning about dynamics to solve the task.

Once INFERNO is trained, we discard the rendering pipeline of our model, and instead feed the scene representations to a 12-layer transformer model to predict the final snitch position. Each object in our representation is given as an input element to the transformer. We add a learned positional encoding to the object representations based on their frame index. Objects in the same frame have the same positional encoding. The transformer last output is given to a MLP head that predicts the logits for the 36 possible output positions.

Following (Ding et al., 2020), we also consider the use of auxiliary SSL loss after pretraining. The SSL loss randomly masks one object per-frame and tries to predict its representation at the corresponding output. The model minimizes the L2 distance between the predicted object representation and the observed one.

We first compare our model that is pretrained to reconstruct image on CATER with a randomly initialized encoder. Using a randomly initialized encoder does not perform well. It indicates that INFERNO pretraining, and not the encoder architecture, is key to learn useful representation for the task.

We next compare our approach with Aloe (Ding et al., 2020), an object-centric baseline, R3D and R3D NL, 3D convolutional models proposed by (Girdhar and Ramanan, 2019) and architectures using strong inductive biases toward object tracking such as OPNet (Shamsian et al., 2020) or Hopper (Zhou et al., 2021). Our model outperforms the R3D and R3D NL models that rely on unstructured representation. This result suggests that the object-centric representation learned by INFERNO is useful for the visual reasoning task. Our model also outperforms Aloe, an object-centric method using 2D object representation, when both methods do not use additional SSL loss.

We observe that our model significantly outperforms the slot-attention model which focus on the 2D geometry of the scene. This result supports the advantage of 3D-aware representation for solving the CATER task.

Our model is able to discover objects in a scene without annotations, and the inferred scene representations are interpretable and amenable to manipulations. Further, the scene representation is useful for visual reasoning downstream tasks such as the snitch localization task in CATER.

For the visual reasoning task, we consider the CATER dataset and uses 5K videos that do not have camera motion.

We first pretrain our INFERNO to reconstruct individual frames from the CATER dataset. We bootstrap a model trained on CLEVR6 for object discovery for 400k iterations and train it on the CATER dataset for an additional 100k iterations to speed-up training, as the iteration time of a model with 10 objects is larger.

After pretraining, we finetune the INFERNO encoder to the supervised task of snitch localization. We discard the rendering pipeline of our model, and instead feed the inferred object slots representations to a transformer that aims at predicting the final position of the snitch. To predict the snitch, we consider a 12 layers transformer with the hidden dimension of 128 which takes the slot representation as input. The transformer treats each object as input element. A learned positional embedding is added each slots representation based on their frames index, i.e. the position of the objects is the same within a frame. The final output to the transformer is given to a 1 layer MLP head with an hidden dimension of 128. It outputs 36 logits that correspond to possible snitch location. We minimize the sum of the cross-entropy between the predicted position and the true target and a the l1 loss between the prediction and target.

To perform unsupervised object discovery we use the outputs of slot attention, since they do not require any upsampling and thus can provide more accurate segmentations.

Input: Figure 2: Manipulations on CLEVR: we show some examples of the manipulations we perform to CLEVR2345 images, including object removal and addition, changing the scale of an object, and object translation. Our model can perform these transformations because it disentangles object pose and appearance.

Input: Table 3: Object Discovery on CLEVR6 INFERNO, despite inferring more complex 3D object segmentations without annotations, is competitive with the current state-of-the-art 2D object discovery methods on CLEVR6.

Input: Figure 3: Object Discovery on CLEVR6: INFERNO identifies the different objects in a scene without supervision. For each input image, we show which regions of the input are attended by each object as well as the background. We include an example of a failed segmentation in the last row, where one object slot (4) is trying to represent multiple objects at the same time.

Input: Figure 9: Object occupancy maps on CLEVR2345. For each input scene we show the reconstructed image as well as each individual object occupancy map. Occupancy maps are obtained by integrating the density outputs of the NeRF function along rays going from the image plane to the scene. We observe that each object is rendering a part of the scene corresponding to a single object instance. Outputs of the NeRF function have low resolution and are upscaled by a neural network. The last row shows an example of a bad grouping of two objects in the scene.

Input: Figure 5: Additional object additions and removals on CLEVR2345. For each scene, we show images with one randomly added object, and with 1-3 random objects removed. Some of these images show out-of-distribution samples with a number of objects not seen during training (2-5 objects).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and claim, I find the statement "We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks" to be moderately overstated.

While there is evidence showing the model performs well on object discovery (ARI score of 96.7%) and the CATER snitch localization task, several limitations weaken the claim:

1. Limited dataset validation (only CLEVR variants and CATER)
2. Lack of comprehensive performance comparisons across diverse downstream tasks
3. Reviewer points out limited experimental validation on "only a single dataset"
4. Some technical limitations noted in reviews regarding pose-dependent appearance and viewpoint rendering

The evidence shows promise but doesn't fully support broad claims about general downstream task utility.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks.

The review comment to be evaluated is:

The paper tackles a very challenging problem, is well-written and easy to follow, and has a good set of experiments for testing object-centric representations. However, my main concern is that the experiments did not focus on testing the '3D-ness' of the inferred representations.
- For scene segmentation, reconstruction, and manipulation, each scene is tested only on a single camera viewpoint. It is unclear whether the learned representations can capture the object appearance from viewpoints different from the input view. Previous work [1, 2] demonstrates this through novel view synthesis.
- The results on the CATER snitch localization task did not strongly support that the '3D' representations learned by the proposed model are more beneficial than 2D representations. One reason might be that this task relies more on long-term reasoning than on 3D reasoning. The authors may want to consider some other tasks that can clearly show the benefit of having 3D representations, like the ones used in [2]. Also, instead of comparing to Aloe, a more direct comparison would be to compare with Slot Attention combined with the same 12-layer transformer used for the proposed model.

**MINOR COMMENTS**
- The rendering function is denoted $g_\gamma$ in Equation 1 but $g_\tau$ in Section 2.1.
- From the paragraph directly below Figure 1, it seems the model can infer the camera location $c$. However, this was not demonstrated in any of the experiments.
- The object NeRFs are shared, which may only be able to deal with similar objects. The paper could be strengthened by experiments on more complex scenes, like the ones used in [1].
- What is the reference frame for the pose vectors? It is mentioned in Appendix B.1 that the manipulations are done by moving the objects forward on the Z or X axes. How are the XYZ axes defined? I would imagine that under different camera viewpoints, the same manipulation would have different effects, but the results in Figure 2 seem quite consistent.

[1] [Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentation](https://arxiv.org/abs/2104.01148).

[2] [ROOTS: Object-Centric Representation and Rendering of 3D Scenes](https://arxiv.org/abs/2006.06130).

The evidence to be evaluated is:Given the rendering pipeline, the goal of our method is to infer representations that reconstruct a given scene. Our inference mechanism computes image features through a neural network encoder and then extracts \(K\) object and a background slots. These slots are then mapped to our structured scene representation through learned neural networks. To extract image features for each object and background slots, we use Slot Attention (Locatello et al., 2020).

When equipped with an inference mechanism, these models learn to segment objects in a scene without annotations, driven by their compositional generative process.

Slot-Attention (Locatello et al., 2020) maps a set of entities, called slots, to image features through multiple rounds of soft attention. The slots compete among themselves to attend to features, making each slot attend to a region of the input image. When driven with alpha-compositing decoder, slot attention learns to segment objects in a scene.

Contrary to most previous approaches which segment 2D shapes, our method infers object-centric scene representations in 3D space.

Then we show that it learns to identify and segment the objects in a scene without supervision. Finally, we highlight the usefulness of such representations for downstream tasks by applying our model on the CATER snitch localization task.

We also investigate different interpretable manipulations of scene representations and visualize the effects in the corresponding output renderings.

We validate that our model is able to render out-of-distribution scenes not corresponding to training examples, such as scenes having 1 or 6 objects, and verify that the pose manipulations have semantically coherent effects.

Additionally, our model can perform inference and manipulations on existing scenes. We use that capability to generate novel scenes by manipulating existing ones. Our model is able to generate novel scenes with additional objects or with altered object shapes and appearances, with better FID than GIRAFFE. In Figure 2 we show some examples of the scene manipulations possible with our model. Given a scene representation, we can remove or add objects, rearrange object poses, translate the objects to new locations or change the object scales. While some of these manipulations can be performed with regular object-centric models, modifications to the scale and location of the objects are hard to implement without explicitly modeling 3D object pose. Unsupervised object discovery consists in learning to segment the objects in a scene without using annotations.

Table 3 reports the ARI metric of our model and different baselines in this task. We observe that INFERNO is competitive with state-of-the-art methods, surpassing MONet and having slightly lower ARI than Slot-Attention and IDDINE. However, our model learns to segment objects in 3D, while the other baselines extract 2D segmentation masks. Figure 3 shows some examples of the discovered object masks. We can see that the model learns to segment different objects and properly discards object slots when the number of objects in the scene is lower than needed.

The goal of this experiment is to show that the representation learned by our model is useful for the snitch localization task from the CATER dataset (Girdhar and Ramanan, 2019). We focus on the CATER task involving videos with a static camera. The objective is to predict the final position of an object (the snitch) in a video. The scenes show multiple objects that move over time, one being the snitch object. The snitch can be occluded and moved around simultaneously by other objects, requiring object tracking and reasoning about dynamics to solve the task.

Once INFERNO is trained, we discard the rendering pipeline of our model, and instead feed the scene representations to a 12-layer transformer model to predict the final snitch position. Each object in our representation is given as an input element to the transformer. We add a learned positional encoding to the object representations based on their frame index. Objects in the same frame have the same positional encoding. The transformer last output is given to a MLP head that predicts the logits for the 36 possible output positions.

Following (Ding et al., 2020), we also consider the use of auxiliary SSL loss after pretraining. The SSL loss randomly masks one object per-frame and tries to predict its representation at the corresponding output. The model minimizes the L2 distance between the predicted object representation and the observed one.

We first compare our model that is pretrained to reconstruct image on CATER with a randomly initialized encoder. Using a randomly initialized encoder does not perform well. It indicates that INFERNO pretraining, and not the encoder architecture, is key to learn useful representation for the task.

We next compare our approach with Aloe (Ding et al., 2020), an object-centric baseline, R3D and R3D NL, 3D convolutional models proposed by (Girdhar and Ramanan, 2019) and architectures using strong inductive biases toward object tracking such as OPNet (Shamsian et al., 2020) or Hopper (Zhou et al., 2021). Our model outperforms the R3D and R3D NL models that rely on unstructured representation. This result suggests that the object-centric representation learned by INFERNO is useful for the visual reasoning task. Our model also outperforms Aloe, an object-centric method using 2D object representation, when both methods do not use additional SSL loss.

We observe that our model significantly outperforms the slot-attention model which focus on the 2D geometry of the scene. This result supports the advantage of 3D-aware representation for solving the CATER task.

Our model is able to discover objects in a scene without annotations, and the inferred scene representations are interpretable and amenable to manipulations. Further, the scene representation is useful for visual reasoning downstream tasks such as the snitch localization task in CATER.

For the visual reasoning task, we consider the CATER dataset and uses 5K videos that do not have camera motion.

We first pretrain our INFERNO to reconstruct individual frames from the CATER dataset. We bootstrap a model trained on CLEVR6 for object discovery for 400k iterations and train it on the CATER dataset for an additional 100k iterations to speed-up training, as the iteration time of a model with 10 objects is larger.

After pretraining, we finetune the INFERNO encoder to the supervised task of snitch localization. We discard the rendering pipeline of our model, and instead feed the inferred object slots representations to a transformer that aims at predicting the final position of the snitch. To predict the snitch, we consider a 12 layers transformer with the hidden dimension of 128 which takes the slot representation as input. The transformer treats each object as input element. A learned positional embedding is added each slots representation based on their frames index, i.e. the position of the objects is the same within a frame. The final output to the transformer is given to a 1 layer MLP head with an hidden dimension of 128. It outputs 36 logits that correspond to possible snitch location. We minimize the sum of the cross-entropy between the predicted position and the true target and a the l1 loss between the prediction and target.

To perform unsupervised object discovery we use the outputs of slot attention, since they do not require any upsampling and thus can provide more accurate segmentations.

Input: Figure 2: Manipulations on CLEVR: we show some examples of the manipulations we perform to CLEVR2345 images, including object removal and addition, changing the scale of an object, and object translation. Our model can perform these transformations because it disentangles object pose and appearance.

Input: Table 3: Object Discovery on CLEVR6 INFERNO, despite inferring more complex 3D object segmentations without annotations, is competitive with the current state-of-the-art 2D object discovery methods on CLEVR6.

Input: Figure 3: Object Discovery on CLEVR6: INFERNO identifies the different objects in a scene without supervision. For each input image, we show which regions of the input are attended by each object as well as the background. We include an example of a failed segmentation in the last row, where one object slot (4) is trying to represent multiple objects at the same time.

Input: Figure 9: Object occupancy maps on CLEVR2345. For each input scene we show the reconstructed image as well as each individual object occupancy map. Occupancy maps are obtained by integrating the density outputs of the NeRF function along rays going from the image plane to the scene. We observe that each object is rendering a part of the scene corresponding to a single object instance. Outputs of the NeRF function have low resolution and are upscaled by a neural network. The last row shows an example of a bad grouping of two objects in the scene.

Input: Figure 5: Additional object additions and removals on CLEVR2345. For each scene, we show images with one randomly added object, and with 1-3 random objects removed. Some of these images show out-of-distribution samples with a number of objects not seen during training (2-5 objects).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence provided, I'll evaluate the claim "We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks."

The evidence shows:
1. Competitive but not superior performance on object discovery (ARI of 96.7% vs 98.8% for SOTA)
2. Mixed results on CATER snitch localization task - outperforms some baselines but the reviewer points out that the task may not strongly demonstrate 3D representation benefits
3. Limited testing of 3D capabilities as experiments focused on single viewpoints
4. The reviewer notes that alternative tasks could better demonstrate 3D representation advantages

While the model shows some utility for downstream tasks, the claim somewhat overstates the strength and breadth of evidence demonstrating the usefulness of the representations. A more measured statement would acknowledge the mixed results and limitations.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks.

The review comment to be evaluated is:

Strengths:

+ The paper is clearly written and easy to follow.
+ It is novel and interesting to see object-centric learning models to infer 3D poses.
+ Validating the usefulness in downstream visual reasoning task is a plus.

Weakness:

My main conern is in the insufficient experiments that do not support the main contributions, i.e., the experiments do not demonstrate the benefits of the proposed 3D, structured object representations.

As for representations being structured (i.e., disentangled shape, appearance and poses):
- For disentanglement of shape and appearance, there is no experiments demonstrating this point. For example, this can be easily shown by swapping appearances of two objects in the scene while keeping their shapes unchanged, and vice versa.
- For disentanglement of 3D pose, the experiments include scaling and translation. However, there is no comparison to 2D baseline methods, which are able to do scaling and translation (at least the experiments on moving to the "right").

As for representations being 3D, there is no comparisons to show the benefits of 3D representation over 2D. In segmentation experiments, the results are worse than 2D baselines. In the visual reasoning task, the performance is worse than Aloe. Perhaps Aloe is better because it is designed for that task, but there is neither comparisons to 2D general object-centric baselines (such as original slot attention) or unstructured baselines.

Other minor weaknesses include:
- No ablation study on the model components to show the benefits of proposed technical contributions, such as the effect of regressing pose vs. not regressing pose (I suppose this should affect background decomposition because that is the only inductive bias that separates background slot from foreground slots).
- It would be good to demonstrate on datasets other than CLEVR.

The evidence to be evaluated is:Given the rendering pipeline, the goal of our method is to infer representations that reconstruct a given scene. Our inference mechanism computes image features through a neural network encoder and then extracts \(K\) object and a background slots. These slots are then mapped to our structured scene representation through learned neural networks. To extract image features for each object and background slots, we use Slot Attention (Locatello et al., 2020).

When equipped with an inference mechanism, these models learn to segment objects in a scene without annotations, driven by their compositional generative process.

Slot-Attention (Locatello et al., 2020) maps a set of entities, called slots, to image features through multiple rounds of soft attention. The slots compete among themselves to attend to features, making each slot attend to a region of the input image. When driven with alpha-compositing decoder, slot attention learns to segment objects in a scene.

Contrary to most previous approaches which segment 2D shapes, our method infers object-centric scene representations in 3D space.

Then we show that it learns to identify and segment the objects in a scene without supervision. Finally, we highlight the usefulness of such representations for downstream tasks by applying our model on the CATER snitch localization task.

We also investigate different interpretable manipulations of scene representations and visualize the effects in the corresponding output renderings.

We validate that our model is able to render out-of-distribution scenes not corresponding to training examples, such as scenes having 1 or 6 objects, and verify that the pose manipulations have semantically coherent effects.

Additionally, our model can perform inference and manipulations on existing scenes. We use that capability to generate novel scenes by manipulating existing ones. Our model is able to generate novel scenes with additional objects or with altered object shapes and appearances, with better FID than GIRAFFE. In Figure 2 we show some examples of the scene manipulations possible with our model. Given a scene representation, we can remove or add objects, rearrange object poses, translate the objects to new locations or change the object scales. While some of these manipulations can be performed with regular object-centric models, modifications to the scale and location of the objects are hard to implement without explicitly modeling 3D object pose. Unsupervised object discovery consists in learning to segment the objects in a scene without using annotations.

Table 3 reports the ARI metric of our model and different baselines in this task. We observe that INFERNO is competitive with state-of-the-art methods, surpassing MONet and having slightly lower ARI than Slot-Attention and IDDINE. However, our model learns to segment objects in 3D, while the other baselines extract 2D segmentation masks. Figure 3 shows some examples of the discovered object masks. We can see that the model learns to segment different objects and properly discards object slots when the number of objects in the scene is lower than needed.

The goal of this experiment is to show that the representation learned by our model is useful for the snitch localization task from the CATER dataset (Girdhar and Ramanan, 2019). We focus on the CATER task involving videos with a static camera. The objective is to predict the final position of an object (the snitch) in a video. The scenes show multiple objects that move over time, one being the snitch object. The snitch can be occluded and moved around simultaneously by other objects, requiring object tracking and reasoning about dynamics to solve the task.

Once INFERNO is trained, we discard the rendering pipeline of our model, and instead feed the scene representations to a 12-layer transformer model to predict the final snitch position. Each object in our representation is given as an input element to the transformer. We add a learned positional encoding to the object representations based on their frame index. Objects in the same frame have the same positional encoding. The transformer last output is given to a MLP head that predicts the logits for the 36 possible output positions.

Following (Ding et al., 2020), we also consider the use of auxiliary SSL loss after pretraining. The SSL loss randomly masks one object per-frame and tries to predict its representation at the corresponding output. The model minimizes the L2 distance between the predicted object representation and the observed one.

We first compare our model that is pretrained to reconstruct image on CATER with a randomly initialized encoder. Using a randomly initialized encoder does not perform well. It indicates that INFERNO pretraining, and not the encoder architecture, is key to learn useful representation for the task.

We next compare our approach with Aloe (Ding et al., 2020), an object-centric baseline, R3D and R3D NL, 3D convolutional models proposed by (Girdhar and Ramanan, 2019) and architectures using strong inductive biases toward object tracking such as OPNet (Shamsian et al., 2020) or Hopper (Zhou et al., 2021). Our model outperforms the R3D and R3D NL models that rely on unstructured representation. This result suggests that the object-centric representation learned by INFERNO is useful for the visual reasoning task. Our model also outperforms Aloe, an object-centric method using 2D object representation, when both methods do not use additional SSL loss.

We observe that our model significantly outperforms the slot-attention model which focus on the 2D geometry of the scene. This result supports the advantage of 3D-aware representation for solving the CATER task.

Our model is able to discover objects in a scene without annotations, and the inferred scene representations are interpretable and amenable to manipulations. Further, the scene representation is useful for visual reasoning downstream tasks such as the snitch localization task in CATER.

For the visual reasoning task, we consider the CATER dataset and uses 5K videos that do not have camera motion.

We first pretrain our INFERNO to reconstruct individual frames from the CATER dataset. We bootstrap a model trained on CLEVR6 for object discovery for 400k iterations and train it on the CATER dataset for an additional 100k iterations to speed-up training, as the iteration time of a model with 10 objects is larger.

After pretraining, we finetune the INFERNO encoder to the supervised task of snitch localization. We discard the rendering pipeline of our model, and instead feed the inferred object slots representations to a transformer that aims at predicting the final position of the snitch. To predict the snitch, we consider a 12 layers transformer with the hidden dimension of 128 which takes the slot representation as input. The transformer treats each object as input element. A learned positional embedding is added each slots representation based on their frames index, i.e. the position of the objects is the same within a frame. The final output to the transformer is given to a 1 layer MLP head with an hidden dimension of 128. It outputs 36 logits that correspond to possible snitch location. We minimize the sum of the cross-entropy between the predicted position and the true target and a the l1 loss between the prediction and target.

To perform unsupervised object discovery we use the outputs of slot attention, since they do not require any upsampling and thus can provide more accurate segmentations.

Input: Figure 2: Manipulations on CLEVR: we show some examples of the manipulations we perform to CLEVR2345 images, including object removal and addition, changing the scale of an object, and object translation. Our model can perform these transformations because it disentangles object pose and appearance.

Input: Table 3: Object Discovery on CLEVR6 INFERNO, despite inferring more complex 3D object segmentations without annotations, is competitive with the current state-of-the-art 2D object discovery methods on CLEVR6.

Input: Figure 3: Object Discovery on CLEVR6: INFERNO identifies the different objects in a scene without supervision. For each input image, we show which regions of the input are attended by each object as well as the background. We include an example of a failed segmentation in the last row, where one object slot (4) is trying to represent multiple objects at the same time.

Input: Figure 9: Object occupancy maps on CLEVR2345. For each input scene we show the reconstructed image as well as each individual object occupancy map. Occupancy maps are obtained by integrating the density outputs of the NeRF function along rays going from the image plane to the scene. We observe that each object is rendering a part of the scene corresponding to a single object instance. Outputs of the NeRF function have low resolution and are upscaled by a neural network. The last row shows an example of a bad grouping of two objects in the scene.

Input: Figure 5: Additional object additions and removals on CLEVR2345. For each scene, we show images with one randomly added object, and with 1-3 random objects removed. Some of these images show out-of-distribution samples with a number of objects not seen during training (2-5 objects).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review, the claim "We demonstrate that the inferred representations are useful for downstream tasks by showing competitive performance in object discovery and reasoning tasks" is somewhat overstated (3.5-4). While there is evidence showing competitive performance in object discovery (96.7% ARI) and some success in the CATER snitch localization task, several limitations noted in the reviewer comments are valid:

1. Lack of comparisons showing benefits of 3D over 2D representations
2. Performance is worse than some baselines (e.g., Slot-Attention, IODINE in segmentation)
3. Limited testing only on CLEVR/CATER datasets
4. Insufficient experiments demonstrating structured representation benefits

The evidence shows usefulness but doesn't fully support claiming broad competitive performance across downstream tasks without acknowledging these important limitations.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Each object representation defines a localized neural radiance field that is used to generate 2D views of the scene through a differentiable rendering process.

The evidence to be evaluated is:Each object is composed of three tensors \(o_{i}=(o_{i}^{shape},o_{i}^{app},o_{i}^{pose})\). The object shape \(o_{i}^{shape}\in\mathbb{R}^{D_{shape}}\) and object appearance \(o^{app}\in\mathbb{R}^{D_{shape}}\) are tensors that respectively describe the 3D shape occupancy and color of an object with an implicit function. The object location \(o_{i}^{pose}\in\mathbb{R}^{4\times 4}\) is an affine matrix that describes the object 3D pose (i.e. scale, translation and rotation) in the scene.

We define a rendering function \(g_{\gamma}\) that takes as input a scene representation and generates a 2D view of that scene \(\hat{x}=g_{\gamma}(f_{\theta}(x))\).

\[\gamma^{*},\theta^{*}=\operatorname*{arg\,min}_{\gamma,\theta}p(X|\gamma,\theta) =\operatorname*{arg\,min}_{\gamma,\theta}\frac{1}{N}\sum_{i\leq N}(x_{i}-g_{ \gamma}(f_{\theta}(x_{i})))^{2} \tag{1}\]

We represent 3D objects as Neural Radiance Fields (NeRFs) (Mildenhall et al., 2020) with a similar setup as that of GIRAFFE (Niemeyer and Geiger, 2021). A NeRF is a function \(g_{\tau}\) that defines a 3D shape implicitly. It takes as input a 3D location \(\mathbf{l}=(x,y,z)\) and a 2D viewing direction \(\mathbf{d}=(\psi,\phi)\) and outputs an occupancy value \(\sigma\) and a color value \(\mathbf{a}=(r,g,b)\).

\[g_{\tau}^{\prime}:\mathbb{R}^{P_{l}}\times\mathbb{R}^{P_{d}}\times\mathbb{R}^ {D_{shape}}\times\mathbb{R}^{D_{app}}\rightarrow\mathbb{R}^{+}\times\mathbb{R }^{3}\quad;\quad(\gamma(\mathbf{l}),\gamma(\mathbf{d}),\mu,\upsilon) \rightarrow(\sigma,\mathbf{a}) \tag{3}\] In INFERNO, we share a single parametrization of a NeRF function across all objects. Each object specific shape and appearance is defined by the shape and appearance latent variables, which correspond to the object attributes \(o^{shape},o^{app}\).

To render a scene, we cast rays from each pixel in the 2D plane defined by a given camera to the 3D scene. We evaluate NeRFs at different points along a given ray, and integrate their occupancy and color outputs to determine pixel values. Rays might traverse multiple object NeRFs in addition to the background NeRF. To determine the occupancy and color of points described by multiple NeRFs, we first query each NeRF at those particular points. To query the object NeRFs, we first need to transform the points from scene space to the particular object space.

Low resolution scenes are then upscaled to the desired output resolution using a convolutional neural network, keeping the entire rendering pipeline differentiable.

Given the rendering pipeline, the goal of our method is to infer representations that reconstruct a given scene. Our inference mechanism computes image features through a neural network encoder and then extracts \(K\) object and a background slots. These slots are then mapped to our structured scene representation through learned neural networks.

NeRFs (Mildenhall et al., 2020) generate scenes by learning a function that outputs the occupancy and color of points in a scene when viewed from a particular direction. By casting rays through a plane and aggregating the output values NeRFs can generate 2D views of an implicitly modeled 3D scenes.

GIRAFFE (Niemeyer and Geiger, 2021) is a GAN-based method that represents multiple objects in a scene using NeRFs as part of their generator.

Our model uses a rendering pipeline inspired by GIRAFFE.

ObjSURF (Stelzner et al., 2021) and UORF (Yu et al., 2021) infer scene representations composed of multiple objects, each object represented with a differently instantiated NeRF.

Our model uses a variant of Slot Attention to decide on which part of a 2D view should each object in our scene attend to, but we infer 3D-aware representations for each object.

Contrary to most previous approaches which segment 2D shapes, our method infers object-centric scene representations in 3D space.

Our model infers 3D object-centric scene representations from single 2D views. These representations disentangle the 3D appearance and pose of objects, which allows for semantic manipulations of the scene not possible otherwise. These manipulations can be validated by rendering the modified scene representations.

In INFERNO, we generate novel scenes by inferring representations for ground-truth images and then manipulating them.

We also investigate different interpretable manipulations of scene representations and visualize the effects in the corresponding output renderings.

We validate that our model is able to render out-of-distribution scenes not corresponding to training examples, such as scenes having 1 or 6 objects, and verify that the pose manipulations have semantically coherent effects.

Our model is able to discover objects in a scene without annotations, and the inferred scene representations are interpretable and amenable to manipulations.

Our model is composed of five main modules: encoder, slot attention, slot to object mapping, NeRF decoder, neural network upscaler.

Once we have inferred the object poses, we infer object shapes and appearances. These are inferred individually for each object using a common 2-layer MLP. The MLPs are conditional on the object pose using Conditional Layer Normalization, that makes the learned parameters of LayerNorm be a function of a condition. Specifically, we map the 7-dimensional pose tensor to LayerNorm parameters with a single linear layer with no activation or normalization. Shapes and appearances are defined by the output of the MLPs, which produce two 128-dimensional tensors. For the background object we only infer shape and appearance, and define its pose to be that of the scene cube. The shape and appearance of the background are inferred through another 2-layer MLP with ReLU and LayerNorm.

With the object shape and appearance tensors we can render them following GRAF (Schwarz et al., 2020). We use one NeRF for the objects and one NeRF MLP for the background.

Note that, to render objects according to their pose, we query their NeRF MLP in a canonical object space by transforming input coordinates in scene space to object space using the object pose. To reduce the computational complexity of rendering with many NeRFs, we render scenes at a fixed resolution of 16x16. Instead of rendering RGB pixels, we render feature images with 128 channels. The output of the rendering is then upscaled and mapped to RGB views with a neural network upscaler.

The neural upscaler takes the low resolution output of the NeRF MLPs and upscales it to the full output resolution. Additionally, it maps the rendered image to RGB space. This module is implemented using a convolutional neural network.

In this section we synthesize novel views of a scene by modifying the camera pose. For each example we show the input image, our reconstruction, then two images as a result of moving the camera \(\pm 15^{\circ}\) in the azimuth axis, and two images as a result of zooming in the scene. While our model is trained without ground-truth camera poses and with single views of scenes, it is able to generalize to small camera pose modifications and render novel views of a scene In this section we show the raw outputs of the NeRF function. These outputs show scene views rendered at low resolution, which are then upscaled with a neural network. While these generations have reduced details due to their resolution, they clearly show the different objects and their location in the scene. NeRFs are rendered at low resolution to ease the computational costs, as the time and memory requirements of rendering with a NeRF are linearly correlated with the number of casted rays/pixels in the output image.

The occupancy maps are obtained by integrating rays going from the image plane to the scene according to the NeRF density, but without taking into account the RGB output. We observe that each object is rendering a part of the scene corresponding to a single object instance.

Input: Figure 9: Object occupancy maps on CLEVR2345. For each input scene we show the reconstructed image as well as each individual object occupancy map. Occupancy maps are obtained by integrating the density outputs of the NeRF function along rays going from the image plane to the scene. We observe that each object is rendering a part of the scene corresponding to a single object instance. Outputs of the NeRF function have low resolution and are upscaled by a neural network. The last row shows an example of a bad grouping of two objects in the scene.

Input: Figure 1: Model Overview: We propose INFERNO, a model that infers and renders object-centric 3D scene representations. 1 Our model first decomposes an input observation into multiple object slots. 2 For each slot we infer a structured 3D representation. 3 The shape and appearance determine canonical objects rendered through NeRFs. 4 Objects are transformed and located in the overall scene according to their pose. 5 We combine objects and background and render a low-resolution scene given a camera location. 6 The input is reconstructed by upscaling the low resolution scene.

Input: Figure 6: Novel view synthesis on CLEVR2345. For each scene, we move the camera ±15◦ on the azimuth axis. Additionally, we zoom in the scene twice. While our model is trained with a fixed default camera pose and single scene views, it is able to generalize to small camera pose modifications and render novel views of a scene.

Input: Figure 7: NeRF outputs on CLEVR2345. For each input scene we show the reconstructed image as well as each the low resolution output of the NeRF function. This output is then upscaled with a neural network to obtain the reconstructed scene. While NeRF output lack full detail, they correctly depict each individual object and their position in the scene.

Input: Figure 4: Additional reconstructions on CLEVR2345.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "Each object representation defines a localized neural radiance field that is used to generate 2D views of the scene through a differentiable rendering process."

Based on the evidence, this claim is well-stated and accurately reflects how the model works. The evidence shows that:
- Each object has shape, appearance, and pose tensors
- Objects are represented as Neural Radiance Fields (NeRFs)
- The rendering process integrates rays through the NeRFs to generate 2D views
- The evidence includes detailed equations and diagrams showing how objects are rendered through NeRFs
- Multiple figures demonstrate the localized nature of object NeRFs through occupancy maps

The claim avoids exaggeration and precisely describes the technical approach used, making it well-aligned with the evidence.

<score>1</score>