Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We design a novel recombination model able to recombine the sampled endpoints to obtain scene-consistent trajectories across the agents.

The review comment to be evaluated is:

### Strong Points

The idea of performing re-estimation based on the computed heat map end-points
seems novel to me. This idea seems like a natural next step to improve multi agent motion
prediction. The fact that you first get the end-point and then reason about the scene is
intuitively sounding and seems to be a good solution for the problem.

The experiments on interact are a nice direction to be taken by the field.

The visualizations of the heat-maps and the different predictions was helpful.

### Weak Points

#### I could not find a clear distinction with literature
While reading the related work section I found hard to contrast the ideas of the paper with the ones proposed on the literature.
The difference between this method and GOHOME was  not clear to me in this sense.
From what I understood the main contribution here  is the modality combination based on the collision
free end-point sampling. However, this was not obvious for me when reading the paper.

#### The implemented baselines seemed to have performed way worse than expected.
For example, both [1] and [7] are methods designed for *joint* prediction and scene level prediction. This is contradicted
by the observations made by the authors on page 8, quote: " ... This problem is aggravated in the joint training case, since the modality selected is
the same for all agents in a training sample....". Both [1] and [3] fairly confidently claim producing multimodal joint predictions.
Either the version implemented by the authors does not correctly match the other works in the literature or there are specific findings
on the limitations existent in other methods.
Also, the paper lack comparison with latent variable based [2][5][[6] multi modal prediction methods. They seem to perform
better when the issue is diverse prediction. Definitely the mode does not tend to be the same for all predictions in those cases.

#### A single dataset evalution might not be enough for the community to access the value of this paper.

 Motion prediction is currently a very competitive scene. Even though I appreciate the idea on focusing reporting results on multi-agents
and joint prediction tasks, it is important to also evaluate the prediction on more standard benchmarks such as argoverse.
With results on argoverse, me and the rest of the community can more quickly access the the capabilities of the proposed
 heat map prediction pipeline, on producing consistent ego-agent motion prediction. I understand that this would not be the main
results, but it would be helpful.

##### Computational cost is apparently not shown.

I could not find the computational cost of running and training the proposed method on the paper. It seems that even
with the optimizations made on the heat map estimation the training time and inference time should be high.
Those kinds of motion prediction models have the necessity to further scale with way more data. Thus, highly
complex methods can be hard to train. There is also the case of potential future execution in an embedded hardware.

### Specific Questions

I am not  sure about the scene modality vectors ( Section 3.2.2), are they like seed parameters which are going to be used
to incorporate the joint consistent scene level ? I couldn't really find any sort of detailed explanation on that.

Why the lack of ADE for scene level metrics or using similar metrics as [3] ?

[1] Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zhengdong Zhang, Hao-Tien Lewis Chiang,
Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, et al. Scene
transformer: A unified multi-task model for behavior prediction and planning. arXiv preprint
arXiv:2106.08417, 2021
[2] Tang, Charlie, and Russ R. Salakhutdinov. "Multiple futures prediction." Advances in Neural Information Processing Systems 32 (2019): 15424-15434.
[3] Li, Lingyun Luke, et al. "End-to-end contextual perception and prediction with interaction transformer." 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020.
[4] Yuan, Ye, et al. "AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting." arXiv preprint arXiv:2103.14023 (2021).
[5] Girgis, Roger, et al. "Autobots: Latent Variable Sequential Set Transformers." arXiv preprint arXiv:2104.00563 (2021).
[6] Casas, Sergio, et al. "Implicit latent variable model for scene-consistent motion forecasting." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28,
[7] Ming Liang, Bin Yang, Rui Hu, Yun Chen, Renjie Liao, Song Feng, and Raquel Urtasun. Learning
lane graph representations for motion forecasting. In ECCV, 2020

The evidence to be evaluated is:However, some methods rather adopt a candidate-based approach where potential endpoints are obtained either from anchor trajectories through clustering (Chai et al., 2020; Phan-Minh et al., 2020) or from a model-based generator (Song et al., 2021). Other approaches use a broader set of candidates from the context graph (Zhang et al., 2020; Zhao et al., 2020; Zeng et al., 2021; Kim et al., 2021) or a dense grid around the target agent (Deo & Trivedi, 2020; Gu et al., 2021; Gilles et al., 2021;a).

LLVM (Casas et al., 2020) uses scene latent representations conditioned on all agents to generate scene-consistent samples, but its variational inference does not provide a confidence score for each modality, hence LookOut (Cui et al., 2021) proposes a scenario scoring function and a diverse sampler to improve sample efficiency.

Finally, none of these previous methods considers the problems of scene consistency for multi-agent prediction.

Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

Having a heatmap as output grants us the good property of letting us decide to privilege coverage in order to give a wide spectrum of candidates to the scene recombination process. From each heatmap, we therefore decode \(K\) end points using the same MissRate optimization algorithm as Gilles et al. (2021). We then generate the full trajectories for each end point using also the same model, a fully-connected MLP. The MLP takes the car history and predicted endpoint as input and produces \(T\) 2D-coordinates representing the full future trajectory.

Initially, the prediction output of the model can be defined as marginal, as all \(A\) agents have been predicted and sampled independently in order to get \(K\) possible endpoint each. Our goal is to output a set of scene predictions \(\mathcal{J}=(L,A)\) from the marginal prediction \(\mathcal{M}=(A,K)\), where each scene modality \(l\) belonging to \(\mathcal{J}\) is an association of endpoints \(p_{l}^{a}\) for each agent \(a\). To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions. For a given agent \(a\), the modality selected for the scene \(l\) would be a combination of this agent available marginal modalities \(p_{k}^{a}\in\mathcal{M}\). Each scene modality \(l\) would select a different association between the agents.

The model learns \(L\) scene modality embeddings \(S_{l}\) of \(D\) features each. \(K\times A\) agent modality vectors \(A_{k}^{a}\) are also derived from each agent modality endpoint.

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\] The recombination module is trained with the scene-level minimum displacement error, where for each modality inside a single scene all predicted agent displacement losses are averaged before taking the minimum (winner-takes-all loss) over the \(L\) scene modalities.

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We have presented THOMAS, a recombination module that can be added after any trajectory prediction module outputing multi-modal predictions. By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it. We show significant performance increase when adding the THOMAS module compared to the vanilla model and achieve state-of-the-art results compared to already existing methods tackling scene-consistent predictions.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "We design a novel recombination model able to recombine the sampled endpoints to obtain scene-consistent trajectories across the agents."

Summary of Evaluation:
The evidence strongly supports this claim through detailed technical descriptions, architectural diagrams, and qualitative results. The paper presents a clear recombination methodology using scene modality embeddings, agent modality vectors, and a differentiable soft argmax approach. The qualitative results in Figures 5 and 6 demonstrate the model's ability to generate collision-free, scene-consistent trajectories. The reviewer's comments also acknowledge the novelty of the re-estimation approach based on heatmap endpoints. The claim is well-stated and accurately reflects the evidence without exaggeration.

<score>1</score>

Justification:
1. Detailed technical evidence of the recombination model's design is provided, including mathematical formulations and architectural details.
2. Visual evidence in figures demonstrates the model's effectiveness in generating scene-consistent trajectories.
3. Quantitative results show improvement in collision rates (7.2% to 2.6%) and joint SMR (14.8% to 11.8%).
4. The reviewer's positive comments confirm the novelty and natural progression of the approach.
5. The claim uses appropriate, measured language without overstating capabilities.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We design a novel recombination model able to recombine the sampled endpoints to obtain scene-consistent trajectories across the agents.

The review comment to be evaluated is:

Strengths:
1. The paper is well written and easy to follow.
2. Good performance was achieved on the Interaction multi-agent prediction challenge.

Weaknesses:
1. The method largely follows previous works, with little technical contributions. And even for these limited technical novelties, many of them are not well supported with experimemts.
In the end of introduction section, the authors summarize three main contributions:

i) an efficient graph-based model. However, the implementation mainly follows LaneGCN and GOHOME.

ii) a collision-aware endpoint sampling. The algorithm mainly follows GOHOME, with a small modification in that when we sample an endpoint for actor a modality k, we not only mask out the same region for other modalities of actor a, but also mask out for other actors of modality k. However, this modification doesn't make sense to me, as we still have a scene-consistent modality recombination module afterwards, therefore the second mask-out won't make much difference here. I also don't see an ablation study for this specific change (ie, do not mask-out for other actors of modality k, which means exactly follows the original algorithm in GOHOME). I want to see this ablation for the last three rows in Table 3.

iii) a novel recombination model that could obtain scene-consistent trajectories across the agents. To me this is the core novelty of the paper. From the last two rows in Table 3 we can see that with the proposed recombination module joint metrics improve a lot. However, the marginal metrics degrade a lot (e.g., miss rate more than doubled). The only explanation is that with the proposed module, the same modality (also the sub-optimal one) of an actor is chosen for multiple times in scene-level multi-modalities. To me this may not be what we want: scene-level consistency, but sacrifices actor-level precision a lot. Further analysis is needed here to convince me that this trade-off is worthwhile.

2. Another technical novelty that's not mentioned is the hierarchical iterative refinement of endpoints from the heatmap. However, no ablation study is done on this. I'd like to see a speed-performance comparison with original endpoint estimation model (as in GOHOME).

3. Also, the overall runtime efficiency of the method is not given.

4. GOHOME is the closest approach but I don't see it as a baseline in the experiments.

The evidence to be evaluated is:However, some methods rather adopt a candidate-based approach where potential endpoints are obtained either from anchor trajectories through clustering (Chai et al., 2020; Phan-Minh et al., 2020) or from a model-based generator (Song et al., 2021). Other approaches use a broader set of candidates from the context graph (Zhang et al., 2020; Zhao et al., 2020; Zeng et al., 2021; Kim et al., 2021) or a dense grid around the target agent (Deo & Trivedi, 2020; Gu et al., 2021; Gilles et al., 2021;a).

LLVM (Casas et al., 2020) uses scene latent representations conditioned on all agents to generate scene-consistent samples, but its variational inference does not provide a confidence score for each modality, hence LookOut (Cui et al., 2021) proposes a scenario scoring function and a diverse sampler to improve sample efficiency.

Finally, none of these previous methods considers the problems of scene consistency for multi-agent prediction.

Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

Having a heatmap as output grants us the good property of letting us decide to privilege coverage in order to give a wide spectrum of candidates to the scene recombination process. From each heatmap, we therefore decode \(K\) end points using the same MissRate optimization algorithm as Gilles et al. (2021). We then generate the full trajectories for each end point using also the same model, a fully-connected MLP. The MLP takes the car history and predicted endpoint as input and produces \(T\) 2D-coordinates representing the full future trajectory.

Initially, the prediction output of the model can be defined as marginal, as all \(A\) agents have been predicted and sampled independently in order to get \(K\) possible endpoint each. Our goal is to output a set of scene predictions \(\mathcal{J}=(L,A)\) from the marginal prediction \(\mathcal{M}=(A,K)\), where each scene modality \(l\) belonging to \(\mathcal{J}\) is an association of endpoints \(p_{l}^{a}\) for each agent \(a\). To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions. For a given agent \(a\), the modality selected for the scene \(l\) would be a combination of this agent available marginal modalities \(p_{k}^{a}\in\mathcal{M}\). Each scene modality \(l\) would select a different association between the agents.

The model learns \(L\) scene modality embeddings \(S_{l}\) of \(D\) features each. \(K\times A\) agent modality vectors \(A_{k}^{a}\) are also derived from each agent modality endpoint.

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\] The recombination module is trained with the scene-level minimum displacement error, where for each modality inside a single scene all predicted agent displacement losses are averaged before taking the minimum (winner-takes-all loss) over the \(L\) scene modalities.

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We have presented THOMAS, a recombination module that can be added after any trajectory prediction module outputing multi-modal predictions. By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it. We show significant performance increase when adding the THOMAS module compared to the vanilla model and achieve state-of-the-art results compared to already existing methods tackling scene-consistent predictions.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I assess that the authors make a somewhat overstated claim about the novelty of their recombination model. While they do present a method for recombining endpoints into scene-consistent trajectories, the reviewer points out that this builds heavily on previous work like GOHOME, with limited technical novelty. The evidence shows they developed an approach using scene modality embeddings and cross-attention to combine agent trajectories, but the "novel" aspect is debatable given existing methods like LLVM that also aimed for scene consistency. The claim would be more accurate if tempered regarding novelty.

<score>3</score>

Justification:
1. While the recombination model is functional and shows improvements in joint metrics (reducing joint SMR from 14.8% to 11.8%), calling it "novel" appears overstated given:
- The reviewer notes it largely follows previous works
- Similar scene consistency goals were pursued by prior methods like LLVM
- The approach builds substantially on existing techniques

2. The evidence demonstrates they developed a working solution but would be better stated as an "effective" or "improved" recombination model rather than emphasizing novelty.

3. The figures and technical details support that they implemented a recombination approach that achieves scene consistency, but the novelty claim isn't strongly substantiated compared to prior art.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

* We design a novel recombination model able to recombine the sampled endpoints to obtain scene-consistent trajectories across the agents.

The review comment to be evaluated is:

- The overall idea of iterative hierarchical heatmap refinement is interesting and appears to be novel. The experiments demonstrate the ideas validity, although there is definitely room for further strengthening.

- The related work is thorough, one quick comment here is to also reference the following work in the discussion of methods which can predict multiple agents at the same time (beginning of the last paragraph before Section 3): "MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control" by B. Ivanovic, A. Elhafsi, G. Rosman, A. Gaidon, M. Pavone in CoRL 2020.

- Acronyms should be explained when they are first used. For instance, it might not be obvious what a UGRU is in Section 3.1.1. What does MR optimization mean in Section 3.1.3?

- Do not use contractions in scientific writing (e.g., change "don't" to "do not").

- A thorough round of editing is necessary, there seem to be writing errors that materially change the meaning of sentences. For instance, "The final output is therefore no strictly a re-ordering..." Should this have said "not strictly"? Same question for "which is JointMR where colliding modalities are also counted as misses even if they are closer than the defined threshold." Should this have just said "where colliding modalities are also counted as misses?" It is not very clear what these metrics mean here.

- The novelty with respect to the original GOHOME work needs to be made clearer, there seem to be many reused components and it is difficult to tell which parts are new in this work.

- Related to the above point, it is a bit difficult to understand this work's core contributions in Sections 3.2.1 and 3.2.2, also because the writing makes this difficult (e.g., a spurious use of "However" makes it difficult to determine if a statement is being negated or not).

- There are some key insights missing from the experimental section, which will be expanded on below.

- Ultimately, the method is only evaluated on one dataset, and it is difficult to determine if its performance would generalize to other datasets (for instance, what about nuScenes, Lyft, Waymo Open, Argoverse, etc?).

The evidence to be evaluated is:However, some methods rather adopt a candidate-based approach where potential endpoints are obtained either from anchor trajectories through clustering (Chai et al., 2020; Phan-Minh et al., 2020) or from a model-based generator (Song et al., 2021). Other approaches use a broader set of candidates from the context graph (Zhang et al., 2020; Zhao et al., 2020; Zeng et al., 2021; Kim et al., 2021) or a dense grid around the target agent (Deo & Trivedi, 2020; Gu et al., 2021; Gilles et al., 2021;a).

LLVM (Casas et al., 2020) uses scene latent representations conditioned on all agents to generate scene-consistent samples, but its variational inference does not provide a confidence score for each modality, hence LookOut (Cui et al., 2021) proposes a scenario scoring function and a diverse sampler to improve sample efficiency.

Finally, none of these previous methods considers the problems of scene consistency for multi-agent prediction.

Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

Having a heatmap as output grants us the good property of letting us decide to privilege coverage in order to give a wide spectrum of candidates to the scene recombination process. From each heatmap, we therefore decode \(K\) end points using the same MissRate optimization algorithm as Gilles et al. (2021). We then generate the full trajectories for each end point using also the same model, a fully-connected MLP. The MLP takes the car history and predicted endpoint as input and produces \(T\) 2D-coordinates representing the full future trajectory.

Initially, the prediction output of the model can be defined as marginal, as all \(A\) agents have been predicted and sampled independently in order to get \(K\) possible endpoint each. Our goal is to output a set of scene predictions \(\mathcal{J}=(L,A)\) from the marginal prediction \(\mathcal{M}=(A,K)\), where each scene modality \(l\) belonging to \(\mathcal{J}\) is an association of endpoints \(p_{l}^{a}\) for each agent \(a\). To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions. For a given agent \(a\), the modality selected for the scene \(l\) would be a combination of this agent available marginal modalities \(p_{k}^{a}\in\mathcal{M}\). Each scene modality \(l\) would select a different association between the agents.

The model learns \(L\) scene modality embeddings \(S_{l}\) of \(D\) features each. \(K\times A\) agent modality vectors \(A_{k}^{a}\) are also derived from each agent modality endpoint.

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\] The recombination module is trained with the scene-level minimum displacement error, where for each modality inside a single scene all predicted agent displacement losses are averaged before taking the minimum (winner-takes-all loss) over the \(L\) scene modalities.

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We have presented THOMAS, a recombination module that can be added after any trajectory prediction module outputing multi-modal predictions. By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it. We show significant performance increase when adding the THOMAS module compared to the vanilla model and achieve state-of-the-art results compared to already existing methods tackling scene-consistent predictions.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the evidence, the claim about designing "a novel recombination model able to recombine the sampled endpoints to obtain scene-consistent trajectories across the agents" appears to be well-stated. The evidence thoroughly demonstrates:
- A detailed technical description of the recombination module and its methodology
- Visual examples showing how the model successfully recombines trajectories to avoid collisions
- Quantitative results showing improvement in scene consistency (joint SMR reduction from 14.8% to 11.8%)
- Clear differentiation from prior work through the novel scene-consistency module
- Concrete implementation details and mathematical formulations

The reviewer comments, while noting some writing clarity issues, do not dispute the technical novelty or functionality of the recombination model.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

This recombination module is able to realign the initially independent modalities so that they do no collide and are coherent with each other.

The evidence to be evaluated is:Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

This encoding \(F_{a}\) is used in the next decoder operations and is also stored to be potentially used in modality recombination described in Sec. 3.2.

The difficulty in multimodal multi-agent prediction resides in having coherent modalities between each agent.

Moreover, these modalities must not collide with each other, as they should represent realistic scenarios.

To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions.

Each scene modality \(l\) would select a different association between the agents.

The scene modality vectors \(S_{l}\) are'specialized' by querying the available modality proposals \(A_{k}^{a}\) of each agent through cross-attention layers. Subsequently, a matching score \(c_{l}^{k,a}\) between each scene modality \(S_{l}\) and each agent modality \(A_{k}^{a}\) is computed. This matching score can be intuitively interpreted as the selection of the role (maneuver \(k\)) that the agent \(a\) would play in the overall traffic scene \(l\):

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\]

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We display four qualitative examples in Fig. 5 with colliding modalities in the \(Before\) model (in dashed orange) and the solved modality (in full line orange) after recombination.

We also display in dashed grey the other 5 predicted \(Before\) modalities, and highlight that the recombination model indeed selects modalities already available in the vanilla set and reorders them so that non-colliding modalities are aligned together.

While the \(Before\) model tries to spread the modalities for all agents to minimize marginal miss-rate, the recombined model presents much less spread compared to the original model, maintaining a multimodal behavior only in presence of very different possible agent intentions such as different possible exits or turn choices.

By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 8: Detailed illustration of our heatmap generator model

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the recombination module's ability to realign modalities for coherence and collision avoidance is well-supported by both textual and visual evidence. The evidence demonstrates:

1. Quantitative improvement in collision reduction (7.2% to 2.6%)
2. Visual examples in Figure 5 showing successful realignment of colliding trajectories
3. Technical details of the cross-attention mechanism that enables modality selection
4. Qualitative results in Figure 6 showing more focused and realistic predictions

The claim accurately reflects the capabilities demonstrated without exaggeration, focusing on the core functionality that is thoroughly evidenced.

<score>1</score>

Justification:
- The claim states exactly what the evidence shows: the module realigns independent modalities to be coherent and collision-free
- Figures 5 and 6 provide clear visual validation of this capability
- The technical explanation of the cross-attention mechanism provides a solid theoretical foundation
- Quantitative metrics confirm the effectiveness
- The claim doesn't overstate capabilities beyond what's demonstrated

The evidence strongly supports that this is a well-stated claim that accurately reflects the system's demonstrated capabilities without exaggeration.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

This recombination module is able to realign the initially independent modalities so that they do no collide and are coherent with each other.

The review comment to be evaluated is:

### Strengths:
- __Originality__: the two main contributions outlined in the summary of the paper are fairly distinctive from previous works.
  - The hierarchical heatmap prediction tackles one of the main issues of heatmap representations, which is their heavy memory consumption.
  - The learnable recombination method is general enough to be suitable for any marginal prediction model, which makes me think it would have a solid impact on the community.
- Introduction and related work are succinct and touch upon the main relevant points to the future prediction task.
- Figures are useful to understand the method as well as to evaluate the performance qualitatively.

### Weaknesses:
- __Writing ambiguity and lack of details__: I am not sure the write-up is polished enough for submission to ICLR. I outline a few reasons below:
  - Throughout the paper, there is no mention or description of the loss functions utilized, and barely any mention of the overall training process (with the exception of a sentence in 3.1.3). In the appendix, the authors describe how the heatmap output is optimized, but there is no mention about how the learnable recombination module is trained, which is one of the main contributions of the paper.
  - Because of this lack of description of the overall training process and loss functions, it is unclear if in the experiments the authors utilize 2 separate THOMAS models - one for marginal prediction and one for joint prediction. For instance, are the marginal metrics and joint metrics in Table 1 coming from a single model, or two separately trained models? A sentence that contributes to this confusion is in 3.1.3, where the authors mention that they decode K end points using the same MR optimization algorithm as Gilles et al. (2021b). However, in 3.2.1 the authors enhance this algorithm with collision-free sampling. Please clarify this in the rebuttal.
  - The paper criticizes generative models in the second paragraph of the related work for not providing a probability value for each prediction, which makes me expect the method to provide such scores. However, there is no mention about it in the approach or experiments sections.
  - In Table 3, the Marg and Joint objectives are not defined.
- It is unclear how some pieces tie together. My main concern is with respect to collision-free end point sampling (3.2.1) and recombination (3.2.2). The heuristic in 3.2.1 seems to assume that the modalities are "already ordered" since the algorithm basically avoids sampling close end points across actors for a given modality k. However, the recombination can perform arbitrary linear combinations across modes, thus not respecting this. Is the idea that only one of the two methods are used to perform inference/training on a model? If so, I would suggest moving 3.2.1 to the experiments as a baseline/ablation, and I definitely would not claim it as a contribution (particularly since it's just a small tweak on top of a previously proposed algorithm).
- Experiments are not as thorough as I would expect. This is understandable for the joint prediction setting as there are very few baselines and the dataset/benchmarks on this task have just started. In fact, I appreciate the effort of the authors of re-implementing ILVM and SceneTransformer. However, with respect to marginal prediction, there are a lot of methods and benchmarks out there. Another aspect that would have been great to get more insight on is the hierarchical heatmap output. What's the tradeoff between computation (both runtime and memory) and accuracy, with respect to dense heatmap approaches? How was the current 3-layer hierarchy chosen?

### Additional feedback:
In this section I provide additional feedback for rebuttal, but please address first the main points above.
- Re: training details in the appendix. I find it surprising that the method is trained with random sampling of up to 8 agents, given that for joint prediction there might be other agents that are particularly relevant to the task. Was this better than lowering the batch size? If so, is it because it is rare that a scene contains > 8 actors in the INTERACTION dataset?
- The equation that describes the weighted linear combination of agent modalities seems incomplete. As far as I understand, there should be a weighted sum in the equation, using the scores coming out of the softmax.
- Related work is missing some relevant works. In the last paragraph, joint autoregressive methods should be mentioned (e.g., PRECOG[A] - see references at the bottom) as they were the first to model the interactions across agents at the output level. With respect to generative models and their random sampling and lack of confidence scores per modality, LookOut[B] is a recent method that addresses both these concerns with a learnable diverse sampler and scene scoring. The authors can treat this as concurrent work since it was just published at ICCV, but please include it in the discussion as it is highly related.
- Is the notation of the joint metrics coming from the Interpret challenge? If not, I would suggest unifying this with the names proposed originally in ILVM (e.g., minJointFDE --> minSFDE, CrossCollisionRate --> SCR).
- While the sparse and hierarchical heatmaps make sense to mitigate their high resource consumption, I am a bit concerned about the fact that endpoints are only sampled from the finest resolution, as this might not contain more unlikely but safety-critical modalities. Overall, the heuristic refinement of the most likely grid cells seems contradictory with the endpoint sampling objective of maximizing coverage.
- The writing would benefit from more intuitions. For instance, the method samples heuristically from the heatmaps to maximize coverage, but why is this important to the overall self-driving task?
- I personally did not get much out of figure 4, perhaps adding the shapes of the tensors could be helpful.
- I would avoid calling the combination method an "ordering", as the method is more general than that.
- How come the CrossCollisionRate is still 2.6% when using the collision-free end point sampling? I would expect it to be zero.

### Additional references:
- [A] Rhinehart, Nicholas, et al. "Precog: Prediction conditioned on goals in visual multi-agent settings." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.
- [B] Cui, Alexander, et al. "LookOut: Diverse multi-future prediction and planning for self-driving." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

The evidence to be evaluated is:Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

This encoding \(F_{a}\) is used in the next decoder operations and is also stored to be potentially used in modality recombination described in Sec. 3.2.

The difficulty in multimodal multi-agent prediction resides in having coherent modalities between each agent.

Moreover, these modalities must not collide with each other, as they should represent realistic scenarios.

To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions.

Each scene modality \(l\) would select a different association between the agents.

The scene modality vectors \(S_{l}\) are'specialized' by querying the available modality proposals \(A_{k}^{a}\) of each agent through cross-attention layers. Subsequently, a matching score \(c_{l}^{k,a}\) between each scene modality \(S_{l}\) and each agent modality \(A_{k}^{a}\) is computed. This matching score can be intuitively interpreted as the selection of the role (maneuver \(k\)) that the agent \(a\) would play in the overall traffic scene \(l\):

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\]

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We display four qualitative examples in Fig. 5 with colliding modalities in the \(Before\) model (in dashed orange) and the solved modality (in full line orange) after recombination.

We also display in dashed grey the other 5 predicted \(Before\) modalities, and highlight that the recombination model indeed selects modalities already available in the vanilla set and reorders them so that non-colliding modalities are aligned together.

While the \(Before\) model tries to spread the modalities for all agents to minimize marginal miss-rate, the recombined model presents much less spread compared to the original model, maintaining a multimodal behavior only in presence of very different possible agent intentions such as different possible exits or turn choices.

By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 8: Detailed illustration of our heatmap generator model

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my analysis of the evidence, the claim is supported by both text descriptions and visual examples demonstrating how the recombination module successfully realigns modalities to achieve coherence and avoid collisions. The evidence shows:

1. A clear mechanism for recombination using cross-attention and weighted linear combinations
2. Quantitative improvements in collision reduction (7.2% to 2.6%)
3. Multiple qualitative examples in Figures 5 and 6 showing resolved collisions
4. Detailed technical explanation of how the module achieves coherence through scene modality vectors and matching scores

While the claim may seem strong, the comprehensive evidence across multiple aspects (theoretical, quantitative, and qualitative) demonstrates that it is well-stated and accurately reflects the module's capabilities without exaggeration.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

This recombination module is able to realign the initially independent modalities so that they do no collide and are coherent with each other.

The review comment to be evaluated is:

### Strong Points

The idea of performing re-estimation based on the computed heat map end-points
seems novel to me. This idea seems like a natural next step to improve multi agent motion
prediction. The fact that you first get the end-point and then reason about the scene is
intuitively sounding and seems to be a good solution for the problem.

The experiments on interact are a nice direction to be taken by the field.

The visualizations of the heat-maps and the different predictions was helpful.

### Weak Points

#### I could not find a clear distinction with literature
While reading the related work section I found hard to contrast the ideas of the paper with the ones proposed on the literature.
The difference between this method and GOHOME was  not clear to me in this sense.
From what I understood the main contribution here  is the modality combination based on the collision
free end-point sampling. However, this was not obvious for me when reading the paper.

#### The implemented baselines seemed to have performed way worse than expected.
For example, both [1] and [7] are methods designed for *joint* prediction and scene level prediction. This is contradicted
by the observations made by the authors on page 8, quote: " ... This problem is aggravated in the joint training case, since the modality selected is
the same for all agents in a training sample....". Both [1] and [3] fairly confidently claim producing multimodal joint predictions.
Either the version implemented by the authors does not correctly match the other works in the literature or there are specific findings
on the limitations existent in other methods.
Also, the paper lack comparison with latent variable based [2][5][[6] multi modal prediction methods. They seem to perform
better when the issue is diverse prediction. Definitely the mode does not tend to be the same for all predictions in those cases.

#### A single dataset evalution might not be enough for the community to access the value of this paper.

 Motion prediction is currently a very competitive scene. Even though I appreciate the idea on focusing reporting results on multi-agents
and joint prediction tasks, it is important to also evaluate the prediction on more standard benchmarks such as argoverse.
With results on argoverse, me and the rest of the community can more quickly access the the capabilities of the proposed
 heat map prediction pipeline, on producing consistent ego-agent motion prediction. I understand that this would not be the main
results, but it would be helpful.

##### Computational cost is apparently not shown.

I could not find the computational cost of running and training the proposed method on the paper. It seems that even
with the optimizations made on the heat map estimation the training time and inference time should be high.
Those kinds of motion prediction models have the necessity to further scale with way more data. Thus, highly
complex methods can be hard to train. There is also the case of potential future execution in an embedded hardware.

### Specific Questions

I am not  sure about the scene modality vectors ( Section 3.2.2), are they like seed parameters which are going to be used
to incorporate the joint consistent scene level ? I couldn't really find any sort of detailed explanation on that.

Why the lack of ADE for scene level metrics or using similar metrics as [3] ?

[1] Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zhengdong Zhang, Hao-Tien Lewis Chiang,
Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, et al. Scene
transformer: A unified multi-task model for behavior prediction and planning. arXiv preprint
arXiv:2106.08417, 2021
[2] Tang, Charlie, and Russ R. Salakhutdinov. "Multiple futures prediction." Advances in Neural Information Processing Systems 32 (2019): 15424-15434.
[3] Li, Lingyun Luke, et al. "End-to-end contextual perception and prediction with interaction transformer." 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020.
[4] Yuan, Ye, et al. "AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting." arXiv preprint arXiv:2103.14023 (2021).
[5] Girgis, Roger, et al. "Autobots: Latent Variable Sequential Set Transformers." arXiv preprint arXiv:2104.00563 (2021).
[6] Casas, Sergio, et al. "Implicit latent variable model for scene-consistent motion forecasting." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28,
[7] Ming Liang, Bin Yang, Rui Hu, Yun Chen, Renjie Liao, Song Feng, and Raquel Urtasun. Learning
lane graph representations for motion forecasting. In ECCV, 2020

The evidence to be evaluated is:Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

This encoding \(F_{a}\) is used in the next decoder operations and is also stored to be potentially used in modality recombination described in Sec. 3.2.

The difficulty in multimodal multi-agent prediction resides in having coherent modalities between each agent.

Moreover, these modalities must not collide with each other, as they should represent realistic scenarios.

To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions.

Each scene modality \(l\) would select a different association between the agents.

The scene modality vectors \(S_{l}\) are'specialized' by querying the available modality proposals \(A_{k}^{a}\) of each agent through cross-attention layers. Subsequently, a matching score \(c_{l}^{k,a}\) between each scene modality \(S_{l}\) and each agent modality \(A_{k}^{a}\) is computed. This matching score can be intuitively interpreted as the selection of the role (maneuver \(k\)) that the agent \(a\) would play in the overall traffic scene \(l\):

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\]

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We display four qualitative examples in Fig. 5 with colliding modalities in the \(Before\) model (in dashed orange) and the solved modality (in full line orange) after recombination.

We also display in dashed grey the other 5 predicted \(Before\) modalities, and highlight that the recombination model indeed selects modalities already available in the vanilla set and reorders them so that non-colliding modalities are aligned together.

While the \(Before\) model tries to spread the modalities for all agents to minimize marginal miss-rate, the recombined model presents much less spread compared to the original model, maintaining a multimodal behavior only in presence of very different possible agent intentions such as different possible exits or turn choices.

By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 8: Detailed illustration of our heatmap generator model

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find that the claim is supported by both textual descriptions and visual examples, particularly in Figures 5 and 6. The evidence demonstrates that the recombination module successfully realigns independent modalities to avoid collisions and create coherent multi-agent predictions. This is quantitatively supported by the reduction in collision rate from 7.2% to 2.6% and improvement in joint SMR from 14.8% to 11.8%. The visualizations clearly show how the module transforms initially colliding trajectories into collision-free, coherent predictions. The claim appears to be well-stated and accurately reflects the capabilities demonstrated in the evidence without exaggeration.

<score>1</score>

Justification:
1. The evidence directly supports the claim through:
- Quantitative metrics showing significant reduction in collisions
- Clear visual examples in Figures 5 and 6 demonstrating the before/after realignment
- Detailed technical explanation of how the recombination works through scene modality vectors and cross-attention mechanisms
- Reviewer comments acknowledge the novelty and effectiveness of the approach

2. The claim uses appropriate, measured language:
- "able to realign" rather than claiming perfect performance
- Focuses on the core capabilities (avoiding collisions and maintaining coherence) without overstating broader implications
- Matches the scope and limitations shown in the evidence

3. No significant exaggeration or overselling of capabilities beyond what is demonstrated in the comprehensive experimental results and visualizations provided.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

This recombination module is able to realign the initially independent modalities so that they do no collide and are coherent with each other.

The review comment to be evaluated is:

Strengths:
1. The paper is well written and easy to follow.
2. Good performance was achieved on the Interaction multi-agent prediction challenge.

Weaknesses:
1. The method largely follows previous works, with little technical contributions. And even for these limited technical novelties, many of them are not well supported with experimemts.
In the end of introduction section, the authors summarize three main contributions:

i) an efficient graph-based model. However, the implementation mainly follows LaneGCN and GOHOME.

ii) a collision-aware endpoint sampling. The algorithm mainly follows GOHOME, with a small modification in that when we sample an endpoint for actor a modality k, we not only mask out the same region for other modalities of actor a, but also mask out for other actors of modality k. However, this modification doesn't make sense to me, as we still have a scene-consistent modality recombination module afterwards, therefore the second mask-out won't make much difference here. I also don't see an ablation study for this specific change (ie, do not mask-out for other actors of modality k, which means exactly follows the original algorithm in GOHOME). I want to see this ablation for the last three rows in Table 3.

iii) a novel recombination model that could obtain scene-consistent trajectories across the agents. To me this is the core novelty of the paper. From the last two rows in Table 3 we can see that with the proposed recombination module joint metrics improve a lot. However, the marginal metrics degrade a lot (e.g., miss rate more than doubled). The only explanation is that with the proposed module, the same modality (also the sub-optimal one) of an actor is chosen for multiple times in scene-level multi-modalities. To me this may not be what we want: scene-level consistency, but sacrifices actor-level precision a lot. Further analysis is needed here to convince me that this trade-off is worthwhile.

2. Another technical novelty that's not mentioned is the hierarchical iterative refinement of endpoints from the heatmap. However, no ablation study is done on this. I'd like to see a speed-performance comparison with original endpoint estimation model (as in GOHOME).

3. Also, the overall runtime efficiency of the method is not given.

4. GOHOME is the closest approach but I don't see it as a baseline in the experiments.

The evidence to be evaluated is:Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

This encoding \(F_{a}\) is used in the next decoder operations and is also stored to be potentially used in modality recombination described in Sec. 3.2.

The difficulty in multimodal multi-agent prediction resides in having coherent modalities between each agent.

Moreover, these modalities must not collide with each other, as they should represent realistic scenarios.

To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions.

Each scene modality \(l\) would select a different association between the agents.

The scene modality vectors \(S_{l}\) are'specialized' by querying the available modality proposals \(A_{k}^{a}\) of each agent through cross-attention layers. Subsequently, a matching score \(c_{l}^{k,a}\) between each scene modality \(S_{l}\) and each agent modality \(A_{k}^{a}\) is computed. This matching score can be intuitively interpreted as the selection of the role (maneuver \(k\)) that the agent \(a\) would play in the overall traffic scene \(l\):

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\]

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We display four qualitative examples in Fig. 5 with colliding modalities in the \(Before\) model (in dashed orange) and the solved modality (in full line orange) after recombination.

We also display in dashed grey the other 5 predicted \(Before\) modalities, and highlight that the recombination model indeed selects modalities already available in the vanilla set and reorders them so that non-colliding modalities are aligned together.

While the \(Before\) model tries to spread the modalities for all agents to minimize marginal miss-rate, the recombined model presents much less spread compared to the original model, maintaining a multimodal behavior only in presence of very different possible agent intentions such as different possible exits or turn choices.

By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 8: Detailed illustration of our heatmap generator model

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim that "This recombination module is able to realign the initially independent modalities so that they do not collide and are coherent with each other" appears somewhat overstated. While the evidence shows the module does help reduce collisions (from 7.2% to 2.6%) and improves joint SMR (from 14.8% to 11.8%), there are significant trade-offs not acknowledged in the claim. The reviewer points out that marginal metrics degrade substantially (miss rate more than doubled), suggesting the module achieves scene-level consistency by sacrificing actor-level precision. The claim would be more accurate if it acknowledged these limitations.

<score>4</score>

Justification:
1. The claim portrays the recombination as fully successful without acknowledging important limitations:
- Evidence shows marginal performance metrics significantly degrade
- Reviewer notes "scene-level consistency, but sacrifices actor-level precision a lot"
- No discussion of this fundamental trade-off in the claim

2. Supporting evidence shows mixed results:
- Positive: Collision rate drops from 7.2% to 2.6%
- Positive: Joint SMR improves from 14.8% to 11.8%
- Negative: Major degradation in marginal metrics
- Qualitative examples in Fig. 5 and Fig. 6 show the model does achieve some coherent realignment

3. The reviewer's comment specifically questions whether this trade-off is worthwhile and notes "Further analysis is needed here to convince me that this trade-off is worthwhile."

A more well-stated claim would acknowledge both the benefits and limitations of the recombination approach, particularly noting the trade-off between scene-level consistency and actor-level precision.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

This recombination module is able to realign the initially independent modalities so that they do no collide and are coherent with each other.

The review comment to be evaluated is:

- The overall idea of iterative hierarchical heatmap refinement is interesting and appears to be novel. The experiments demonstrate the ideas validity, although there is definitely room for further strengthening.

- The related work is thorough, one quick comment here is to also reference the following work in the discussion of methods which can predict multiple agents at the same time (beginning of the last paragraph before Section 3): "MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control" by B. Ivanovic, A. Elhafsi, G. Rosman, A. Gaidon, M. Pavone in CoRL 2020.

- Acronyms should be explained when they are first used. For instance, it might not be obvious what a UGRU is in Section 3.1.1. What does MR optimization mean in Section 3.1.3?

- Do not use contractions in scientific writing (e.g., change "don't" to "do not").

- A thorough round of editing is necessary, there seem to be writing errors that materially change the meaning of sentences. For instance, "The final output is therefore no strictly a re-ordering..." Should this have said "not strictly"? Same question for "which is JointMR where colliding modalities are also counted as misses even if they are closer than the defined threshold." Should this have just said "where colliding modalities are also counted as misses?" It is not very clear what these metrics mean here.

- The novelty with respect to the original GOHOME work needs to be made clearer, there seem to be many reused components and it is difficult to tell which parts are new in this work.

- Related to the above point, it is a bit difficult to understand this work's core contributions in Sections 3.2.1 and 3.2.2, also because the writing makes this difficult (e.g., a spurious use of "However" makes it difficult to determine if a statement is being negated or not).

- There are some key insights missing from the experimental section, which will be expanded on below.

- Ultimately, the method is only evaluated on one dataset, and it is difficult to determine if its performance would generalize to other datasets (for instance, what about nuScenes, Lyft, Waymo Open, Argoverse, etc?).

The evidence to be evaluated is:Finally, we recombine the sampled endpoints into scene-consistent modalities across agents and build the full trajectories for each agent.

Furthermore, we add a novel scene-consistency module that recombines the marginal outputs into a joint prediction.

This encoding \(F_{a}\) is used in the next decoder operations and is also stored to be potentially used in modality recombination described in Sec. 3.2.

The difficulty in multimodal multi-agent prediction resides in having coherent modalities between each agent.

Moreover, these modalities must not collide with each other, as they should represent realistic scenarios.

To achieve this, our main hypothesis lays in the fact that good trajectory proposals are already present in the marginal predictions, but they need to be coherently aligned among agents to achieve a set of overall consistent scene predictions.

Each scene modality \(l\) would select a different association between the agents.

The scene modality vectors \(S_{l}\) are'specialized' by querying the available modality proposals \(A_{k}^{a}\) of each agent through cross-attention layers. Subsequently, a matching score \(c_{l}^{k,a}\) between each scene modality \(S_{l}\) and each agent modality \(A_{k}^{a}\) is computed. This matching score can be intuitively interpreted as the selection of the role (maneuver \(k\)) that the agent \(a\) would play in the overall traffic scene \(l\):

Since argmax is non-differentiable, we employ a soft argmax as a weighted linear combination of the agent modalities \(p_{k}^{a}\) using a softmax on the \(c_{l}^{k,a}\) scores: \[p_{l}^{a}=\sum_{k}softmax(c_{l}^{k,a})p_{k}^{a}\]

Compared to these baselines, THOMAS can be seen as an hybrid sampling method that takes the result of deterministic sampling as input and learns to recombine it into a more coherent solution. With regard to the joint algorithmic sampling that only tackled collisions but has little to no effect on actual consistency, as highlighted by the big drop in collision rate from \(7.2\%\) to \(2.6\%\) but a similar joint SMR, THOMAS actually brings a lot of consistency in the multi-agent prediction and drops the joint SMR from \(14.8\%\) to \(11.8\%\).

We display four qualitative examples in Fig. 5 with colliding modalities in the \(Before\) model (in dashed orange) and the solved modality (in full line orange) after recombination.

We also display in dashed grey the other 5 predicted \(Before\) modalities, and highlight that the recombination model indeed selects modalities already available in the vanilla set and reorders them so that non-colliding modalities are aligned together.

While the \(Before\) model tries to spread the modalities for all agents to minimize marginal miss-rate, the recombined model presents much less spread compared to the original model, maintaining a multimodal behavior only in presence of very different possible agent intentions such as different possible exits or turn choices.

By design, THOMAS allows to generate scene-consistent modalities across all agents by making the scene modalities select coherent agent modalities and restricting the modality budget on the agents that truly need it.

We designed a deterministic sampling algorithm based on the heatmaps generated in previous section in order to sample endpoints for each agent in a collision aware manner.

For a single modality \(k\), we predict the possible endpoint of a first agent \(a\) by taking the maximum accumulated predicted probability under an area of radius \(r\). We then not only set to zero the heatmap values of this agent heatmap \(\mathcal{I}^{a^{\prime}}_{k^{\prime}}\) around the sampled location so not to sample it in the next modalities \(k^{\prime}\), but we also set to zero the same area on the heatmaps \(\mathcal{I}^{a^{\prime}}_{k}\) of the other agents \(a^{\prime}\) on the same modality \(k\), so that these other agents cannot be sampled at the same position for this modality. This way, we try to enforce collision-free endpoints, and expect that considering collisions brings logic to improve the overall consistency of the predictions.

Input: Figure 1: Illustration of the THOMAS multi-agent prediction pipeline

Input: Figure 5: Qualitative examples of recombination model assembling collision-free modalities together compared to initial colliding modalities. For each example we display the general context with highlighted agents and area of interest, then two zooms in on the agents, one displaying the initial best modality before recombination in dashed orange and all the other available modalities in grey. The second zooms shows the best modality after recombination in full line orange.

Input: Figure 8: Detailed illustration of our heatmap generator model

Input: Figure 6: Qualitative examples of recombination model selecting fewer but more pronounced and impactful agent modalities compared to initial colliding modalities. For each example we display on the left the vanilla model modalities with dashed lines, with the initial best modality in dashed orange and all the other available modalities in grey. On the right we display the selected modalities after recombination, where the model focuses on the most likely occurrence in most agents.

Input: Figure 4: Illustration of THOMAS methods for generation of scene-consistent agent modalities

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the provided evidence, I find this claim to be well-stated and supported. The evidence demonstrates that:

1. The recombination module successfully realigns modalities through a cross-attention mechanism that computes matching scores between scene modalities and agent modalities
2. Quantitative results show collision rates dropping from 7.2% to 2.6% after recombination
3. Visual evidence in Figures 5 and 6 clearly illustrates how the module resolves colliding trajectories into coherent, non-colliding predictions
4. The methodology is thoroughly explained with mathematical formulations showing how modalities are realigned

While the reviewer notes some writing clarity issues, these don't impact the fundamental validity of this specific claim about the module's capabilities.

<score>1</score>