Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Figure 1 shows an overview of our approach. We start with the graph structure (as shown on the left) and construct _Virtual Edges_ (middle) that capture the propagation of information between nodes at multiple propagation steps. This allows our approach to zoom out of the local message passing and relates distant nodes that do not have a direct connection in the original graph. We then use the virtual edges to direct the transformer attention (shown on the right) and node representations.

The evidence to be evaluated is:Gilmer et al. (2017) add "virtual edges" to shorten long distances, and Scarselli et al. (2008) add "supersource nodes". None of these works, however, integrate such virtual nodes or edges into the Transformer. Another line of work uses the attention mechanism Velickovic et al. (2018); Brody et al. (2022) to dynamically propagate information in the graph rather than use the original adjacency matrix or Laplacian.

Diffusion and other distance measures were used before to increase the expressivity of MP-GNNs Li et al. (2020), replace the local message passing scheme( Atwood and Towsley (2016); Wu et al. (2019); Klicpera et al. (2019, 2018)) or modulate the Transformers attention Mialon et al. (2021).

To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

First, GD embeds the structural relations between distant nodes in what we refer to as Virtual Edges. Then, the Transformer processes the nodes while using the virtual edges to direct the computation at different layers. _Virtual Edges_ are high-dimensional representations constructed between distant nodes in the graph. They contain rich information on the structural and positional relationships. To consider relations between distant nodes, the first step is broadening the receptive field on which the architecture operates.

We stack different powers of \(A\) into a 3 dimensional tensor \(E\in R^{n\times n\times k}\) \[E=[I|A|A^{2}|..|A^{k}] \tag{1}\] where \(k\) is the number of stacks and \(|\) denotes stacking matrices along the 3ed dimension and \(I\) is the identity matrix.

Xu et al. (2018), however, considering multiple powers of the matrix at once, we zoom out of the local Message Passing paradigm and discover information that GNNs may not detect. Virtual Edges contain structural information such as if nodes are on an odd length cycle, can distinguish many non-isomorphic graphs, and cover many distance measures such as shortest distance and generalized PageRank Li et al. (2020). In order to mix information between different propagation steps and extract meaningful structural relations, each stack is processed by a fully connected edge-wise feed-forward network.

Rather than using the original adjacency matrix in equation 1, we found it beneficial to learn a new adjacency matrix using the node and edge features. \[\hat{A_{ij}}=ReLU(BN([x_{i};x_{j},e_{ij}]W_{1}))W_{2} \tag{4}\] \[A_{ij}=normalize(\sigma(\hat{A_{ij}})) \tag{5}\] Where \(;\) means contacting vectors, \(\sigma\) is the sigmoid function and normalize means \(l1\) row normalization.

This aligns with existing approaches such as GAT (Velickovic et al. (2018); Brody et al. (2022)), which find it beneficial to dynamically learn a new adjacency matrix. Most Graph Transformers use the graph structure to either alter the node's representations or to affect the attention mechanism itself.

In the input layer, Self-Virtual Edges are added to the node representation as positional encoding, and at each attention layer, the Virtual Edges are reduced to an attention matrix that is combined with the standard dot-product attention. At each of the Transformer attention layers, we linearly project each virtual edge \(E_{ij}\) separately to get the positional attention score between \(i\) and \(j\). \[\hat{Att}_{ij}^{Position}=E_{ij}W_{p} \tag{6}\]We then combine the positional attention and the content(dot-product) attention scores and apply row-wise normalization. \[\hat{Att}^{Content}_{ij}=\frac{QK^{T}}{\sqrt{d}}_{ij} \tag{7}\] \[Att=normalize(exp(\hat{Att}^{Content})\odot\sigma(\hat{Att}^{Position})) \tag{8}\] Where \(\sigma\) is the sigmoid function and \(\odot\) means element-wise multiplication. In multi-head attention with \(h\) heads, the projection in equation 6 is done \(h\) times. equation 8 can be viewed as scaling the content attention coefficients based on the positional attention. As we will see in the next section, separating the attention based on content and positions and then combining them seems to assist with learning meaningful connections in the data and provides a natural visualization mechanism. Self-Virtual-Edges, \(E_{ii}\), are Virtual Edges between a node to itself. They contain important structural information, such as whether the node is part of an odd length cycle and the degrees of adjacent nodes. We project Self-Virtual-Edges to the node's representation dimensionality and add them as positional encoding.

Self-walks were shown (Dwivedi et al. (2022)) to be an effective positional encoding. Adding such features to the node representations allows the Transformer to use structural information in all of its layers, including the feed-forward network.

Looking at Figure 1(b), we can see the model learns to use the different attention mechanisms in an intuitive way, with content attention detecting nodes in the same color and position attention detecting nodes in the same row or column.

Using a controlled example, we demonstrated its effectiveness in modeling long-range interactions while providing better interpretability.

Another area of interest is amplifying virtual edges, for example, by considering paths between nodes rather than just information propagation.

Input: Figure 1: Illustration of our positional attention, focusing on node B. Information propagation from multiple propagation steps is combined to create Virtual Edges(colored) between distant nodes, which then direct the Transformer’s attention in each layer.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the claim provides a fair representation of the approach, supported by detailed mathematical formulations and clear explanations in the evidence. The figure accurately illustrates the three main components described: graph structure, virtual edges construction, and transformer attention direction. The evidence provides comprehensive technical details about virtual edge construction (Equations 1, 4-5), attention mechanisms (Equations 6-8), and integration with transformers. While some novelty claims are made, they are appropriately contextualized within prior work (Gilmer et al., 2017; Velickovic et al., 2018).

<score>1</score>

Justification:
1. The claim directly aligns with the mathematical formulation and implementation details provided in the evidence
2. Figure 1 precisely matches the three-step process described in the claim
3. The evidence includes thorough technical details supporting each component
4. Prior work is acknowledged appropriately
5. The description maintains academic rigor without overextending beyond what's demonstrated
6. Limitations and future work are openly acknowledged ("Another area of interest is amplifying virtual edges...")


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Figure 1 shows an overview of our approach. We start with the graph structure (as shown on the left) and construct _Virtual Edges_ (middle) that capture the propagation of information between nodes at multiple propagation steps. This allows our approach to zoom out of the local message passing and relates distant nodes that do not have a direct connection in the original graph. We then use the virtual edges to direct the transformer attention (shown on the right) and node representations.

The review comment to be evaluated is:

Strength
- The method is motivated by the failure of current architectures over a synthetic task. The proposed method brings significant improvement to it.
-  The method demonstrates empirical gains over some datasets, mostly on ones that requires capturing long-range interactions

Weaknesses
- Some important parts of the paper are confusing, mostly due to the ordering of the sections and the phrases:
    - It is not clear to me what "learn to combine information propagation over multiple different propagation steps in an
end-to-end manner" means. What are the "multiple different propagation steps"?
    - The use of the phrase "virtual edge" is somewhat confusing. Based on the later formulas, it seems these virtual edges are essentially generated features between all pair of nodes (based on 4.1), and replace the original edges/edge features. But the phrase itself and Figure 1 makes it seem like the virtual edges are added to the original set of edges.
- Some of the statements seem to be over-claimed:
    - "However, given the arbitrary structure of graphs, incorporating the input into the Transformer remains a challenging aspect in designing GTs, and so far, there has been no universal solution. We propose a simple architecture for incorporating structural data into the Transformer, Graph Diffuser (GD)..." It should be clarified that a lot of the current graph transformers designs are expressive enough incorporate arbitrary structure of graphs (e.g. Graphormer, GRAPHGPS, etc) and showed strong results. The authors should narrow down the claim to improving the modeling of long-range interaction.
    - "this work is the first Graph Transformer to [...] learn to combine information propagation over multiple different propagation steps in an end-to-end manner." It is not particularly clear to me what this statement means, but seems this is referring to feeding stacked random walk matrices through feed-forward neural networks in 4.1.2. I believe this has already been mentioned in GRAPHGPS and GraphiT (i.e. Relative PE using random walks).
- The first step of learning a dense adjacency matrix could be very expensive for large graphs.
- The empirical gain on ZINC and most of the OGB datasets seem quite small compared to the variance.

The evidence to be evaluated is:Gilmer et al. (2017) add "virtual edges" to shorten long distances, and Scarselli et al. (2008) add "supersource nodes". None of these works, however, integrate such virtual nodes or edges into the Transformer. Another line of work uses the attention mechanism Velickovic et al. (2018); Brody et al. (2022) to dynamically propagate information in the graph rather than use the original adjacency matrix or Laplacian.

Diffusion and other distance measures were used before to increase the expressivity of MP-GNNs Li et al. (2020), replace the local message passing scheme( Atwood and Towsley (2016); Wu et al. (2019); Klicpera et al. (2019, 2018)) or modulate the Transformers attention Mialon et al. (2021).

To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

First, GD embeds the structural relations between distant nodes in what we refer to as Virtual Edges. Then, the Transformer processes the nodes while using the virtual edges to direct the computation at different layers. _Virtual Edges_ are high-dimensional representations constructed between distant nodes in the graph. They contain rich information on the structural and positional relationships. To consider relations between distant nodes, the first step is broadening the receptive field on which the architecture operates.

We stack different powers of \(A\) into a 3 dimensional tensor \(E\in R^{n\times n\times k}\) \[E=[I|A|A^{2}|..|A^{k}] \tag{1}\] where \(k\) is the number of stacks and \(|\) denotes stacking matrices along the 3ed dimension and \(I\) is the identity matrix.

Xu et al. (2018), however, considering multiple powers of the matrix at once, we zoom out of the local Message Passing paradigm and discover information that GNNs may not detect. Virtual Edges contain structural information such as if nodes are on an odd length cycle, can distinguish many non-isomorphic graphs, and cover many distance measures such as shortest distance and generalized PageRank Li et al. (2020). In order to mix information between different propagation steps and extract meaningful structural relations, each stack is processed by a fully connected edge-wise feed-forward network.

Rather than using the original adjacency matrix in equation 1, we found it beneficial to learn a new adjacency matrix using the node and edge features. \[\hat{A_{ij}}=ReLU(BN([x_{i};x_{j},e_{ij}]W_{1}))W_{2} \tag{4}\] \[A_{ij}=normalize(\sigma(\hat{A_{ij}})) \tag{5}\] Where \(;\) means contacting vectors, \(\sigma\) is the sigmoid function and normalize means \(l1\) row normalization.

This aligns with existing approaches such as GAT (Velickovic et al. (2018); Brody et al. (2022)), which find it beneficial to dynamically learn a new adjacency matrix. Most Graph Transformers use the graph structure to either alter the node's representations or to affect the attention mechanism itself.

In the input layer, Self-Virtual Edges are added to the node representation as positional encoding, and at each attention layer, the Virtual Edges are reduced to an attention matrix that is combined with the standard dot-product attention. At each of the Transformer attention layers, we linearly project each virtual edge \(E_{ij}\) separately to get the positional attention score between \(i\) and \(j\). \[\hat{Att}_{ij}^{Position}=E_{ij}W_{p} \tag{6}\]We then combine the positional attention and the content(dot-product) attention scores and apply row-wise normalization. \[\hat{Att}^{Content}_{ij}=\frac{QK^{T}}{\sqrt{d}}_{ij} \tag{7}\] \[Att=normalize(exp(\hat{Att}^{Content})\odot\sigma(\hat{Att}^{Position})) \tag{8}\] Where \(\sigma\) is the sigmoid function and \(\odot\) means element-wise multiplication. In multi-head attention with \(h\) heads, the projection in equation 6 is done \(h\) times. equation 8 can be viewed as scaling the content attention coefficients based on the positional attention. As we will see in the next section, separating the attention based on content and positions and then combining them seems to assist with learning meaningful connections in the data and provides a natural visualization mechanism. Self-Virtual-Edges, \(E_{ii}\), are Virtual Edges between a node to itself. They contain important structural information, such as whether the node is part of an odd length cycle and the degrees of adjacent nodes. We project Self-Virtual-Edges to the node's representation dimensionality and add them as positional encoding.

Self-walks were shown (Dwivedi et al. (2022)) to be an effective positional encoding. Adding such features to the node representations allows the Transformer to use structural information in all of its layers, including the feed-forward network.

Looking at Figure 1(b), we can see the model learns to use the different attention mechanisms in an intuitive way, with content attention detecting nodes in the same color and position attention detecting nodes in the same row or column.

Using a controlled example, we demonstrated its effectiveness in modeling long-range interactions while providing better interpretability.

Another area of interest is amplifying virtual edges, for example, by considering paths between nodes rather than just information propagation.

Input: Figure 1: Illustration of our positional attention, focusing on node B. Information propagation from multiple propagation steps is combined to create Virtual Edges(colored) between distant nodes, which then direct the Transformer’s attention in each layer.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim about the approach overview and virtual edges based on the provided evidence and review comments:

Summary: The claim provides a high-level description of the approach but requires some qualification. While the evidence supports the basic mechanism of virtual edge construction and transformer attention direction, the reviewer points out some important clarification needs around terminology and novelty claims. The figure and mathematical formulations do support the general workflow described, but the claim would benefit from more precise language about how virtual edges relate to the original graph structure.

<score>3</score>

Justification:
1. The evidence supports the basic three-part workflow shown in Figure 1 (graph structure → virtual edges → transformer attention), but some aspects are oversimplified:

2. Areas where the claim is supported:
- The mathematical formulations (Equations 1-8) demonstrate how virtual edges are constructed and used
- Figure 1 illustrates the progression from graph to virtual edges to attention
- Evidence shows how virtual edges direct transformer attention through positional encoding

3. Areas of overstatement:
- The description implies virtual edges simply "capture propagation" but the evidence shows a more complex learned process (Equations 4-5)
- The claim doesn't acknowledge that virtual edges replace rather than supplement original edges (per reviewer)
- The relationship between virtual edges and the original graph structure needs more precise explanation

4. The reviewer's comments highlight legitimate concerns about clarity of terminology and potential confusion about how virtual edges relate to the original graph structure.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Figure 1 shows an overview of our approach. We start with the graph structure (as shown on the left) and construct _Virtual Edges_ (middle) that capture the propagation of information between nodes at multiple propagation steps. This allows our approach to zoom out of the local message passing and relates distant nodes that do not have a direct connection in the original graph. We then use the virtual edges to direct the transformer attention (shown on the right) and node representations.

The review comment to be evaluated is:

Strength:
1. The proposed method performs favorably on benchmarks datasets.
2. The method is effective but is also quite simple in term of implementing it.
3. The counter example of the 2-d grid histogram counting problem for graph transformer is interesting as all state of the art model fail to solve it.

Weakness:
1. The technical contribution is incremental. Graph diffusion (both in discrete and the ODE form) has been studied and applied in the domain of graph neutral network. While adding graph diffusion to the positional attention is new, it is a trivial extension. [1][2][3]
2. While it is surprising that most graph transformer could not solve the grid based counting problem, this problem itself in the context of graph can be easily solved by diffusion matrix itself if node color is expressed as node feature with one-hot encoding.

[1] https://papers.nips.cc/paper/2019/hash/23c894276a2c5a16470e6a31f4618d73-Abstract.html
[2] https://aclanthology.org/2021.emnlp-main.642.pdf
[3] https://openreview.net/forum?id=EMxu-dzvJk

The evidence to be evaluated is:Gilmer et al. (2017) add "virtual edges" to shorten long distances, and Scarselli et al. (2008) add "supersource nodes". None of these works, however, integrate such virtual nodes or edges into the Transformer. Another line of work uses the attention mechanism Velickovic et al. (2018); Brody et al. (2022) to dynamically propagate information in the graph rather than use the original adjacency matrix or Laplacian.

Diffusion and other distance measures were used before to increase the expressivity of MP-GNNs Li et al. (2020), replace the local message passing scheme( Atwood and Towsley (2016); Wu et al. (2019); Klicpera et al. (2019, 2018)) or modulate the Transformers attention Mialon et al. (2021).

To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

First, GD embeds the structural relations between distant nodes in what we refer to as Virtual Edges. Then, the Transformer processes the nodes while using the virtual edges to direct the computation at different layers. _Virtual Edges_ are high-dimensional representations constructed between distant nodes in the graph. They contain rich information on the structural and positional relationships. To consider relations between distant nodes, the first step is broadening the receptive field on which the architecture operates.

We stack different powers of \(A\) into a 3 dimensional tensor \(E\in R^{n\times n\times k}\) \[E=[I|A|A^{2}|..|A^{k}] \tag{1}\] where \(k\) is the number of stacks and \(|\) denotes stacking matrices along the 3ed dimension and \(I\) is the identity matrix.

Xu et al. (2018), however, considering multiple powers of the matrix at once, we zoom out of the local Message Passing paradigm and discover information that GNNs may not detect. Virtual Edges contain structural information such as if nodes are on an odd length cycle, can distinguish many non-isomorphic graphs, and cover many distance measures such as shortest distance and generalized PageRank Li et al. (2020). In order to mix information between different propagation steps and extract meaningful structural relations, each stack is processed by a fully connected edge-wise feed-forward network.

Rather than using the original adjacency matrix in equation 1, we found it beneficial to learn a new adjacency matrix using the node and edge features. \[\hat{A_{ij}}=ReLU(BN([x_{i};x_{j},e_{ij}]W_{1}))W_{2} \tag{4}\] \[A_{ij}=normalize(\sigma(\hat{A_{ij}})) \tag{5}\] Where \(;\) means contacting vectors, \(\sigma\) is the sigmoid function and normalize means \(l1\) row normalization.

This aligns with existing approaches such as GAT (Velickovic et al. (2018); Brody et al. (2022)), which find it beneficial to dynamically learn a new adjacency matrix. Most Graph Transformers use the graph structure to either alter the node's representations or to affect the attention mechanism itself.

In the input layer, Self-Virtual Edges are added to the node representation as positional encoding, and at each attention layer, the Virtual Edges are reduced to an attention matrix that is combined with the standard dot-product attention. At each of the Transformer attention layers, we linearly project each virtual edge \(E_{ij}\) separately to get the positional attention score between \(i\) and \(j\). \[\hat{Att}_{ij}^{Position}=E_{ij}W_{p} \tag{6}\]We then combine the positional attention and the content(dot-product) attention scores and apply row-wise normalization. \[\hat{Att}^{Content}_{ij}=\frac{QK^{T}}{\sqrt{d}}_{ij} \tag{7}\] \[Att=normalize(exp(\hat{Att}^{Content})\odot\sigma(\hat{Att}^{Position})) \tag{8}\] Where \(\sigma\) is the sigmoid function and \(\odot\) means element-wise multiplication. In multi-head attention with \(h\) heads, the projection in equation 6 is done \(h\) times. equation 8 can be viewed as scaling the content attention coefficients based on the positional attention. As we will see in the next section, separating the attention based on content and positions and then combining them seems to assist with learning meaningful connections in the data and provides a natural visualization mechanism. Self-Virtual-Edges, \(E_{ii}\), are Virtual Edges between a node to itself. They contain important structural information, such as whether the node is part of an odd length cycle and the degrees of adjacent nodes. We project Self-Virtual-Edges to the node's representation dimensionality and add them as positional encoding.

Self-walks were shown (Dwivedi et al. (2022)) to be an effective positional encoding. Adding such features to the node representations allows the Transformer to use structural information in all of its layers, including the feed-forward network.

Looking at Figure 1(b), we can see the model learns to use the different attention mechanisms in an intuitive way, with content attention detecting nodes in the same color and position attention detecting nodes in the same row or column.

Using a controlled example, we demonstrated its effectiveness in modeling long-range interactions while providing better interpretability.

Another area of interest is amplifying virtual edges, for example, by considering paths between nodes rather than just information propagation.

Input: Figure 1: Illustration of our positional attention, focusing on node B. Information propagation from multiple propagation steps is combined to create Virtual Edges(colored) between distant nodes, which then direct the Transformer’s attention in each layer.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that most aspects of the claim are well-supported by the provided evidence. The figure and text clearly demonstrate:
1. The graph structure transformation into virtual edges
2. The mathematical formulation of how virtual edges are constructed (Equations 1, 4-5)
3. How these edges direct transformer attention (Equations 6-8)

However, the claim makes a slightly stronger statement about "zooming out of local message passing" than what's fully substantiated. While the method does consider multiple propagation steps, the evidence acknowledges this builds on existing work on graph diffusion, as noted in the reviewer comments.

The approach itself is technically sound but incrementally novel, aligning with reviewer weakness #1. The visualization and implementation details support the core claims, but some statements about uniqueness may be moderately overstated.

<score>2</score>

The claim receives a 2 - slightly overstated but generally well-supported by evidence. The technical description accurately reflects the methodology shown in the figure and equations, with just minor overemphasis on novelty rather than incremental improvement.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Figure 1 shows an overview of our approach. We start with the graph structure (as shown on the left) and construct _Virtual Edges_ (middle) that capture the propagation of information between nodes at multiple propagation steps. This allows our approach to zoom out of the local message passing and relates distant nodes that do not have a direct connection in the original graph. We then use the virtual edges to direct the transformer attention (shown on the right) and node representations.

The review comment to be evaluated is:

Pros:
The experiments can effectively show the performance of the proposed method.

Cons:
1. There are existing works [1, 2] about incorporating graph diffusion in attention-based GNN that are not covered in this work. Besides, in the Related Work section, the subsection "Diffusion" might be revised to "Diffusion in Graph" to avoid any ambiguity with "Diffusion Models".
2. The improvement on many benchmarks is limited compared to the standard deviations.
3. The models in Table 1 are not well described in the main context. For example, the settings of "GNN →Transformer" and "GNN | Transformer+RWPE" can be briefly mentioned in 5.1.1. Moreover, the necessity of designing "GNN →Transformer" and "GNN | Transformer+RWPE" for comparison is not clear.
4. About the figure drawing: Figure 1 is a little hard to understand since it contains terms like "Edge-Wise FFN/projection" and unclear attention maps that are only described in the latter sections. The Transformer architecture in Figure 3 is not well drawn. The Positional Attention module is even partly outside the Multi-Head Attention module.

[1] Wang, Guangtao, Rex Ying, Jing Huang, and Jure Leskovec. "Multi-hop attention graph neural network." arXiv preprint arXiv:2009.14332 (2020).

[2] Liu, Yonghao, Renchu Guan, Fausto Giunchiglia, Yanchun Liang, and Xiaoyue Feng. "Deep attention diffusion graph neural networks for text classification." In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8142-8152. 2021.

The evidence to be evaluated is:Gilmer et al. (2017) add "virtual edges" to shorten long distances, and Scarselli et al. (2008) add "supersource nodes". None of these works, however, integrate such virtual nodes or edges into the Transformer. Another line of work uses the attention mechanism Velickovic et al. (2018); Brody et al. (2022) to dynamically propagate information in the graph rather than use the original adjacency matrix or Laplacian.

Diffusion and other distance measures were used before to increase the expressivity of MP-GNNs Li et al. (2020), replace the local message passing scheme( Atwood and Towsley (2016); Wu et al. (2019); Klicpera et al. (2019, 2018)) or modulate the Transformers attention Mialon et al. (2021).

To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

First, GD embeds the structural relations between distant nodes in what we refer to as Virtual Edges. Then, the Transformer processes the nodes while using the virtual edges to direct the computation at different layers. _Virtual Edges_ are high-dimensional representations constructed between distant nodes in the graph. They contain rich information on the structural and positional relationships. To consider relations between distant nodes, the first step is broadening the receptive field on which the architecture operates.

We stack different powers of \(A\) into a 3 dimensional tensor \(E\in R^{n\times n\times k}\) \[E=[I|A|A^{2}|..|A^{k}] \tag{1}\] where \(k\) is the number of stacks and \(|\) denotes stacking matrices along the 3ed dimension and \(I\) is the identity matrix.

Xu et al. (2018), however, considering multiple powers of the matrix at once, we zoom out of the local Message Passing paradigm and discover information that GNNs may not detect. Virtual Edges contain structural information such as if nodes are on an odd length cycle, can distinguish many non-isomorphic graphs, and cover many distance measures such as shortest distance and generalized PageRank Li et al. (2020). In order to mix information between different propagation steps and extract meaningful structural relations, each stack is processed by a fully connected edge-wise feed-forward network.

Rather than using the original adjacency matrix in equation 1, we found it beneficial to learn a new adjacency matrix using the node and edge features. \[\hat{A_{ij}}=ReLU(BN([x_{i};x_{j},e_{ij}]W_{1}))W_{2} \tag{4}\] \[A_{ij}=normalize(\sigma(\hat{A_{ij}})) \tag{5}\] Where \(;\) means contacting vectors, \(\sigma\) is the sigmoid function and normalize means \(l1\) row normalization.

This aligns with existing approaches such as GAT (Velickovic et al. (2018); Brody et al. (2022)), which find it beneficial to dynamically learn a new adjacency matrix. Most Graph Transformers use the graph structure to either alter the node's representations or to affect the attention mechanism itself.

In the input layer, Self-Virtual Edges are added to the node representation as positional encoding, and at each attention layer, the Virtual Edges are reduced to an attention matrix that is combined with the standard dot-product attention. At each of the Transformer attention layers, we linearly project each virtual edge \(E_{ij}\) separately to get the positional attention score between \(i\) and \(j\). \[\hat{Att}_{ij}^{Position}=E_{ij}W_{p} \tag{6}\]We then combine the positional attention and the content(dot-product) attention scores and apply row-wise normalization. \[\hat{Att}^{Content}_{ij}=\frac{QK^{T}}{\sqrt{d}}_{ij} \tag{7}\] \[Att=normalize(exp(\hat{Att}^{Content})\odot\sigma(\hat{Att}^{Position})) \tag{8}\] Where \(\sigma\) is the sigmoid function and \(\odot\) means element-wise multiplication. In multi-head attention with \(h\) heads, the projection in equation 6 is done \(h\) times. equation 8 can be viewed as scaling the content attention coefficients based on the positional attention. As we will see in the next section, separating the attention based on content and positions and then combining them seems to assist with learning meaningful connections in the data and provides a natural visualization mechanism. Self-Virtual-Edges, \(E_{ii}\), are Virtual Edges between a node to itself. They contain important structural information, such as whether the node is part of an odd length cycle and the degrees of adjacent nodes. We project Self-Virtual-Edges to the node's representation dimensionality and add them as positional encoding.

Self-walks were shown (Dwivedi et al. (2022)) to be an effective positional encoding. Adding such features to the node representations allows the Transformer to use structural information in all of its layers, including the feed-forward network.

Looking at Figure 1(b), we can see the model learns to use the different attention mechanisms in an intuitive way, with content attention detecting nodes in the same color and position attention detecting nodes in the same row or column.

Using a controlled example, we demonstrated its effectiveness in modeling long-range interactions while providing better interpretability.

Another area of interest is amplifying virtual edges, for example, by considering paths between nodes rather than just information propagation.

Input: Figure 1: Illustration of our positional attention, focusing on node B. Information propagation from multiple propagation steps is combined to create Virtual Edges(colored) between distant nodes, which then direct the Transformer’s attention in each layer.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim is slightly overstated, though generally supported by the evidence. While the overall approach of using virtual edges to capture multi-hop information propagation is well-documented in the evidence (equations 1-8), there are a few areas of concern:

1. The figure (Fig 1) that the claim references is described as "hard to understand" in the review comments, with terms that are only explained later. This undermines the claim's assertion about showing a clear overview.

2. The paper fails to acknowledge prior work on incorporating graph diffusion in attention-based GNNs, as noted in the review comments.

3. The evidence does support the technical aspects of virtual edge construction and their use in transformer attention, but the clarity and accessibility of the visual explanation is overstated.

<score>3</score>

Justification:
- The core technical claims about virtual edge construction and usage are well-supported by detailed equations and methodology
- The review comments highlight important limitations in presentation and novelty
- The claim overstates the clarity and comprehensiveness of Figure 1 as an overview
- Some prior related work is not adequately acknowledged
- The fundamental technical contribution appears sound but is presented more definitively than warranted


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Figure 1 shows an overview of our approach. We start with the graph structure (as shown on the left) and construct _Virtual Edges_ (middle) that capture the propagation of information between nodes at multiple propagation steps. This allows our approach to zoom out of the local message passing and relates distant nodes that do not have a direct connection in the original graph. We then use the virtual edges to direct the transformer attention (shown on the right) and node representations.

The review comment to be evaluated is:

Strength:
1. The paper presents two novel design for a graph transformer, the graph diffuser and context attention.
2. The paper provides many experiments in different datasets to demonstrate the performance of their approach.
3. The detail description of the experimental setup is also useful in understanding the benchmarks.

Weakness:
1. The authors should add more literature review. By googling "graph transformer", there is a survey [1] on arXiv that mentioned various methods regarding graph transformer. But some of those methods are not reviewed by the authors. Also, the experiments should include more baselines.
2. When use weighted adjacency, the model does not leverage any information from original adjacency.
3. More ablation study are needed to reveal the importance of each component. For example, context attention only, context attention + weighted adj, context attention + diffuser + weighted adj, and so on.

[1] Min, E., Chen, R., Bian, Y., Xu, T., Zhao, K., Huang, W., ... & Rong, Y. (2022). Transformer for Graphs: An Overview from Architecture Perspective. arXiv preprint arXiv:2202.08455. (https://arxiv.org/abs/2202.08455)

The evidence to be evaluated is:Gilmer et al. (2017) add "virtual edges" to shorten long distances, and Scarselli et al. (2008) add "supersource nodes". None of these works, however, integrate such virtual nodes or edges into the Transformer. Another line of work uses the attention mechanism Velickovic et al. (2018); Brody et al. (2022) to dynamically propagate information in the graph rather than use the original adjacency matrix or Laplacian.

Diffusion and other distance measures were used before to increase the expressivity of MP-GNNs Li et al. (2020), replace the local message passing scheme( Atwood and Towsley (2016); Wu et al. (2019); Klicpera et al. (2019, 2018)) or modulate the Transformers attention Mialon et al. (2021).

To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

First, GD embeds the structural relations between distant nodes in what we refer to as Virtual Edges. Then, the Transformer processes the nodes while using the virtual edges to direct the computation at different layers. _Virtual Edges_ are high-dimensional representations constructed between distant nodes in the graph. They contain rich information on the structural and positional relationships. To consider relations between distant nodes, the first step is broadening the receptive field on which the architecture operates.

We stack different powers of \(A\) into a 3 dimensional tensor \(E\in R^{n\times n\times k}\) \[E=[I|A|A^{2}|..|A^{k}] \tag{1}\] where \(k\) is the number of stacks and \(|\) denotes stacking matrices along the 3ed dimension and \(I\) is the identity matrix.

Xu et al. (2018), however, considering multiple powers of the matrix at once, we zoom out of the local Message Passing paradigm and discover information that GNNs may not detect. Virtual Edges contain structural information such as if nodes are on an odd length cycle, can distinguish many non-isomorphic graphs, and cover many distance measures such as shortest distance and generalized PageRank Li et al. (2020). In order to mix information between different propagation steps and extract meaningful structural relations, each stack is processed by a fully connected edge-wise feed-forward network.

Rather than using the original adjacency matrix in equation 1, we found it beneficial to learn a new adjacency matrix using the node and edge features. \[\hat{A_{ij}}=ReLU(BN([x_{i};x_{j},e_{ij}]W_{1}))W_{2} \tag{4}\] \[A_{ij}=normalize(\sigma(\hat{A_{ij}})) \tag{5}\] Where \(;\) means contacting vectors, \(\sigma\) is the sigmoid function and normalize means \(l1\) row normalization.

This aligns with existing approaches such as GAT (Velickovic et al. (2018); Brody et al. (2022)), which find it beneficial to dynamically learn a new adjacency matrix. Most Graph Transformers use the graph structure to either alter the node's representations or to affect the attention mechanism itself.

In the input layer, Self-Virtual Edges are added to the node representation as positional encoding, and at each attention layer, the Virtual Edges are reduced to an attention matrix that is combined with the standard dot-product attention. At each of the Transformer attention layers, we linearly project each virtual edge \(E_{ij}\) separately to get the positional attention score between \(i\) and \(j\). \[\hat{Att}_{ij}^{Position}=E_{ij}W_{p} \tag{6}\]We then combine the positional attention and the content(dot-product) attention scores and apply row-wise normalization. \[\hat{Att}^{Content}_{ij}=\frac{QK^{T}}{\sqrt{d}}_{ij} \tag{7}\] \[Att=normalize(exp(\hat{Att}^{Content})\odot\sigma(\hat{Att}^{Position})) \tag{8}\] Where \(\sigma\) is the sigmoid function and \(\odot\) means element-wise multiplication. In multi-head attention with \(h\) heads, the projection in equation 6 is done \(h\) times. equation 8 can be viewed as scaling the content attention coefficients based on the positional attention. As we will see in the next section, separating the attention based on content and positions and then combining them seems to assist with learning meaningful connections in the data and provides a natural visualization mechanism. Self-Virtual-Edges, \(E_{ii}\), are Virtual Edges between a node to itself. They contain important structural information, such as whether the node is part of an odd length cycle and the degrees of adjacent nodes. We project Self-Virtual-Edges to the node's representation dimensionality and add them as positional encoding.

Self-walks were shown (Dwivedi et al. (2022)) to be an effective positional encoding. Adding such features to the node representations allows the Transformer to use structural information in all of its layers, including the feed-forward network.

Looking at Figure 1(b), we can see the model learns to use the different attention mechanisms in an intuitive way, with content attention detecting nodes in the same color and position attention detecting nodes in the same row or column.

Using a controlled example, we demonstrated its effectiveness in modeling long-range interactions while providing better interpretability.

Another area of interest is amplifying virtual edges, for example, by considering paths between nodes rather than just information propagation.

Input: Figure 1: Illustration of our positional attention, focusing on node B. Information propagation from multiple propagation steps is combined to create Virtual Edges(colored) between distant nodes, which then direct the Transformer’s attention in each layer.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the claim generally aligns well with the provided technical details and figures, with only minor overstatement. The evidence clearly shows:

1. The graph structure and virtual edge construction process is demonstrated in Figure 1
2. The mathematical foundations for virtual edges (Equations 1-8) support the claim's description
3. The transition from graph structure to virtual edges to transformer attention is well-documented
4. The integration of content and positional attention is substantiated

However, the claim could be more precise about the "zooming out" metaphor, as the evidence shows it's more about mathematical combinations of adjacency matrices and attention mechanisms rather than a literal zooming process.

<score>2</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Finally, we demonstrate the effectiveness of our approach on eight benchmarks spanning multiple domains by showing it outperforms state-of-the-art with no hyperparameter tuning.

The evidence to be evaluated is:To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

However, as we show in Section 5.1.2, it defeats all existing GNN and Graph Transformer techniques, while Graph Diffuser succeeds.

Throughout our experiments, we take an existing Graph Transformer architecture and use it "as is" as our Transformer module without doing any hyper-parameter search. That is, we take a graph Transformer with the same hyperparameters used by the original work and only add our positional attention and encoding.2 For 5 of the datasets, ZINC, OGBG-{molpcba, ppa,code2}, PCQM-Contact and PCQM4Mv2, we use the very competitive GPS Rampasek et al. (2022) as the Transformer model. For the LRGB benchmarks, Peptides-func and Peptides-struct, there are no official GPS hyperparameters, and we use a Transformer with the authors' baseline hyperparameters and a CLS token added to the input.

We evaluate our model on \(8\) datasets overall and compare our results with those of popular MPGNNs, Graph Transformers and other recent state-of-the-art models. Graph Diffuser exceeds SOTA in 6 of them and achieves very competitive results in the rest.

Table 2 shows the results, where we exceed SOTA results by a substitutional margin.

We evaluate GD on three graph-level prediction tasks from different domains and report the results at Table 3

Here we rank the second highest and the first among all GNN or Transformer architectures.

The previously highest-ranking model is GPS, and adding our positional attention and encoding to it proves to be an efficient strategy that reaches a new state-of-the-art.

Since positional encoding was not found to be significant by the previous three highest ranking models in this dataset, and considering the large improvement of our model over GPS, it seems that positional attention brings significant benefits in this benchmark, which allows us to reach a new state-of-the-art.

We are the first, other than the author's baselines, to evaluate our model on these datasets and currently rank first in all of them.

We then showed that this translates to real-world problems by evaluating GD on eight benchmarks from multiple domains. With minimal hyperparameter tuning, Graph Diffuser achieves a new state-of-the-art on most datasets and reaches very competitive results on the rest.

We Conducted a simplified experiment to see the effects of GD when used as an out-of-the-box addition to different Transformer compositions, with no hyperparameter tuning. \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **MAE \(\downarrow\)** \\ \hline GCN Kipf \& Welling (2017) & 0.367 \(\pm\) 0.011 \\ GIN Xu et al. (2019a) & 0.526 \(\pm\) 0.051 \\ GAT Velicković et al. (2018) & 0.384 \(\pm\) 0.007 \\ GatedGCN Bresson \& Laurent (2017); Dwivedi et al. (2020) & 0.282 \(\pm\) 0.015 \\ GatedGCN-LSP Dwivedi et al. (2022a) & 0.090 \(\pm\) 0.001 \\ CRaWI Toenshoff et al. (2021) & 0.085 \(\pm\) 0.004 \\ GIN-Ak Zhao et al. (2022) & 0.080 \(\pm\) 0.001 \\ SAN Kreuzer et al. (2021) & 0.139 \(\pm\) 0.006 \\ Graphormer Ying et al. (2021) & 0.122 \(\pm\) 0.006 \\ K-Sunggraph SAT Chen et al. (2022) & 0.094 \(\pm\) 0.008 \\ GPS Rampásek et al. (2022) & 0.070 \(\pm\) 0.004 \\ \hline Graph Diffuser(ours) & **0.0683 \(\pm\) 0.002** \\ \hline \hline \end{tabular}
\end{table}
Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input: Table 3: Test results in OGB graph-level benchmarks Hu et al. (2020). Pre-trained or ensemble models are not included. Bold:first, Underlined:Second.

Input: Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the statement "outperforms state-of-the-art with no hyperparameter tuning" to be partially overstated. While the evidence shows Graph Diffuser achieves strong results, it does not consistently outperform SOTA across all eight benchmarks. The paper explicitly states they "exceed SOTA in 6 of them and achieve very competitive results in the rest." Additionally, while they claim "no hyperparameter tuning," they do use existing optimized hyperparameters from previous models like GPS. The results are impressive but don't fully support the absolute nature of the claim.

<score>3</score>

Justification:
1. The claim overstates by saying it "outperforms state-of-the-art" without qualifying that this is true for 6 out of 8 benchmarks, not all.
2. Tables 2 and 3 show strong but not universal SOTA performance:
   - ZINC dataset: Marginal improvement over GPS (0.0683 vs 0.070)
   - In ogbg-molpcba: Achieves 0.2931, slightly underperforming CRaWl's 0.2986
3. The "no hyperparameter tuning" claim is somewhat misleading since they utilize pre-optimized hyperparameters from existing models rather than starting from scratch.
4. The evidence does support strong performance and efficient adaptation of existing architectures, but not complete SOTA dominance as implied.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Finally, we demonstrate the effectiveness of our approach on eight benchmarks spanning multiple domains by showing it outperforms state-of-the-art with no hyperparameter tuning.

The review comment to be evaluated is:

Strength
- The method is motivated by the failure of current architectures over a synthetic task. The proposed method brings significant improvement to it.
-  The method demonstrates empirical gains over some datasets, mostly on ones that requires capturing long-range interactions

Weaknesses
- Some important parts of the paper are confusing, mostly due to the ordering of the sections and the phrases:
    - It is not clear to me what "learn to combine information propagation over multiple different propagation steps in an
end-to-end manner" means. What are the "multiple different propagation steps"?
    - The use of the phrase "virtual edge" is somewhat confusing. Based on the later formulas, it seems these virtual edges are essentially generated features between all pair of nodes (based on 4.1), and replace the original edges/edge features. But the phrase itself and Figure 1 makes it seem like the virtual edges are added to the original set of edges.
- Some of the statements seem to be over-claimed:
    - "However, given the arbitrary structure of graphs, incorporating the input into the Transformer remains a challenging aspect in designing GTs, and so far, there has been no universal solution. We propose a simple architecture for incorporating structural data into the Transformer, Graph Diffuser (GD)..." It should be clarified that a lot of the current graph transformers designs are expressive enough incorporate arbitrary structure of graphs (e.g. Graphormer, GRAPHGPS, etc) and showed strong results. The authors should narrow down the claim to improving the modeling of long-range interaction.
    - "this work is the first Graph Transformer to [...] learn to combine information propagation over multiple different propagation steps in an end-to-end manner." It is not particularly clear to me what this statement means, but seems this is referring to feeding stacked random walk matrices through feed-forward neural networks in 4.1.2. I believe this has already been mentioned in GRAPHGPS and GraphiT (i.e. Relative PE using random walks).
- The first step of learning a dense adjacency matrix could be very expensive for large graphs.
- The empirical gain on ZINC and most of the OGB datasets seem quite small compared to the variance.

The evidence to be evaluated is:To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

However, as we show in Section 5.1.2, it defeats all existing GNN and Graph Transformer techniques, while Graph Diffuser succeeds.

Throughout our experiments, we take an existing Graph Transformer architecture and use it "as is" as our Transformer module without doing any hyper-parameter search. That is, we take a graph Transformer with the same hyperparameters used by the original work and only add our positional attention and encoding.2 For 5 of the datasets, ZINC, OGBG-{molpcba, ppa,code2}, PCQM-Contact and PCQM4Mv2, we use the very competitive GPS Rampasek et al. (2022) as the Transformer model. For the LRGB benchmarks, Peptides-func and Peptides-struct, there are no official GPS hyperparameters, and we use a Transformer with the authors' baseline hyperparameters and a CLS token added to the input.

We evaluate our model on \(8\) datasets overall and compare our results with those of popular MPGNNs, Graph Transformers and other recent state-of-the-art models. Graph Diffuser exceeds SOTA in 6 of them and achieves very competitive results in the rest.

Table 2 shows the results, where we exceed SOTA results by a substitutional margin.

We evaluate GD on three graph-level prediction tasks from different domains and report the results at Table 3

Here we rank the second highest and the first among all GNN or Transformer architectures.

The previously highest-ranking model is GPS, and adding our positional attention and encoding to it proves to be an efficient strategy that reaches a new state-of-the-art.

Since positional encoding was not found to be significant by the previous three highest ranking models in this dataset, and considering the large improvement of our model over GPS, it seems that positional attention brings significant benefits in this benchmark, which allows us to reach a new state-of-the-art.

We are the first, other than the author's baselines, to evaluate our model on these datasets and currently rank first in all of them.

We then showed that this translates to real-world problems by evaluating GD on eight benchmarks from multiple domains. With minimal hyperparameter tuning, Graph Diffuser achieves a new state-of-the-art on most datasets and reaches very competitive results on the rest.

We Conducted a simplified experiment to see the effects of GD when used as an out-of-the-box addition to different Transformer compositions, with no hyperparameter tuning. \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **MAE \(\downarrow\)** \\ \hline GCN Kipf \& Welling (2017) & 0.367 \(\pm\) 0.011 \\ GIN Xu et al. (2019a) & 0.526 \(\pm\) 0.051 \\ GAT Velicković et al. (2018) & 0.384 \(\pm\) 0.007 \\ GatedGCN Bresson \& Laurent (2017); Dwivedi et al. (2020) & 0.282 \(\pm\) 0.015 \\ GatedGCN-LSP Dwivedi et al. (2022a) & 0.090 \(\pm\) 0.001 \\ CRaWI Toenshoff et al. (2021) & 0.085 \(\pm\) 0.004 \\ GIN-Ak Zhao et al. (2022) & 0.080 \(\pm\) 0.001 \\ SAN Kreuzer et al. (2021) & 0.139 \(\pm\) 0.006 \\ Graphormer Ying et al. (2021) & 0.122 \(\pm\) 0.006 \\ K-Sunggraph SAT Chen et al. (2022) & 0.094 \(\pm\) 0.008 \\ GPS Rampásek et al. (2022) & 0.070 \(\pm\) 0.004 \\ \hline Graph Diffuser(ours) & **0.0683 \(\pm\) 0.002** \\ \hline \hline \end{tabular}
\end{table}
Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input: Table 3: Test results in OGB graph-level benchmarks Hu et al. (2020). Pre-trained or ensemble models are not included. Bold:first, Underlined:Second.

Input: Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and reviewer comments, this claim appears overstated. While the method does show improvements on several benchmarks, there are several key issues:

1. The claim states "outperforms state-of-the-art" on "eight benchmarks" but:
- The improvements on ZINC and some OGB datasets are within variance margins
- Reviewer notes that "empirical gain on ZINC and most of the OGB datasets seem quite small compared to the variance"

2. The claim states "no hyperparameter tuning" but the evidence shows:
- They used existing hyperparameters from previous works
- For some datasets (LRGB benchmarks), they had to use baseline hyperparameters with modifications

3. Some datasets (Peptides) had no previous comparisons except author baselines, weakening the "state-of-the-art" claim.

<score>4</score>

The claim would be better stated by specifying the exact improvements and acknowledging the limitations in performance gains and dataset comparisons.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Finally, we demonstrate the effectiveness of our approach on eight benchmarks spanning multiple domains by showing it outperforms state-of-the-art with no hyperparameter tuning.

The review comment to be evaluated is:

Strength:
1. The proposed method performs favorably on benchmarks datasets.
2. The method is effective but is also quite simple in term of implementing it.
3. The counter example of the 2-d grid histogram counting problem for graph transformer is interesting as all state of the art model fail to solve it.

Weakness:
1. The technical contribution is incremental. Graph diffusion (both in discrete and the ODE form) has been studied and applied in the domain of graph neutral network. While adding graph diffusion to the positional attention is new, it is a trivial extension. [1][2][3]
2. While it is surprising that most graph transformer could not solve the grid based counting problem, this problem itself in the context of graph can be easily solved by diffusion matrix itself if node color is expressed as node feature with one-hot encoding.

[1] https://papers.nips.cc/paper/2019/hash/23c894276a2c5a16470e6a31f4618d73-Abstract.html
[2] https://aclanthology.org/2021.emnlp-main.642.pdf
[3] https://openreview.net/forum?id=EMxu-dzvJk

The evidence to be evaluated is:To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

However, as we show in Section 5.1.2, it defeats all existing GNN and Graph Transformer techniques, while Graph Diffuser succeeds.

Throughout our experiments, we take an existing Graph Transformer architecture and use it "as is" as our Transformer module without doing any hyper-parameter search. That is, we take a graph Transformer with the same hyperparameters used by the original work and only add our positional attention and encoding.2 For 5 of the datasets, ZINC, OGBG-{molpcba, ppa,code2}, PCQM-Contact and PCQM4Mv2, we use the very competitive GPS Rampasek et al. (2022) as the Transformer model. For the LRGB benchmarks, Peptides-func and Peptides-struct, there are no official GPS hyperparameters, and we use a Transformer with the authors' baseline hyperparameters and a CLS token added to the input.

We evaluate our model on \(8\) datasets overall and compare our results with those of popular MPGNNs, Graph Transformers and other recent state-of-the-art models. Graph Diffuser exceeds SOTA in 6 of them and achieves very competitive results in the rest.

Table 2 shows the results, where we exceed SOTA results by a substitutional margin.

We evaluate GD on three graph-level prediction tasks from different domains and report the results at Table 3

Here we rank the second highest and the first among all GNN or Transformer architectures.

The previously highest-ranking model is GPS, and adding our positional attention and encoding to it proves to be an efficient strategy that reaches a new state-of-the-art.

Since positional encoding was not found to be significant by the previous three highest ranking models in this dataset, and considering the large improvement of our model over GPS, it seems that positional attention brings significant benefits in this benchmark, which allows us to reach a new state-of-the-art.

We are the first, other than the author's baselines, to evaluate our model on these datasets and currently rank first in all of them.

We then showed that this translates to real-world problems by evaluating GD on eight benchmarks from multiple domains. With minimal hyperparameter tuning, Graph Diffuser achieves a new state-of-the-art on most datasets and reaches very competitive results on the rest.

We Conducted a simplified experiment to see the effects of GD when used as an out-of-the-box addition to different Transformer compositions, with no hyperparameter tuning. \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **MAE \(\downarrow\)** \\ \hline GCN Kipf \& Welling (2017) & 0.367 \(\pm\) 0.011 \\ GIN Xu et al. (2019a) & 0.526 \(\pm\) 0.051 \\ GAT Velicković et al. (2018) & 0.384 \(\pm\) 0.007 \\ GatedGCN Bresson \& Laurent (2017); Dwivedi et al. (2020) & 0.282 \(\pm\) 0.015 \\ GatedGCN-LSP Dwivedi et al. (2022a) & 0.090 \(\pm\) 0.001 \\ CRaWI Toenshoff et al. (2021) & 0.085 \(\pm\) 0.004 \\ GIN-Ak Zhao et al. (2022) & 0.080 \(\pm\) 0.001 \\ SAN Kreuzer et al. (2021) & 0.139 \(\pm\) 0.006 \\ Graphormer Ying et al. (2021) & 0.122 \(\pm\) 0.006 \\ K-Sunggraph SAT Chen et al. (2022) & 0.094 \(\pm\) 0.008 \\ GPS Rampásek et al. (2022) & 0.070 \(\pm\) 0.004 \\ \hline Graph Diffuser(ours) & **0.0683 \(\pm\) 0.002** \\ \hline \hline \end{tabular}
\end{table}
Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input: Table 3: Test results in OGB graph-level benchmarks Hu et al. (2020). Pre-trained or ensemble models are not included. Bold:first, Underlined:Second.

Input: Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the statement is moderately overstated. While the evidence shows strong performance across multiple benchmarks, several key points contradict or qualify the claim:

1. The claim states "outperforms state-of-the-art" without qualification, but the evidence shows they:
- Exceeded SOTA in 6 out of 8 datasets (not all)
- Ranked second in some cases
- Were the first to evaluate on some datasets besides baselines (implying limited comparison)

2. Regarding "no hyperparameter tuning":
- They did use existing optimized hyperparameters from previous works
- They specifically chose different transformer architectures for different datasets

The review comments also highlight that some technical contributions were incremental rather than breakthrough innovations.

While the method shows strong performance, more measured language acknowledging these limitations would better reflect the evidence.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Finally, we demonstrate the effectiveness of our approach on eight benchmarks spanning multiple domains by showing it outperforms state-of-the-art with no hyperparameter tuning.

The review comment to be evaluated is:

Pros:
The experiments can effectively show the performance of the proposed method.

Cons:
1. There are existing works [1, 2] about incorporating graph diffusion in attention-based GNN that are not covered in this work. Besides, in the Related Work section, the subsection "Diffusion" might be revised to "Diffusion in Graph" to avoid any ambiguity with "Diffusion Models".
2. The improvement on many benchmarks is limited compared to the standard deviations.
3. The models in Table 1 are not well described in the main context. For example, the settings of "GNN →Transformer" and "GNN | Transformer+RWPE" can be briefly mentioned in 5.1.1. Moreover, the necessity of designing "GNN →Transformer" and "GNN | Transformer+RWPE" for comparison is not clear.
4. About the figure drawing: Figure 1 is a little hard to understand since it contains terms like "Edge-Wise FFN/projection" and unclear attention maps that are only described in the latter sections. The Transformer architecture in Figure 3 is not well drawn. The Positional Attention module is even partly outside the Multi-Head Attention module.

[1] Wang, Guangtao, Rex Ying, Jing Huang, and Jure Leskovec. "Multi-hop attention graph neural network." arXiv preprint arXiv:2009.14332 (2020).

[2] Liu, Yonghao, Renchu Guan, Fausto Giunchiglia, Yanchun Liang, and Xiaoyue Feng. "Deep attention diffusion graph neural networks for text classification." In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8142-8152. 2021.

The evidence to be evaluated is:To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

However, as we show in Section 5.1.2, it defeats all existing GNN and Graph Transformer techniques, while Graph Diffuser succeeds.

Throughout our experiments, we take an existing Graph Transformer architecture and use it "as is" as our Transformer module without doing any hyper-parameter search. That is, we take a graph Transformer with the same hyperparameters used by the original work and only add our positional attention and encoding.2 For 5 of the datasets, ZINC, OGBG-{molpcba, ppa,code2}, PCQM-Contact and PCQM4Mv2, we use the very competitive GPS Rampasek et al. (2022) as the Transformer model. For the LRGB benchmarks, Peptides-func and Peptides-struct, there are no official GPS hyperparameters, and we use a Transformer with the authors' baseline hyperparameters and a CLS token added to the input.

We evaluate our model on \(8\) datasets overall and compare our results with those of popular MPGNNs, Graph Transformers and other recent state-of-the-art models. Graph Diffuser exceeds SOTA in 6 of them and achieves very competitive results in the rest.

Table 2 shows the results, where we exceed SOTA results by a substitutional margin.

We evaluate GD on three graph-level prediction tasks from different domains and report the results at Table 3

Here we rank the second highest and the first among all GNN or Transformer architectures.

The previously highest-ranking model is GPS, and adding our positional attention and encoding to it proves to be an efficient strategy that reaches a new state-of-the-art.

Since positional encoding was not found to be significant by the previous three highest ranking models in this dataset, and considering the large improvement of our model over GPS, it seems that positional attention brings significant benefits in this benchmark, which allows us to reach a new state-of-the-art.

We are the first, other than the author's baselines, to evaluate our model on these datasets and currently rank first in all of them.

We then showed that this translates to real-world problems by evaluating GD on eight benchmarks from multiple domains. With minimal hyperparameter tuning, Graph Diffuser achieves a new state-of-the-art on most datasets and reaches very competitive results on the rest.

We Conducted a simplified experiment to see the effects of GD when used as an out-of-the-box addition to different Transformer compositions, with no hyperparameter tuning. \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **MAE \(\downarrow\)** \\ \hline GCN Kipf \& Welling (2017) & 0.367 \(\pm\) 0.011 \\ GIN Xu et al. (2019a) & 0.526 \(\pm\) 0.051 \\ GAT Velicković et al. (2018) & 0.384 \(\pm\) 0.007 \\ GatedGCN Bresson \& Laurent (2017); Dwivedi et al. (2020) & 0.282 \(\pm\) 0.015 \\ GatedGCN-LSP Dwivedi et al. (2022a) & 0.090 \(\pm\) 0.001 \\ CRaWI Toenshoff et al. (2021) & 0.085 \(\pm\) 0.004 \\ GIN-Ak Zhao et al. (2022) & 0.080 \(\pm\) 0.001 \\ SAN Kreuzer et al. (2021) & 0.139 \(\pm\) 0.006 \\ Graphormer Ying et al. (2021) & 0.122 \(\pm\) 0.006 \\ K-Sunggraph SAT Chen et al. (2022) & 0.094 \(\pm\) 0.008 \\ GPS Rampásek et al. (2022) & 0.070 \(\pm\) 0.004 \\ \hline Graph Diffuser(ours) & **0.0683 \(\pm\) 0.002** \\ \hline \hline \end{tabular}
\end{table}
Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input: Table 3: Test results in OGB graph-level benchmarks Hu et al. (2020). Pre-trained or ensemble models are not included. Bold:first, Underlined:Second.

Input: Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find several concerning aspects that suggest overstatement:

1. While the method shows improvements on some benchmarks, the review points out that "improvement on many benchmarks is limited compared to the standard deviations."

2. The claim states "outperforms state-of-the-art" generally, but the evidence shows this isn't true for all 8 benchmarks - they "exceed SOTA in 6 of them and achieves very competitive results in the rest."

3. The claim states "no hyperparameter tuning" but the evidence shows they used existing hyperparameters from previous works, which isn't the same as completely avoiding tuning.

4. Tables 2 and 3 show improvements that are often within error margins of competing methods.

5. The review highlights missing comparisons to relevant prior work in graph diffusion attention.

This represents a moderate overstatement of the actual results and their significance.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Finally, we demonstrate the effectiveness of our approach on eight benchmarks spanning multiple domains by showing it outperforms state-of-the-art with no hyperparameter tuning.

The review comment to be evaluated is:

Strength:
1. The paper presents two novel design for a graph transformer, the graph diffuser and context attention.
2. The paper provides many experiments in different datasets to demonstrate the performance of their approach.
3. The detail description of the experimental setup is also useful in understanding the benchmarks.

Weakness:
1. The authors should add more literature review. By googling "graph transformer", there is a survey [1] on arXiv that mentioned various methods regarding graph transformer. But some of those methods are not reviewed by the authors. Also, the experiments should include more baselines.
2. When use weighted adjacency, the model does not leverage any information from original adjacency.
3. More ablation study are needed to reveal the importance of each component. For example, context attention only, context attention + weighted adj, context attention + diffuser + weighted adj, and so on.

[1] Min, E., Chen, R., Bian, Y., Xu, T., Zhao, K., Huang, W., ... & Rong, Y. (2022). Transformer for Graphs: An Overview from Architecture Perspective. arXiv preprint arXiv:2202.08455. (https://arxiv.org/abs/2202.08455)

The evidence to be evaluated is:To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

However, as we show in Section 5.1.2, it defeats all existing GNN and Graph Transformer techniques, while Graph Diffuser succeeds.

Throughout our experiments, we take an existing Graph Transformer architecture and use it "as is" as our Transformer module without doing any hyper-parameter search. That is, we take a graph Transformer with the same hyperparameters used by the original work and only add our positional attention and encoding.2 For 5 of the datasets, ZINC, OGBG-{molpcba, ppa,code2}, PCQM-Contact and PCQM4Mv2, we use the very competitive GPS Rampasek et al. (2022) as the Transformer model. For the LRGB benchmarks, Peptides-func and Peptides-struct, there are no official GPS hyperparameters, and we use a Transformer with the authors' baseline hyperparameters and a CLS token added to the input.

We evaluate our model on \(8\) datasets overall and compare our results with those of popular MPGNNs, Graph Transformers and other recent state-of-the-art models. Graph Diffuser exceeds SOTA in 6 of them and achieves very competitive results in the rest.

Table 2 shows the results, where we exceed SOTA results by a substitutional margin.

We evaluate GD on three graph-level prediction tasks from different domains and report the results at Table 3

Here we rank the second highest and the first among all GNN or Transformer architectures.

The previously highest-ranking model is GPS, and adding our positional attention and encoding to it proves to be an efficient strategy that reaches a new state-of-the-art.

Since positional encoding was not found to be significant by the previous three highest ranking models in this dataset, and considering the large improvement of our model over GPS, it seems that positional attention brings significant benefits in this benchmark, which allows us to reach a new state-of-the-art.

We are the first, other than the author's baselines, to evaluate our model on these datasets and currently rank first in all of them.

We then showed that this translates to real-world problems by evaluating GD on eight benchmarks from multiple domains. With minimal hyperparameter tuning, Graph Diffuser achieves a new state-of-the-art on most datasets and reaches very competitive results on the rest.

We Conducted a simplified experiment to see the effects of GD when used as an out-of-the-box addition to different Transformer compositions, with no hyperparameter tuning. \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **MAE \(\downarrow\)** \\ \hline GCN Kipf \& Welling (2017) & 0.367 \(\pm\) 0.011 \\ GIN Xu et al. (2019a) & 0.526 \(\pm\) 0.051 \\ GAT Velicković et al. (2018) & 0.384 \(\pm\) 0.007 \\ GatedGCN Bresson \& Laurent (2017); Dwivedi et al. (2020) & 0.282 \(\pm\) 0.015 \\ GatedGCN-LSP Dwivedi et al. (2022a) & 0.090 \(\pm\) 0.001 \\ CRaWI Toenshoff et al. (2021) & 0.085 \(\pm\) 0.004 \\ GIN-Ak Zhao et al. (2022) & 0.080 \(\pm\) 0.001 \\ SAN Kreuzer et al. (2021) & 0.139 \(\pm\) 0.006 \\ Graphormer Ying et al. (2021) & 0.122 \(\pm\) 0.006 \\ K-Sunggraph SAT Chen et al. (2022) & 0.094 \(\pm\) 0.008 \\ GPS Rampásek et al. (2022) & 0.070 \(\pm\) 0.004 \\ \hline Graph Diffuser(ours) & **0.0683 \(\pm\) 0.002** \\ \hline \hline \end{tabular}
\end{table}
Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input: Table 3: Test results in OGB graph-level benchmarks Hu et al. (2020). Pre-trained or ensemble models are not included. Bold:first, Underlined:Second.

Input: Table 2: MAE for the ZINC dataset from Dwivedi et al. (2020b).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim somewhat overstated. While the evidence shows strong performance across multiple benchmarks, several key points contradict or qualify the claim:

1. The claim of "outperforms state-of-the-art" is only true for 6 out of 8 datasets, not all eight as implied
2. The authors note they achieve "very competitive results" in the remaining datasets, rather than outperforming SOTA
3. While they used existing hyperparameters without tuning, they did make architectural choices (like using GPS vs. other transformers) for different datasets
4. The review points out missing baselines and limited ablation studies that could better validate the broad effectiveness claim

The claim would be more accurate if it acknowledged the mixed results rather than suggesting universal SOTA performance.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We propose _Graph Diffuser_ (GD) to address this challenge.

The evidence to be evaluated is:In contrast, Graph Diffuser learns to combine multiple distances.

However, all of the works above use the original adjacency matrix or Laplacian to learn positional or relative encoding, unlike this work that learns the adjacency matrix using node and edge features.

To the best of our knowledge, this work is the first Graph Transformer to: (1) learn to construct a new adjacency matrix using node and edge features to generate positional or relative encoding, and (2) learn to combine information propagation over multiple different propagation steps in an end-to-end manner.

Our approach consists of 2 stages, illustrated in the left and right halves of Figure 3. First, GD embeds the structural relations between distant nodes in what we refer to as Virtual Edges. Then, the Transformer processes the nodes while using the virtual edges to direct the computation at different layers. _Virtual Edges_ are high-dimensional representations constructed between distant nodes in the graph. They contain rich information on the structural and positional relationships.

Graph Diffuser combines the 2 approaches. In the input layer, Self-Virtual Edges are added to the node representation as positional encoding, and at each attention layer, the Virtual Edges are reduced to an attention matrix that is combined with the standard dot-product attention. At each of the Transformer attention layers, we linearly project each virtual edge \(E_{ij}\) separately to get the positional attention score between \(i\) and \(j\).

As we will see in the next section, separating the attention based on content and positions and then combining them seems to assist with learning meaningful connections in the data and provides a natural visualization mechanism.

Finally, Graph Diffuser nearly solves the task entirely by combining both positional and content attention. Looking at Figure 1(b), we can see the model learns to use the different attention mechanisms in an intuitive way, with content attention detecting nodes in the same color and position attention detecting nodes in the same row or column.

In this work, we introduced a simple and effective architecture for machine learning on graphs, Graph Diffuser. Using a controlled example, we demonstrated its effectiveness in modeling long-range interactions while providing better interpretability. We then showed that this translates to real-world problems by evaluating GD on eight benchmarks from multiple domains. With minimal hyperparameter tuning, Graph Diffuser achieves a new state-of-the-art on most datasets and reaches very competitive results on the rest.

We describe Graph Diffuser in detail in the text and figures and mention all relevant hyperparameters to reproduce our experiments in the appendix. Moreover, we will release our code as open-source with clear instructions on how to reproduce all of our results, as well as how to extend our model and apply it to new datasets.

For Graph Difffuser, we found 3 layers sufficient.

We Conducted a simplified experiment to see the effects of GD when used as an out-of-the-box addition to different Transformer compositions, with no hyperparameter tuning.

For each variation, we evaluate the unmodified architecture with no positional embedding or positional attention, the architecture with our positional encoding and attention added, but not learning the adjacency matrix(+ _Graph Diffuser_), and one we learn a new weighted adjacency matrix(+ _Weighted Adjacency_), as described in Section 4.1.3.

Input: Figure 1: Illustration of our positional attention, focusing on node B. Information propagation from multiple propagation steps is combined to create Virtual Edges(colored) between distant nodes, which then direct the Transformer’s attention in each layer.

Input: Figure 3: An illustration of Graph Diffuser with 2 Transformer layers. Virtual Edges are created by combining multiple powers of the weighted adjacency matrix(left). Virtual Edges modulate each of the Transformers attention layers(right). Self-Virtual Edges are added to the Transformer input as positional encoding(bottom right)

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim "We propose Graph Diffuser (GD) to address this challenge" appears relatively well-stated, though somewhat understated. The evidence demonstrates that GD is indeed a novel approach with:
1. Clear technical innovation in learning adjacency matrices and combining multiple distances
2. Detailed architectural description with figures
3. Documented experimental validation
4. Open-source code commitment for reproducibility
5. Demonstrated effectiveness on benchmarks

The claim could have been more specific about GD's key innovations, but it avoids exaggeration and is supported by substantial technical evidence and experimental results. The modest phrasing actually understates the contribution somewhat.

<score>1</score>

Justification:
- The evidence shows GD is "the first Graph Transformer" with specific novel capabilities (learning adjacency matrix, combining multiple propagation steps)
- Detailed technical description and figures (Fig 1, 3) illustrate the architecture's components and operation
- Results demonstrate "new state-of-the-art on most datasets"
- Authors commit to reproducibility through code release and detailed documentation
- No overstated claims about capabilities - findings are presented with appropriate scope
- Experimental validation is described with specific parameters (e.g., "3 layers sufficient")
- Limitations and experimental conditions are clearly acknowledged