Title: Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions. Abstract: Most graph neural networks (GNNs) rely on the message passing paradigm to propagate node features and build interactions. Recent studies point out that different graph learning tasks require different ranges of interactions between nodes. In this work, we explore the capacity of GNNs to capture multi-order interactions between nodes, and the order represents the complexity of the context where interactions take place. We study two standard graph construction methods, namely, \emph{K-nearest neighbor} (KNN) graphs and \emph{fully-connected} (FC) graphs, and concentrate on scientific problems in the 3D Euclidean space. 
We demonstrate that the inductive bias introduced by KNN-graphs and FC-graphs prevents GNNs from learning interactions of the most appropriate complexity.  We found that such a phenomenon is broadly shared by several GNNs for diverse graph learning tasks, so we name it a \emph{representation bottleneck}. To overcome that, we propose a novel graph rewiring approach based on interaction strengths of various orders to adjust the receptive fields of each node dynamically. 
Extensive experiments in molecular property prediction and dynamic system forecast prove the superiority of our method over state-of-the-art graph rewiring baselines. This paper provides a reasonable explanation of why subgraphs play a vital role in determining graph properties. The code is available at \url{https://github.com/smiles724/bottleneck}