the encoder. There are three different levels inside the encoder, including point level, scale level, and region level. At each level, we introduce a self-attention module to learn self-attention weights by mining the correlations among the corresponding feature elements. Consequently, three self-attention modules are designed to propagate features from the lower level to the higher level. Supposed the input of the self-attention module is a feature map $\bm{x} \in \mathbb{R}^{D_1 \times D_2}$, where $D_1$, $D_2$ are the dimensions of the feature map. Therefore, $D_1$, $D_2$ are equal to $K_t$, $3$ in the point level, equal to $T$, $D$ in the scale level and equal to $M$, $D$ in the region level, respectively.

As depicted in Figure \[fig:c\_e\], the feature map $\bm{x}$ is first transformed into two feature spaces $\bm{f}$ and $\bm{g}$ to calculate the attention below, where $\bm{f}(\bm{x}) = \bm{W_f x}$, $\bm{g}(\bm{x}) = \bm{W_g x}$, $$\beta_{j,i} = \frac{exp(s_{ij})}{\sum_{i=1}^{D_1} exp(s_{ij})}, \text{where } s_{ij} = \bm{f}(\bm{x_i})^T\bm{g}(\bm{x_j}),$$ and $\beta_{j,i}$ evaluates the attention degree which the model pays to the $i^{th}$ location when synthesizing the $j^{th}$ feature vector. Then the attention result is $\bm{r} = (\bm{r_1}, \bm{r_2}, \cdots, \bm{r_j}, \cdots, \bm{r_{D_1}}) \in \mathbb{R}^{D_1 \times D_2}$, where $$\bm{r_j} = \sum_{i=1}^{D_1} \beta_{j,i}\bm{h}(\bm{x_i}), \text{where }\bm{h}(\bm{x_i}) = \bm{W_h 