${W_1,W_2}$. We compute the $r$ weighted sums by multiplying the annotation matrix $A$ and LSTM hidden states $H_i^t$:

![\[fig:attention\]Self attention mechanism](attention.PNG){width="100.00000%" height="4.0in"}

$$\begin{aligned}
    N_i^t = AH_i^t\end{aligned}$$

Eventually, the sentence encoding vector $H_i^t$ then becomes a matrix $N_i^t\in R^{r*2u}$ and we use $N_i^t$ to represent the $ith$ news title in date $t$ after encoding. By using multi-hop attention mechanism, the vector representation usually focuses on a specific component of the sentence, like a special set of related words or phrases. Therefore, it is expected to reflect an aspect, or component of the semantics in a sentence instead of adding attention on a specific word.

Shown in equation (12), we apply another MLP layer on top of our self-attention layer to learn which attention group should be rewarded with the highest assign value. We name this as attention-over-attention, the weight matrix have a shape of $W_3\in R^r$, and the final representation of the sentence encoding vector $N_i^t$ are shaped as $N_i^t\in R^{2u}$. At last, we use $N_i^t$to represent a news title encoded from the input.

$$\begin{aligned}
    N_i^t = tanh(W_3N_i^t+b_1)\end{aligned}$$

***News level Self-attention layer***: Not all news contributes equally to predicting the stock trend. Hence, in order to reward the news that offers 