self-attention layer

[1.0]{} $$\begin{aligned}
    \overrightarrow{h_i} = \overrightarrow{LSTM}(D_t)\end{aligned}$$ $$\begin{aligned}
    \overleftarrow{h_i} = \overleftarrow{LSTM}(D_t)\end{aligned}$$

We adopt day level Bi-LSTM to encode the temporal sequence of corpus vectors $D_i$, $t\in [1,N]$. Shown in Equation (16) and (17), We obtain the hidden vectors($\overrightarrow{h_i}$ and $\overleftarrow{h_i}$) from the day-level Bi-LSTM and concatenate them to $H_i=[\overrightarrow{h_1};\overleftarrow{h_1},....,\overrightarrow{h_N};\overleftarrow{h_N}]$, $H_i$ represents a vector that encodes the temporal sequence $D_t$ where $N$ refers to the sequence number. Since the news published at different dates contribute to the stock trend unequally, we adopt self-attention mechanism again to reward the dates that contribute most to the stock trend prediction, Shown in Equation below:

[1.0]{} $$\begin{aligned}
    A = softmax(W_8tanh(W_7H_i^\intercal))\end{aligned}$$ $$\begin{aligned}
    V = AD\end{aligned}$$ $$\begin{aligned}
    V = tanh(W_9V+b_3)\end{aligned}$$

In the formula, $D=(D_1,....,D_t)$ and $D\in R^{N*2v}$ , $V\in R^{2v}$ represents the final vector for all news proposed before the prediction date $t+1$ in a delay window with size $N$, where $v$ is the hidden unit number in the day level Bi-LSTM. Note that the weight matrix $[{W_7,W_8,W_9,b_3}]$ in the day level attention layer are different from the weight matrices mentioned in the previous section.

### Output and Prediction Layer

The last stage of our At-LSTM model is a traditional fully connected layer with softmax as activation 