Hardware accelerator for compressed Long Short Term Memory (LSTM) is disclosed. The accelerator comprise a sparse matrix-vector multiplication module for performing multiplication operation between all sparse matrices in the LSTM and vectors to sequentially obtain a plurality of sparse matrix-vector multiplication results. A addition tree module are also included for accumulating a plurality of said sparse matrix multiplication results to obtain an accumulated result. And a non-linear operation module passes the accumulated results through an activation function to generate non-linear operation result. That is, the present accelerator adopts pipeline design to overlap the time of data transfer and computation for compressed LSTM.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A hardware accelerator for compressed Long Short Term Memory (LSTM), comprising: a sparse matrix-vector multiplication module for performing multiplication operation between all sparse matrices in the LSTM and vectors to sequentially obtain a plurality of sparse matrix-vector multiplication results; an addition tree module for accumulating a plurality of said sparse matrix multiplication results to obtain an accumulated result; and a non-linear operation module for passing the accumulated results through an activation function to generate non-linear operation result. 2. The hardware accelerator of claim 1, further comprising: an element-wise multiplication module for performing element-wise multiplication and addition operations on a plurality of said non-linear operation results to obtain a first intermediate result vector of the current cycle. 3. The hardware accelerator of claim 2, further comprising: a diagonal matrix multiplication module for performing a multiplication operation between a diagonal matrix and the first intermediate result vector of the current cycle or the first intermediate result vector of the previous cycle to obtain a diagonal matrix multiplication result, Wherein the addition tree module further accumulates the diagonal matrix multiplication results into the accumulated results. 4. The hardware accelerator of claim 3, wherein, the diagonal matrix multiplication module is a second element-wise multiplication module. 5. The hardware accelerator of claim 4, wherein the input timing of the plurality of different data and the operation timing of the plurality of calculation processes are set in parallel, per the dependency of the calculation processes on the data, wherein: an input operation or other calculation process for e data involved in a calculation process is set before said one calculation process; setting a data input operation unrelated to the calculation process in parallel; the calculation process is divided into two layers in parallel: one layer is the multiplication operation of the sparse matrix-vector multiplication module which in turn performs the multiplication operation between the plurality of sparse matrices and the corresponding vectors, and the other layer is the other calculation operation of the other calculation modules than the sparse matrix-vector multiplication module, the other computation operations are performed serially. 6. The hardware accelerator of claim 5, wherein, the diagonal matrix multiplication module performs the multiplication operation of the corresponding diagonal matrix and the first intermediate result vector of the previous cycle before the addition tree module completes the first accumulation operation of the current cycle; and the diagonal matrix multiplication module performs a multiplication operation of the diagonal matrix and the first intermediate result vector of the current cycle after the element-wise multiplication module obtains the first intermediate result vector of the current cycle. 7. The hardware accelerator of claim 2, wherein, the sparse matrix-vector multiplication module performs a sparse matrix multiplication operation on the input vector of the current cycle xt, the output vector of the previous cycle yt-1, and the second intermediate result vector of the current cycle mt, wherein the second intermediate result vector of the current cycle mt is calculated based on the input vector of the current cycle xt, the output vector of the previous cycle yt-1, and the first intermediate result vector of the current cycle Ct; and the sparse matrix-vector multiplication module obtains the output vector of the current cycle yt by performing sparse matrix-vector multiplication operation on the second intermediate result vector of the current cycle mt. 8. The hardware accelerator of claim 7, wherein, the non-linear operation result comprising: the input gate control vector of the current cycle it; the forget gate control vector of the current cycle ft; the output gate control vector of the current cycle ot; the cell input activation vector of the current cycle gt; and the cell output activation vector of the current cycle ht, the first intermediate result vector is a cell activation vector of the current cycle ct, the second intermediate result vector is a cell output activation vector of the current cycle mt. 9. The hardware accelerator of claim 1, wherein, the sparse matrix-vector multiplication module further adds a bias vector to the sparse matrix-vector multiplication result; or the addition tree module further accumulates a bias vector into the accumulated result. 10. The hardware accelerator of claim 1, wherein, the sparse matrix-vector multiplication module comprises: two sparse matrix buffers configured in alternate buffer mode for buffering sparse matrix coefficients, two pointer buffers configured in alternate buffer mode for buffering pointers representing non-zero coefficient start positions in each column of said sparse matrix; and two output buffers configured in alternate buffer mode to output the calculation result from one output buffer while the other output buffer is used to buffer the calculation result. 11. The hardware accelerator of claim 1, further comprising: a vector input module for inputting a vector to be calculated to the sparse matrix-vector multiplication module; and an output module connected to the sparse matrix-vector multiplication module for outputting a final calculation result vector of the current cycle, wherein the output module comprises a buffer for buffering the final calculation result vector of the previous cycle and supplying it to the vector input module. 12. The hardware accelerator of claim 11, wherein, the vector input module comprising a plurality of FIFOs, the sparse matrix-vector multiplication module comprising a plurality of processing elements, each FIFO provides its corresponding processing element with different elements in the vector to be calculated, and the plurality of processing elements execute the partial sparse matrix-vector multiplication operation for the vector in parallel. 13. The hardware accelerator of claim 1, wherein, the activation function is one of a sigmoid function, and a tanh function. 14. The hardware accelerator of claim 1, wherein, the hardware accelerator is implemented as an Application Specific Integrated Circuit (ASIC) core. 15. The hardware accelerator of claim 1, wherein, the hardware accelerator is implemented as a field-programmable gate array (FPGA). 16. The hardware accelerator of claim 1, wherein, the compressed LSTM is configured to execute speech recognition. 17. A speech recognition engineer accelerator system, comprising: a processor configured to execute speech recognition that use a compressed Long Short Term Memory (LSTM) as an acoustic model; and a hardware accelerator for compressed Long Short Term Memory(LSTM), comprising: a sparse matrix-vector multiplication module for performing multiplication operation between all sparse matrices in the LSTM and vectors to sequentially obtain a plurality of sparse matrix-vector multiplication results; an addition tree module for accumulating a plurality of said sparse matrix multiplication results to obtain an accumulated result; and a non-linear operation module for passing the accumulated results through an activation function to generate non-linear operation result. 18. The system of claim 17, wherein, the hardware accelerator is implemented as a part of a FPGA structure, the FPGA structure further comprising: on-chip buffers, including input buffer and output buffer, for preparing data to be consumed by the hardware accelerator and store the generated results; and controller for scheduling PCIE/MEM controller for data-fetch and the LSTM computation pipeline flow of the hardware accelerator.