The present technical disclosure relates to artificial neural networks, e.g., gated recurrent unit (GRU). In particular, the present technical disclosure relates to how to implement a hardware accelerator for compressed GRU based on an embedded FPGA. Specifically, it proposes an overall design processing method of matrix decoding, matrix-vector multiplication, vector accumulation and activation function. In another aspect, the present technical disclosure proposes an overall hardware design to implement and accelerate the above process.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A device for implementing compressed GRU (recurrent neural network), said device comprising: a receiving unit, which is used to receive a plurality of input vectors and distribute them to a plurality of processing elements (PE); a plurality of processing elements (PE), each of which comprising: a reading unit configured to read weight matrices W, said W indicates weights of said GRU; an ALU configured to perform multiplication and addition calculation of said weight matrices W; a calculation buffer configured to store results of matrix-vector multiplication and output results to an assembling unit; an assembling unit configured to receive results from PEs and assemble them into a complete resultant vector; a hidden layer computation module, configured to read matrix-vector multiplication result from said processing elements, and to compute update signal, reset signal and the hidden layer's activation output vector h; and a controller unit configured for controlling said plurality of processing elements. 2. The device of claim 1, said hidden layer computation module further comprising: an activation function unit, configured to perform hidden layer's activation function of said GRU; a selector, configured to receive data from said assembling unit and element-wise multiplier and select one of the received data to be output to an adder tree; a Wx buffer, configured to receive and store matrix-vector multiplication results from the assembling unit and output corresponding result to the adder tree according to the instruction from the controller unit; an adder tree, configured to conduct vector accumulation operation on vectors received from the Wx buffer and the selector; an element-wise multiplier, configured to conduct element-wise multiplication on vectors received from the assembling unit and the activation function unit, and to output the multiplication result to the selector. 3. The device of claim 1, wherein said receiving unit further comprises: a plurality of first-in-first-out buffer, and each of which corresponds to a PE. 4. The device of claim 1, wherein said reading unit further comprises: a location unit configured to read and store location information of non-zero elements in the weight matrix W; a decoding unit configured to decoding the weight matrix to obtain the weight value of the weight matrix W. 5. The device of claim 1, wherein said ALU further comprises: a multiplier configured to perform multiplication operation on matrix elements and input vector elements; an adder configured to perform addition operation on the results of above multiplication operation. 6. The device of claim 1, wherein said calculation buffer further comprises: a first output buffer and a second output buffer, said first and second buffer receive and output computation result alternatively, wherein while one buffer receives the present computation result, the other buffer outputs the previous computation result. 7. The device of claim 1, wherein said hidden layer computation module further comprises: a {tilde over (h)} buffer, configured to receive and store a candidate activation {tilde over (h)} from the activation function unit, and to send the received candidate activation {tilde over (h)} to a linear interpolation unit to compute a hidden layer activation h; a Z buffer, configured to receive and store a update signal Z, and to send the received update signal Z to the linear interpolation unit in order to compute a hidden layer activation h; a linear interpolation unit, configured to compute said hidden layer activation h by conducting linear interpolation operation on data received from the {tilde over (h)} buffer, Z buffer and a vector buffer; and a vector buffer, configured to receive and store the layer activation h of respective hidden layers. 8. A method for implementing compressed GRU based on FPGA, comprising: a) receiving data related to GRU computation, including input vector, bias vector and weight matrices; b) decoding the data received in step a) in order to obtain the real weights; c) conducting matrix computation by performing matrix-vector multiplication using FPGA on-chip processing elements; d) computing an update signal and a reset signal of GRU, by performing vector accumulating and activation function computing; e) computing a candidate hidden layer activation of GRU, by performing element-wise multiplication, addition and activation function computing on the reset signal and matrix computation results so as to obtain a candidate hidden layer activation of GRU; f) computing a hidden layer activation of GRU, by performing a linear interpolation operation on the candidate hidden layer activation, update signal and hidden layer activation applied to previous input vector; iterating the above steps a), b), c), d), e), f) to obtain GRU's activation sequences and computing GRU's output on the basis of the GRU's activation sequences. 9. The method of claim 8, further comprising: distributing received data to a plurality of parallel processing elements (PE) after receiving data in step a); and assembling results from each PE to obtain a complete result vector of matrix-vector multiplication after computation in step c). 10. The method of claim 8, in at least one of said steps a), b), c), providing a pair of ping-pong buffers in a FPGA on-chip memory. 11. A method for implementing a Gated Recurrent Unit (GRU) network, wherein the weights of said GRU being characterized by Wz, Wr, W, Wx, Uz, Ur, U and Uh, where Wz is the update gate's weight matrix being applied to inputs, Wr is the reset gate's weight matrix being applied to inputs, W is the transformation matrix being applied to inputs, Wx is a matrix combining Wz, Wr and W, Uz is the update gate's weigh matrix being applied to hidden layers, Ur is the reset gate's weigh matrix being applied to hidden layers, U is the transformation matrix being applied to hidden layers, Uh is a matrix combining Ur, U and Uh, where an activation to be applied to an input vector by said hidden layers is ht, the input of said GRU is a series of input vectors x=(x1,x2 . . . , xT), said method comprises: initialization step of reading necessary data for computing WxX into a FPGA on-chip memory; by processing elements of said FPGA, step 1 of computing WxX, and reading necessary data for computing Uhht−1 into the FPGA on-chip memory, where ht−1 is an activation to be applied to the previous input vector by the hidden layer; by processing elements of said FPGA, step 2 of computing Uhht−1 and reading necessary data for computing the next WxX into the FPGA on-chip memory; iteratively repeating said step 1 and step 2. 12. The method of claim 11, wherein each of said Step 1 and Step 2 further comprises: while computing matrix-vector multiplication for the present input vector, computing the update signal Z, reset signal R, candidate hidden layer activation h and hidden layer activation ht. 13. The method of claim 12, wherein: said initial step, step 1 and step 2 are processed sequentially; and said step 1 and step 2 are processed periodically.