The present invention relates to artificial neural network (ANN), for example, convolutional neural network (CNN). In particular, the present invention relates to how to implement and optimize a convolutional neural network based on an embedded FPGA. Specifically, it proposes a CPU+FPGA heterogeneous architecture to accelerate ANNs.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A deep processing unit (DPU) for implementing an Artificial Neural Network (ANN), comprising: a CPU, configured for scheduling a programmable logic module and a direct memory access (DMA), a direct memory access (DMA), connected to the CPU, an external memory and a programmable logic module, used for communication between the external memory and the programmable logic module; an external memory, coupled to the CPU and the DMA, configured for storing instructions of the ANN and data to be processed by said ANN; a programmable logic module (PL), comprising: a controller, configured for getting instructions and scheduling operations of a computing complex on the basis of the instructions; a computing complex, including a plurality of processing elements (PEs), configured for performing operations on the basis of the instructions and data; a buffer, configured for preparing the data and instructions for the computing complex; wherein the CPU is configured to control the DMA to transmit data and instructions between the external memory and the programmable logic module. 2. The DPU of claim 1, wherein the DMA is configured to transmit data between the external memory and the programmable logic module via FIFO; the DMA is configured to transmit instructions between the external memory and the programmable logic module via FIFO. 3. The DPU of claim 1, wherein the CPU further comprises: status monitoring module, configure for monitoring status of a Finite State Machine (FSM) of the controller in the programmable logic module. 4. The DPU of claim 1, the PE further comprises: a convolver complex, coupled to the buffer to receive weights of ANN and said data, configured for performing convolutional operations of the ANN; adder tree, coupled to the convolver complex, configured for summing results of convolution operation; non-linear (NL) module, coupled to the adder tree, configured for applying a non-linear function to the output of adder tree. 5. The DPU of claim 1, the PE further comprises: pooling module, coupled to the NL module, configured for performing max-pooling operation on the output of NL module. 6. The DPU of claim 1, the buffer further comprises: input buffer, configured for preparing the data, instructions for said convolver complex; output buffer, for storing and outputting data results. 7. The DPU of claim 6, the buffer further comprises: bias shift, coupled to the input buffer, configured for shifting weights of ANN between different numerical ranges and providing said shifted weights to the adder tree, wherein the weights are quantized fixed-point numbers. 8. The DPU of claim 2, wherein the CPU, the programmable logic module and the DMA are implemented in one single SOC. 9. The DPU of claim 8, wherein the external memory is implemented by a separate memory chip. 10. A deep processing unit (DPU) for implementing an Artificial Neural Network (ANN), comprising: a CPU, configured for scheduling a programmable logic module and a direct memory access (DMA), a direct memory access (DMA), connected to the CPU, an external memory and a programmable logic module, used for communication between the external memory and the programmable logic module; an external memory, coupled to the CPU, the DMA and a programmable logic module, configured for storing instructions of the ANN and data to be processed by said ANN; a programmable logic module (PL), comprising: a controller, configured for getting instructions and scheduling operations of a computing complex on the basis of the instructions; a computing complex, including a plurality of processing elements (PEs), configured for performing operations on the basis of the instructions and data; a buffer, configured for preparing the data and instructions for the computing complex; wherein the CPU is configured to control the DMA to transmit instructions between the external memory and the programmable logic module; and wherein the external memory and the programmable logic module transmit data to each other directly. 11. The DPU of claim 10, wherein the DMA is configured to transmit instructions between the external memory and the programmable logic module via FIFO. 12. The DPU of claim 10, wherein the CPU further comprises: status monitoring module, configure for monitoring status of a Finite State Machine (FSM) of the controller in the programmable logic module. 13. The DPU of claim 10, the PE further comprises: a convolver complex, coupled to the buffer to receive weights of ANN and said data, configured for performing convolutional operations of the ANN; adder tree, coupled to the convolver complex, configured for summing results of convolution operation; non-linear (NL) module, coupled to the adder tree, configured for applying a non-linear function to the output of adder tree. 14. The DPU of claim 10, the PE further comprises: pooling module, coupled to the NL module, configured for performing max-pooling operation on the output of NL module. 15. The DPU of claim 10, the buffer further comprises: input buffer, configured for preparing the data, instructions for said convolver complex; output buffer, configured for storing and outputting data results. 16. The DPU of claim 15, the buffer further comprises: bias shift, coupled to the input buffer, configured for shifting weights of ANN between different numerical ranges and providing said shifted weights to the adder tree, wherein the weights are quantized fixed-point numbers. 17. The DPU of claim 10, wherein the CPU, the programmable logic module and the DMA are implemented in one single SOC. 18. The DPU of claim 17, wherein the external memory is implemented by a separate memory chip. 19. A method for implementing an Artificial Neural Network (ANN), comprising: providing a CPU for scheduling a programmable logic module and a direct memory access (DMA), providing a direct memory access (DMA) connected to the CPU, an external memory and a programmable logic module, for communication between the external memory and the programmable logic module; providing an external memory coupled to the CPU and the DMA, for storing instructions of the ANN and data to be processed by said ANN; providing a programmable logic module (PL), comprising: a controller, configured for getting instructions and scheduling operations of a computing complex on the basis of the instructions; a computing complex, including a plurality of processing elements (PEs), configured for performing operations on the basis of the instructions and data; a buffer, configured for preparing the data and instructions for the computing complex; transmitting data and instructions between the external memory and the programmable logic module via DMA under the control of CPU. 20. A method for implementing an Artificial Neural Network (ANN), comprising: providing a CPU for scheduling a programmable logic module and a direct memory access (DMA), providing a direct memory access (DMA) coupled to the CPU, an external memory and a programmable logic module, for communication between the external memory and the programmable logic module; providing an external memory coupled to the CPU, the DMA and a programmable logic module, for storing instructions of the ANN and data to be processed by said ANN; providing a programmable logic module (PL), comprising: a controller, configured for getting instructions and scheduling operations of a computing complex on the basis of the instructions; a computing complex, including a plurality of processing elements (PEs), configured for performing operations on the basis of the instructions and data; a buffer, configured for preparing the data and instructions for the computing complex; transmitting instructions between the external memory and the programmable logic module via DMA under the control of CPU; and transmitting data between the external memory and the programmable logic module directly.