A method, computer readable medium, and system are disclosed for neural network pruning. The method includes the steps of receiving first-order gradients of a cost function relative to layer parameters for a trained neural network and computing a pruning criterion for each layer parameter based on the first-order gradient corresponding to the layer parameter, where the pruning criterion indicates an importance of each neuron that is included in the trained neural network and is associated with the layer parameter. The method includes the additional steps of identifying at least one neuron having a lowest importance and removing the at least one neuron from the trained neural network to produce a pruned neural network.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A neural network pruning system, comprising: a processor configured to: receive first-order gradients of a cost function with respect to layer parameters for a trained neural network; compute a pruning criterion for each layer parameter based on the first-order gradient corresponding to the layer parameter, wherein the pruning criterion indicates an importance of each neuron that is included in the trained neural network and is associated with the layer parameter; identify at least one neuron having a lowest importance; and remove the at least one neuron from the trained neural network to produce a pruned neural network. 2. The neural network pruning system of claim 1, wherein the trained neural network is a trained using a first dataset that is general before being trained using a second dataset that is specific. 3. The neural network pruning system of claim 2, wherein the first-order gradients are computed while the trained neural network is trained using the second dataset. 4. The neural network pruning system of claim 1, wherein the trained neural network is a convolutional neural network. 5. The neural network pruning system of claim 1, wherein computing the pruning criterion for each layer parameter comprises taking an absolute value of the layer parameter scaled by the first-order gradient corresponding to the layer parameter. 6. The neural network pruning system of claim 1, wherein computing the pruning criterion for each layer parameter comprises summing a square of the layer parameter scaled by the first-order gradient corresponding to the layer parameter over input channels and dimensions of a convolution kernel. 7. The neural network pruning system of claim 1, wherein the layer parameter is an activation value. 8. The neural network pruning system of claim 1, wherein the layer parameter is a weight. 9. The neural network pruning system of claim 1, wherein the processor is further configured to perform fine-tuning on the pruned neural network. 10. The neural network pruning system of claim 1, wherein the at least one neuron includes neurons having importances below a threshold value. 11. The neural network pruning system of claim 1, wherein the at least one neuron comprises a predetermined percentage of all of the neurons in the trained neural network. 12. A computer-implemented method, comprising: receiving first-order gradients of a cost function relative to layer parameters for a trained neural network; computing a pruning criterion for each layer parameter based on the first-order gradient corresponding to the layer parameter, wherein the pruning criterion indicates an importance of each neuron that is included in the trained neural network and is associated with the layer parameter; identifying at least one neuron having a lowest importance; and removing the at least one neuron from the trained neural network to produce a pruned neural network. 13. The computer-implemented method of claim 12, wherein the trained neural network is trained using a first dataset that is general before being trained using a second dataset that is specific. 14. The computer-implemented method of claim 13, wherein the first-order gradients are computed while the trained neural network is trained using the second dataset. 15. The computer-implemented method of claim 12, wherein the trained neural network is a convolutional neural network. 16. The computer-implemented method of claim 12, wherein computing the pruning criterion for each layer parameter comprises taking an absolute value of the layer parameter scaled by the first-order gradient corresponding to the layer parameter. 17. The computer-implemented method of claim 12, wherein computing the pruning criterion for each layer parameter comprises summing a square of the layer parameter scaled by the first-order gradient corresponding to the layer parameter over input channels and dimensions of a convolution kernel. 18. The computer-implemented method of claim 12, wherein the layer parameter is an activation. 19. The computer-implemented method of claim 12, wherein the layer parameter is a weight. 20. A non-transitory, computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform steps comprising: receiving first-order gradients of a cost function relative to layer parameters for a trained neural network; computing a pruning criterion for each layer parameter based on the first-order gradient corresponding to the layer parameter, wherein the pruning criterion indicates an importance of each neuron that is included in the trained neural network and is associated with the layer parameter; identifying at least one neuron having a lowest importance; and removing the at least one neuron from the trained neural network to produce a pruned neural network.