In accordance with an example embodiment of the present invention, a method comprising: obtaining a plurality of training samples; employing a set of activation functions on a plurality of layers of a deep neural network, wherein the set of activation functions varies with the plurality of layers; and applying the activation functions on the plurality of training samples.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method, comprising: obtaining a plurality of training samples; employing a set of activation functions on a plurality of layers of a deep neural network, wherein the set of activation functions varies with the plurality of layers; and applying the activation functions on the plurality of training samples. 2. The method of claim 1, wherein the set of activation functions comprises piece-wise linear activation functions and the slopes of the positive part of activation functions vary with the plurality of layers. 3. The method of claim 2, wherein the slopes of the positive part of activation functions decrease as the layer number increases. 4. The method of claim 2, wherein the slopes of the positive part of activation functions decrease as the layer number increases, and the slopes of the negative part of activation functions decrease as the layer number increases. 5. The method of claim 1, wherein the set of activation functions comprises piece-wise linear activation functions and smooth activation functions. 6. The method of claim 5, wherein the piece-wise linear activation functions is applied before the smooth activation functions. 7. The method of claim 6, wherein the first half of the plurality of layers use piece-wise linear activation functions and the second half of the plurality of layers use smooth activation functions. 8. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising: obtaining a plurality of training samples; employing a set of activation functions on a plurality of layers of a deep neural network, wherein the set of activation functions varies with the plurality of layers; and applying the activation functions on the plurality of training samples. 9. The computer storage medium of claim 8, wherein the set of activation functions comprises piece-wise linear activation functions and the slopes of the positive part of activation functions vary with the plurality of layers. 10. The computer storage medium of claim 9, wherein the slopes of the positive part of activation functions decrease as the layer number increases. 11. The computer storage medium of claim 9, wherein the slopes of the positive part of activation functions decrease as the layer number increases, and the slopes of the negative part of activation functions decrease as the layer number increases. 12. The computer storage medium of claim 1, wherein the set of activation functions comprises piece-wise linear activation functions and smooth activation functions. 13. The computer storage medium of claim 12, wherein the piece-wise linear activation functions is applied before the smooth activation functions. 14. The computer storage medium of claim 13, wherein the first half of the plurality of layers use piece-wise linear activation functions and the second half of the plurality of layers use smooth activation functions. 15. An apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the apparatus to at least: obtain a plurality of training samples; employ a set of activation functions on a plurality of layers of a deep neural network, wherein the set of activation functions varies with the plurality of layers; and apply the activation functions on the plurality of training samples. 16. The apparatus of claim 15, wherein the set of activation functions comprises piece-wise linear activation functions and the slopes of the positive part of activation functions vary with the plurality of layers. 17. The apparatus of claim 16, wherein the slopes of the positive part of activation functions decrease as the layer number increases. 18. The apparatus of claim 16, wherein the slopes of the positive part of activation functions decrease as the layer number increases, and the slopes of the negative part of activation functions decrease as the layer number increases. 19. The apparatus of claim 15, wherein the set of activation functions comprises piece-wise linear activation functions and smooth activation functions. 20. The apparatus of claim 19, wherein the piece-wise linear activation functions is applied before the smooth activation functions.