Systems, methods, and apparatus, including computer programs encoded on a computer storage medium for processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer. In one aspect, the methods include obtaining a layer output generated by the one or more initial neural network layers and processing the layer output through the softmax output layer to generate a neural network output. Processing the layer output through the softmax output layer includes determining, for each possible output value, a number of occurrences in the layer output values; for each possible output value occurring in the layer output values, determining a respective exponentiation measure; determining a normalization factor for the layer output by combining the exponentiation measures in accordance with the number of occurrences of the possible output values; and determining, for each of layer output values, a softmax probability value.
Please help me write a proper abstract based on the patent claims. CLAIM: 1-20. (canceled) 21. A method of processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer to generate a neural network output for the network input, wherein the initial neural network layers are implemented by a processing system that performs computations specified by the one or more initial neural network layers using quantized arithmetic such that output values generated by the processing system can take only values from a predetermined finite set of values, the method comprising: determining a respective exponentiation measure for each possible output value of a predetermined finite set of possible output values that can be included in layer outputs generated by processing network inputs through the one or more initial neural network layers using the processing system; storing the respective exponentiation measures; obtaining a first layer output generated by the processing system processing a first network input through the one or more initial neural network layers, the first layer output having a plurality of first layer output values, and each first layer output value being a respective one of the predetermined finite set of possible output values; and processing the layer output through the softmax output layer to generate a first neural network output for the network input, comprising: determining, for each possible output value in the predetermined finite set of possible output values, a count of occurrences of the possible output value among the plurality of first layer output values that are in the first layer output; for each possible output value that occurs at least once in the in the plurality of layer output values, accessing the stored exponentiation measure for the possible output value; determining a normalization factor for the layer output by combining the stored exponentiation measures for the possible output values that occur in the first layer output at least once in accordance with the counts of occurrences of the possible output values; and determining, for each of the plurality of first layer output values, a softmax probability value from the respective exponentiation measure for the first layer output value and the normalization factor. 22. The method of claim 21, wherein obtaining the first layer output comprises: receiving a plurality of initial layer output values from the processing system, the plurality of initial layer output values being unmapped output values of the one or more initial neural network layers that are each a respective one of the predetermined finite set of values; obtaining mapping data defining a mapping from the predetermined finite set of values to the predetermined finite set of possible layer output values; and determining, for each initial layer output value, a corresponding first layer output value based on the mapping data. 23. The method of claim 22, wherein the mapping data specifies a scaling factor for scaling each of the plurality of initial layer output values to generate the layer output values. 24. (canceled) 25. (canceled) 26. The method of claim 21, wherein each of the finite set of possible output values map to a respective value of an integer data type. 27. The method of claim 21, further comprising: generating the first network input by converting one or more floating point values to fixed point values. 28. The method of claim 21, wherein determining the respective exponentiation measure for each possible output value comprises exponentiating Euler's number by a multiplication of the possible output value. 29. The method of claim 21, wherein each softmax probability value is determined by dividing each respective exponentiation measure by the normalization factor. 30. The method of claim 21, wherein each of the finite set of possible output values is an output of a mapping function. 31. The method of claim 21, wherein each of the finite set of possible output values is an output of a compression function. 32. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer to generate a neural network output for the network input, wherein the initial neural network layers are implemented by a processing system that performs computations specified by the one or more initial neural network layers using quantized arithmetic such that output values generated by the processing system can take only values from a predetermined finite set of values, the operations comprising: determining a respective exponentiation measure for each possible output value of a predetermined finite set of possible output values that can be included in layer outputs generated by processing network inputs through the one or more initial neural network layers using the processing system; storing the respective exponentiation measures; obtaining a first layer output generated by the processing system processing a first network input through the one or more initial neural network layers, the first layer output having a plurality of first layer output values, and each first layer output value being a respective one of the predetermined finite set of possible output values; and processing the layer output through the softmax output layer to generate a first neural network output for the network input, comprising: determining, for each possible output value in the predetermined finite set of possible output values, a count of occurrences of the possible output value among the plurality of first layer output values that are in the first layer output; for each possible output value that occurs at least once in the in the plurality of layer output values, accessing the stored exponentiation measure for the possible output value; determining a normalization factor for the layer output by combining the stored exponentiation measures for the possible output values that occur in the first layer output at least once in accordance with the counts of occurrences of the possible output values; and determining, for each of the plurality of first layer output values, a softmax probability value from the respective exponentiation measure for the first layer output value and the normalization factor. 33. The method of claim 32, wherein obtaining the first layer output comprises: receiving a plurality of initial layer output values from the processing system, the plurality of initial layer output values being unmapped output values of the one or more initial neural network layers that are each a respective one of the predetermined finite set of values; obtaining mapping data defining a mapping from the predetermined finite set of values to the predetermined finite set of possible layer output values; and determining, for each initial layer output value, a corresponding first layer output value based on the mapping data. 34. The method of claim 33, wherein the mapping data specifies a scaling factor for scaling each of the plurality of initial layer output values to generate the layer output values. 35. (canceled) 36. (canceled) 37. The method of claim 32, wherein each of the finite set of possible output values map to a respective value of an integer data type. 38. The method of claim 32, further comprising: generating the first network input by converting one or more floating point values to fixed point values. 39. The method of claim 32, wherein determining the respective exponentiation measure for each possible output value comprises exponentiating Euler's number by a multiplication of the possible output value. 40. A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations for processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer to generate a neural network output for the network input, wherein the initial neural network layers are implemented by a processing system that performs computations specified by the one or more initial neural network layers using quantized arithmetic such that output values generated by the processing system can take only values from a predetermined finite set of values, the operations comprising: determining a respective exponentiation measure for each possible output value of a predetermined finite set of possible output values that can be included in layer outputs generated by processing network inputs through the one or more initial neural network layers using the processing system; storing the respective exponentiation measures; obtaining a first layer output generated by the processing system processing a first network input through the one or more initial neural network layers, the first layer output having a plurality of first layer output values, and each first layer output value being a respective one of the predetermined finite set of possible output values; and processing the layer output through the softmax output layer to generate a first neural network output for the network input, comprising: determining, for each possible output value in the predetermined finite set of possible output values, a count of occurrences of the possible output value among the plurality of first layer output values that are in the first layer output; for each possible output value that occurs at least once in the in the plurality of layer output values, accessing the stored exponentiation measure for the possible output value; determining a normalization factor for the layer output by combining the stored exponentiation measures for the possible output values that occur in the first layer output at least once in accordance with the counts of occurrences of the possible output values; and determining, for each of the plurality of first layer output values, a softmax probability value from the respective exponentiation measure for the first layer output value and the normalization factor.