Systems, apparatus and methods are described including operations for a flexible neural network accelerator.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A neural network accelerator, comprising: a plurality of fixed function hardware logic blocks that are logically parallel so that multiple ones of the logic blocks have substantially the same arrangement of logic elements to compute a value associated with a layer of a neural network; wherein the logic elements of multiple logic blocks of the plurality of logic blocks are to provide the option to alternatively use weights of at least two different bit lengths comprising weights of a first bit length as inputted to the logic block and weights of a second bit length as inputted to the logic block, and to apply a scale factor to weights provided in at least the first bit length, and to omit the scale factor when weights are provided in at least the second bit length. 2. The neural network accelerator of claim 1 wherein the logic blocks are each arranged to multiply a scaled or non-scaled weight by an input value to compute a weighted input, the neural network accelerator comprising an accumulator portion that sums the weighted inputs; and an activation function unit that uses the sum of the weighted input to compute an output for a node of the layer of the neural network. 3. The neural network accelerator of claim 1 wherein alternative application of the scale factor controls a dynamic range of a resulting sum from the logic blocks. 4. The neural network accelerator of claim 1 wherein the logic blocks are arranged to provide the option to apply no scale factor to a layer or to apply a scale factor individually to one or more layers. 5. The neural network accelerator of claim 1 wherein each logic block receives an input value to modify and to provide to accumulate with modified input values of other ones of the logic blocks to be used to compute a single output of a single node of the neural network, and wherein the scale factor value applied to the input value at each logic block for the single output is the same. 6. The neural network accelerator of claim 1 comprising an activation function unit to compute an output associated with the value of the neural network and using a piecewise linear function supported by more than one activation function layer type, wherein the logic blocks are arranged to alternatively be re-used to provide weighted inputs of the variety of neural network layer types in a variety of layer sequences, and while using the same logic elements of the logic blocks. 7. The neural network accelerator of claim 6 wherein the neural network layer types comprises at least two of: a recurrent layer, a convolutional layer, and an affine layer. 8. The neural network accelerator of claim 1 wherein the logic elements of multiple individual logic blocks are arranged to alternatively provide a value of a Gaussian mixture model (GMM). 9. The neural network accelerator of claim 1 wherein the logic elements of multiple individual logic blocks are arranged to alternatively (1) multiply the scale factor by a neural network weight and (2) square a mean-input value difference of a GMM. 10. The neural network accelerator of claim 1 wherein the logic elements of multiple individual logic blocks are arranged to alternatively (1) multiply a GMM variance by a squared difference of a mean and input value and (2) multiply a node input of a layer of the neural network by a weight. 11. The neural network accelerator of claim 1 comprising an active list buffer having an index indicating which subset of outputs to compute that is less than all outputs in a single layer of the neural network. 12. The neural network accelerator of claim 1 comprising a layer descriptor register to hold at least one layer descriptor and that indicates a layer is at least one of a set of pre-defined primitive layers available from the neural network, the neural network accelerator being arranged to process the primitive layers using data indicated by the layer descriptors. 13. The neural network accelerator of claim 12 wherein a state machine is provided to read the data of the layer descriptor in the layer descriptor register to direct layer data to the logic blocks. 14. The neural network accelerator of claim 1 comprising a pipeline comprising a multiplication addition circuit (MAC) unit comprising the fixed function hardware logic blocks arranged to provide weighted inputs and an accumulator to accumulate the weighted inputs into a sum, and an activation function unit arranged to receive the sum from the MAC and to compute an output, wherein the MAC generates a sum for one set of inputs simultaneously to the activation function unit generating an output associated with another set of inputs. 15. The neural network accelerator of claim 1 wherein the device performs neural network operations as part of automatic speech recognition, wherein the output of the neural network indicates one or more recognized words or sounds from an input audio signal. 16. The neural network accelerator of claim 1 wherein the logic blocks are each arranged to multiply a scaled or non-scaled weight by an input value to compute a weighted input, the neural network accelerator comprising an accumulator portion that sums the weighted inputs; and an activation function unit that uses the sum of the weighted input to compute an output for a node of the layer of the neural network; wherein alternative application of the scale factor controls a dynamic range of a resulting sum from the logic blocks; wherein the logic blocks are arranged to provide the option to apply no scale factor to a layer or to apply a scale factor individually to one or more layers; wherein each logic block receives an input value to modify and to provide to accumulate with modified input values of other ones of the logic blocks to be used to compute a single output of a single node of the neural network, and wherein the scale factor value applied to the input value at each logic block for the single output is the same; and the neural network accelerator comprising an activation function unit to compute an output associated with the value of the neural network and using a piecewise linear function supported by more than one activation function layer type, wherein the logic blocks are arranged to alternatively be re-used to provide weighted inputs of the variety of neural network layer types in a variety of layer sequences, and while using the same logic elements of the logic blocks. 17. A computer-implemented system, comprising: at least one internal buffer to hold layer data associated with at least one layer to be processed; and a plurality of fixed function logic blocks that are logically parallel so that multiple ones of the logic blocks have substantially the same arrangement of logic elements to compute a value associated with the layer of the neural network and by receiving the layer data of the at least one internal buffer, wherein the internal buffer is disposed locally to the fixed function logic blocks; wherein the logic elements of multiple logic blocks of the plurality of logic blocks are to provide the option to alternatively use weights of at least two different bit lengths comprising weights of a first bit length as inputted to the logic block and weights of a second bit length as inputted to the logic block, and to apply a scale factor to weights provided in at least the first bit length, and to omit the scale factor when weights are provided in at least the second bit length. 18. The system of claim 17 wherein the logic elements of each block comprise a multiplier to alternatively (1) multiply the scale factor by a neural network weight and (2) square a difference between a mean and an input value of a GMM. 19. The system of claim 17 wherein the logic elements of each block comprise a multiplier to alternatively (1) multiply a GMM variance by a squared difference of a mean and input value and (2) multiply a node input of a layer of the neural network by a weight. 20. The system of claim 17 wherein the weights are either 16 bit or 8 bit integer weights, and wherein the 8 bit weights are scaled. 21. The system of claim 17 comprising an active list buffer having an index of selected outputs of a layer to be determined that is a subset of outputs of the layer. 22. The system of claim 17 comprising at least one first state machine to store one layer descriptor at a time indicating the parameters and state of a layer of a neural network to be processed, wherein the at least one internal buffer holds layer data associated with the current layer descriptor at the first state machine. 23. The system of claim 22 comprising at least one sequencer that is a state machine to read the layer descriptor and direct the placement of the layer data into the at least one internal buffer depending on the data of the layer descriptor. 24. The system of claim 17 comprising an input buffer at the internal buffer and receiving input values of an input array in a memory comprising receiving a portion of the input values less than the entire input array, and portion by portion, when the input array is larger than the capacity of the input buffer, wherein the portion is sized to fit within the input buffer, wherein the fixed function logic blocks receiving the input values in the input buffer; the system comprising: an accumulator to compute at least one intermediate sum output for each portion and to operate by: saving and accumulating the intermediate sum outputs to form a single complete sum output; and providing the complete sum output to be input to an activation function to determine a final output. 25. The system of claim 24 wherein the input buffer receives data from an interleaved input array having rows or columns of groups of input sets of input elements, wherein each group in the input array is used to form a different final output of a single neural network layer, and provides input elements from multiple groups in the input buffer to the fixed function logic blocks rather than all of the input elements of one group after another group in the input array. 26. The system of claim 17 wherein the logic elements of each block comprise a multiplier to alternatively (1) multiply the scale factor by a neural network weight and (2) square a difference between a mean and an input value of a GMM; wherein the logic elements of each block comprise a multiplier to alternatively (1) multiply a GMM variance by a squared difference of a mean and input value and (2) multiply a node input of a layer of the neural network by a weight; wherein the weights are either 16 bit or 8 bit integer weights, and wherein the 8 bit weights are scaled; the system of claim 17 comprising: an active list buffer having an index of selected outputs of a layer to be determined that is a subset of outputs of the layer; at least one first state machine to store one layer descriptor at a time indicating the parameters and state of a layer of a neural network to be processed, wherein the at least one internal buffer holds layer data associated with the current layer descriptor at the first state machine; at least one sequencer that is a state machine to read the layer descriptor and direct the placement of the layer data into the at least one internal buffer depending on the data of the layer descriptor; an input buffer at the internal buffer and receiving input values of an input array in a memory comprising receiving a portion of the input values less than the entire input array, and portion by portion, when the input array is larger than the capacity of the input buffer, wherein the portion is sized to fit within the input buffer, wherein the fixed function logic blocks receiving the input values in the input buffer; an accumulator to compute at least one intermediate sum output for each portion and to operate by: saving and accumulating the intermediate sum outputs to form a single complete sum output; and providing the complete sum output to be input to an activation function to determine a final output; wherein the input buffer receives data from an interleaved input array having rows or columns of groups of input sets of input elements, wherein each group in the input array is used to form a different final output of a single neural network layer, and provides input elements from multiple groups in the input buffer to the fixed function logic blocks rather than all of the input elements of one group after another group in the input array. 27. A computer implemented method of neural network propagation, comprising: computing a value of a layer of a neural network by using a plurality of fixed function hardware logic blocks that are substantially logically parallel so that multiple ones of the logic blocks have substantially the same arrangement of logic elements; inputting a weight having a bit length from at least two alternative bit lengths available to input weights to the plurality of logic blocks wherein the available bit lengths comprise at least a first bit length and a second bit length; and applying a scale factor to weights provided in at least the first bit length, and to omit the scale factor when weights are provided in at least the second bit length. 28. The method according to claim 27 comprising multiplying, by the logic blocks, a scaled or non-scaled weight by an input value to compute a weighted input, summing the weighted inputs by an accumulator portion; and using the sum of the weighted input to compute an output for a node of the layer of the neural network and by an activation function unit; wherein alternative application of the scale factor controls a dynamic range of a resulting sum from the logic blocks; wherein the logic blocks are arranged to provide the option to apply no scale factor to a layer or to apply a scale factor individually to one or more layers; wherein each logic block receives an input value to modify and to provide to accumulate with modified input values of other ones of the logic blocks to be used to compute a single output of a single node of the neural network, and wherein the scale factor value applied to the input value at each logic block for the single output is the same. 29. A computer-implemented method of neural network propagation, comprising: receiving layer descriptors for individual primitive layers of a neural network, the individual layer descriptors indicating: a primitive layer type of the primitive layer and from a set of pre-determined primitive layer types that are available in a neural network, and a selected weight bit length to be used with the primitive layer and from a predetermined set of a plurality of available weight bit lengths for a layer in the neural network; receiving weight values for individual layers depending, at least in part, on the primitive layer type and with bit lengths depending on the selected weight bit length; inputting the weight values to fixed function hardware to perform a neural network computation that alternatively performs the computation with weights having one of the available bit lengths; and determining one or more outputs for the individual primitive layers depending, at least in part, on the primitive layer type. 30. The method of claim 29 comprising performing automatic speech recognition using the neural network, wherein inputs and outputs of the layers are associated with sounds or words, and wherein the outputs of an output layer of the neural network are associated with one or more recognized sounds or words. 31. The method of claim 29 wherein the available primitive layer types comprises at least a recurrent layer, a convolutional layer, and an affine layer. 32. The method of claim 29 wherein the available primitive layer types comprises at least one of an active list (AL) affine layer wherein less than all outputs of a layer are to be processed, and a diagonal affine layer where a diagonal matrix is used as the weight matrix. 33. The method of claim 29 wherein the available primitive layer types comprises a transpose layer comprising at least one of an interleave layer that provides groups of input elements that each form a single final output of a layer, and formed in the opposite direction of the row or column major storage of an input array so that input elements in corresponding positions on multiple groups are provided to an input buffer, or a de-interleave layer that holds the groups of input elements in the same direction as the row or column major storage and arranged to be processed group by group. 34. The method of claim 29 wherein the primitive layer types comprises: an recurrent layer; a convolutional layer; an affine layer; an active list affine layer that processes less than all outputs on a layer; a diagonal affine layer where a diagonal matrix is used as the weight matrix; an interleave layer that groups input elements into multiple groups that each form a final output of a single layer and to be processed across the multiple groups; a de-interleave layer that holds the groups of input elements arranged to be processed group by group; and a copy layer to concatenate input or output data of one group with that of another group. 35. The method of claim 29 comprising, by a fixed function hardware state machine: reading data of a layer descriptor of a current layer to be processed; and depending on the data of the layer descriptor, directing layer data of the current layer to and from the fixed function hardware performing the neural network computations. 36. The method of claim 29 comprising processing sequences of layers wherein an activation function on any one primitive layer of the sequence is not required to be the same as an activation function on another primitive layer of the sequence in order to use the fixed function hardware and to use the activation function to compute the one or more outputs of the primitive layer. 37. The method of claim 29 comprising performing a piecewise linear function to approximate a supported activation function to compute individual outputs of a layer and comprising performing a binary-type of search to locate a sum output computed by using the fixed function hardware and on a range of sum output values as potential inputs of a segmented piecewise linear function comprising comparing an input to threshold sum output values along the range of sum output values wherein each threshold is associated with a different segment of the segmented piecewise linear function and over a number of iterations eliminating half of the available sum output values of the range at individual iterations depending on whether the input is larger or smaller than the threshold sum output value of a segment. 38. The method of claim 29 comprising: receiving an array of sum output values each corresponding to an end of a different segment of a piecewise linear function having a plurality of segments, an activation function output associated with the individual ends, and a slope of the individual segments; determining which segment an input value belongs wherein the input is a sum output computed by using the fixed function hardware and comprising: in a plurality of iterations, comparing the input to a sum output value of the segment end and from the array, and comprising determining whether the input is larger or smaller than the sum output value; on individual iterations, eliminating the half of the remaining segments that are on the opposite side of the sum output value just compared to relative to the input; performing the iterations until the input is placed on a single segment; and determining the activation function output using an equation of the piecewise linear function at the single segment, and the determined sum output value. 39. The method of claim 29 wherein the piecewise linear activation function alternatively supports at least two of tan h, sigmoid, and rectifier linear unit (reLU) activation functions along the same sequence of layers. 40. The method of claim 29 comprising performing the neural network computations on fixed function hardware that forms a plurality of parallel logic blocks, wherein each block has substantially the same logic elements to multiply a weight value by an input value, and wherein the logic elements are to alternatively perform Gaussian mixture model (GMM) propagation. 41. A method of claim 29 wherein the primitive layer types comprises: an recurrent layer; a convolutional layer; an affine layer; an active list affine layer that processes less than all outputs on a layer; a diagonal affine layer where a diagonal matrix is used as the weight matrix; an interleave layer that groups input elements into multiple groups that each form a final output of a single layer and to be processed across the multiple groups; a de-interleave layer that holds the groups of input elements arranged to be processed group by group; and a copy layer to concatenate input or output data of one group with that of another group. the method comprising: by a fixed function hardware state machine: reading data of a layer descriptor of a current layer to be processed; and depending on the data of the layer descriptor, directing layer data of the current layer to and from the fixed function hardware performing the neural network computations; and processing sequences of layers wherein an activation function on any one primitive layer of the sequence is not required to be the same as an activation function on another primitive layer of the sequence in order to use the fixed function hardware and to use the activation function to compute the one or more outputs of the primitive layer. 42. A computer implemented system of neural network propagation comprising: fixed function hardware with logic elements to perform neural network computations; a sequencer state machine to direct layer data to and from the fixed function hardware, and operated by: receiving data of layer descriptors for individual primitive layers of a neural network, the individual layer descriptors indicating: a primitive layer type of the corresponding layer and from a set of pre-determined primitive layer types that are available in the neural network, and a selected weight bit length to be used with the corresponding layer and from a predetermined set of available weight bit lengths for a layer in the neural network; directing the placement of weight values for individual layers and into an internal buffer depending, at least in part, on the primitive layer type and with bit lengths depending on the selected weight bit length; input the weight values to the neural network circuit to perform a neural network computation that alternatively performs the computation with weights having one of the available bit lengths; and determine one or more outputs for the individual layers depending, at least in part, on the primitive layer type. 43. The system of claim 42 wherein the primitive layer types comprises at least two of: a recurrent layer, a convolutional layer, and an affine layer. 44. The system of claim 43 wherein the primitive layer types comprise: an active list affine layer that processes less than all outputs on a layer; a diagonal affine layer wherein a diagonal matrix is used as the weight matrix; an interleave layer that groups input elements into multiple groups that each form a final output of a single layer and to be processed across the multiple groups; a de-interleave layer that holds the groups of input elements arranged to be processed group by group. 45. The system of claim 42 wherein a single artificial macro neural network layer is to be divided into a plurality of the primitive layers each with a layer descriptor received by the neural network accelerator. 46. The system of claim 42 comprising a register state machine that holds one layer descriptor at a time and associated with a current layer being processed, and wherein the sequencer state machine reads the layer descriptor at the register state machine to direct layer data to and from the internal buffer. 47. The system of claim 42 an activation function unit to compute an output associated with the value of the neural network and using a piecewise linear function supported by more than one activation function layer type. 48. The system of claim 42 wherein the neural network circuit comprises a plurality of parallel logic blocks to compute a weighted input, each logic block to alternatively use weights of the available bit lengths; and wherein the logic elements of the neural network circuit are arranged to alternatively compute values for a Gaussian mixture model (GMM). 49. The system of claim 42 wherein comprising performing automatic speech recognition using the neural network, wherein inputs and outputs of the layers are associated with sounds or words, and wherein the outputs of an output layer of the neural network are associated with one or more recognized sounds or words. 50. The system of claim 42 wherein the primitive layer types comprises: an active list affine layer that processes less than all outputs on a layer; a diagonal affine layer wherein a diagonal matrix is used as the weight matrix; an interleave layer that groups input elements into multiple groups that each form a final output of a single layer and to be processed across the multiple groups; and a de-interleave layer that holds the groups of input elements arranged to be processed group by group; wherein a single artificial macro neural network layer is to be divided into a plurality of the primitive layers each with a layer descriptor received by the neural network accelerator; the system comprising: a register state machine that holds one layer descriptor at a time and associated with a current layer being processed, and wherein the sequencer state machine reads the layer descriptor at the register state machine to direct layer data to and from the internal buffer; and an activation function unit to compute an output associated with the value of the neural network and using a piecewise linear function supported by more than one activation function layer type.