A machine learning hardware accelerator architecture and associated techniques are disclosed. The architecture features multiple memory banks of very wide SRAM that may be concurrently accessed by a large number of parallel operational units. Each operational unit supports an instruction set specific to machine learning, including optimizations for performing tensor operations and convolutions. Optimized addressing, an optimized shift reader and variations on a multicast network that permutes and copies data and associates with an operational unit that support those operations are also disclosed.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A system to perform hardware acceleration for machine learning, comprising: multiple banks of random access memory; a shift reader; a plurality of operational units configured to access at least a portion of the random access memory concurrently; a multicast network communicatively coupled to the memory and the shift reader; and an instruction decoder comprised of an accumulator, one or more registers, and a random number generator, wherein the instruction decoder is configured to process instructions restricted to an instruction set comprised machine learning primitives, configured to perform memory transfers and dispatch instructions to the operational units. 2. The system of claim 1, wherein the random access memory is comprised of static random access memory and is configured to be accessed with a memory width sufficient to perform vector operations with a single access. 3. The system of claim 1, wherein each bank of random access memory is coupled with shift readers sufficient to perform shifts such that the shift readers need not change the address of the memory rotating the entire bank of random access memory. 4. The system of claim 1, further comprising a plurality of index registers, wherein the instructions of the instruction set are configured to calculate memory addresses from the index registers and update the index registers within the atomic operation of the instruction. 5. The system of claim 1, wherein the shift reader is configured to perform a shift operation between the random access memory banks via: simultaneously reading data from a plurality of the memory banks based on a logical offset; concatenating the read data read into a single logical list; rotating the read data in the single logical list based on the logical offset and the width of memory that may be read in a single operation; and shifting the read data based on the logical offset and the width of memory that may be read in a single operation. 6. The system of claim 1, wherein the multicast network is configured to permute data and copy data from the shift reader and the memory via operation of the operational units. 7. The system of claim 6, wherein the multicast network is comprised of a plurality of permutation layers and a plurality of duplication layers, the permutation layers configured to perform atomic transpose operations of tensor tiles. 8. The system of claim 6, wherein the multicast network is a broadcast rearrangeable non-blocking network with 2×2 switching units, each unit retaining state as to a permutation or broadcast function to be performed, such that the network may be dynamically configured to an arbitrary multicast algorithm. 9. The system of claim 6, wherein the multicast network is a is a parity rearrangeable non-blocking network such that a full set of switching nodes is configured based at least on a set of parameters less than the number of switching nodes and a node index. 10. The system of claim 6, wherein the multicast network is a is a non-broadcast rearrangeable non-blocking network such that wherein duplication is performed via simple duplication. 11. The system of claim 1, further comprising a transfer unit configured to perform wide bulk copy operations between non-local memory and the local random access memory. 12. The system of claim 1, wherein the operational units are configured to have an accumulator with a memory width sufficient to perform vector operations with a single access. 13. The system of claim 1, wherein the operational units are configured with a pseudo-random number generator and a pseudo-random number register. 14. The system of claim 13, wherein the operational units are configured to process an atomic random noise generation instruction by accessing the pseudo-random number generator. 15. The system of claim 1, wherein the operational units are configured to process an atomic multiply and accumulate maximum (MMA) operation. 16. The system of claim 1, wherein the operational units are configured to perform floating point operations with a mantissa precision of less than 24 bits. 17. A method to perform machine learning operations with a machine learning hardware accelerator, comprising: fetching an instruction based on an instruction pointer; updating index registers based at least on the fetched instruction; computing addresses for a transfer of memory between local memory and non-local memory based at least on the fetched instruction; computing addresses for a first tensor element and a second tensor element based at least on the fetched instruction; loading data at the addresses of the first tensor element and the second tensor element into a plurality of banks of random access memory based at least on the fetched instruction; propagating the loaded data via shift loaders based at least on the fetched instruction; propagating the loaded data to a plurality of operational units via a multicast network; and performing at the operational units, operations based at least on the fetched instruction. 18. The method of claim 17 comprising performing an index register test after the index registers are updated. 19. The method of claim 17 wherein the fetched instruction is from a set of instructions limited to atomic operations to perform tensor operations, noise generation, and non-linear operations. 20. A system to perform hardware acceleration for machine learning, comprising: multiple banks of random access memory; a shift reader; a plurality of operational units configured to access at least a portion of the random access memory concurrently; a multicast network communicatively coupled to the memory and the shift reader; and an instruction decoder, wherein the instruction decoder is configured to process instructions restricted to an instruction set comprised of the following operations: multiply accumulate, add random noise, perform linear transform, shift right, compare range and update, output data, generate seed random number, and return from call.