Systems and methods are disclosed for parallel machine learning with a cluster of N parallel machine network nodes by determining k network nodes as a subset of the N network nodes to update learning parameters, wherein k is selected to disseminate the updates across all nodes directly or indirectly and to optimize predetermined goals including freshness, balanced communication and computation ratio in the cluster; sending learning unit updates to fewer nodes to reduce communication costs with learning convergence; and sending reduced learning updates and ensuring that the nodes send/receive learning updates in a uniform fashion.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method for machine learning, comprising: with a cluster of N parallel machine network nodes, determining k network nodes as a subset of the N network nodes to update learning parameters, wherein k is selected to disseminate the updates across all nodes directly or indirectly and to optimize predetermined goals including freshness, balanced communication and computation ratio in the cluster; sending learning unit updates to fewer nodes to reduce communication costs with learning convergence; and sending reduced learning updates and ensuring that the nodes send/receive learning updates in a uniform fashion. 2. The method of claim 1, comprising training the parallel network nodes in a peer-peer fashion with per-sender receive queues. 3. The method of claim 1, comprising performing uniform dissemination of network node updates with Halton series. 4. The method of claim 3, wherein each learning node is selected in a uniform fashion to ensure that updates are distributed across all nodes. 5. The method of claim 3, wherein each learning node propagates updates to k nodes, comprising picking k node IDs based on a uniform random sequence that generates successive points that uniformly cover an ID range. 6. The method of claim 5, wherein the sequence comprises a Halton sequence. 7. The method of claim 6, comprising maintain at each network node list of log(N) nodes and wherein the node list contains nodes in a Halton series to send updates. 8. The method of claim 1, comprising marking each network node as 1 . . . N, wherein node 1 sends updates to a Halton series including N/2, N/4, 3N/4, N/8, 3N/8, 5N/8. 9. The method of claim 1, comprising sending total updates in every iteration with a 2. The method of claim 1, comprising training the parallel network nodes in a peer-peer fashion with per-sender receive queues. 3. The method of claim 1, comprising performing uniform dissemination of network node updates with Halton series. 4. The method of claim 3, wherein each learning node is selected in a uniform fashion to ensure that updates are distributed across all nodes. 5. The method of claim 3, wherein each learning node propagates updates to k nodes, comprising picking k node IDs based on a uniform random sequence that generates successive points that uniformly cover an ID range. 6. The method of claim 5, wherein the sequence comprises a Halton sequence. 7. The method of claim 6, comprising maintain at each network node list of log(N) nodes and wherein the node list contains nodes in a Halton series to send updates. 8. The method of claim 1, comprising marking each network node as 1 . . . N, wherein node 1 sends updates to a Halton series including N/2, N/4, 3N/4, N/8, 3N/8, 5N/8. 9. The method of claim 1, comprising sending total updates in every iteration with a complexity of O(N log(N)). 10. The method of claim 1, comprising ensuring updates are sent uniformly across a range of nodes using any uniform random sequence other than Halton. 11. The method of claim 1, comprising: picking a value for a number of outbound communication nodes; and balancing computation with communication in parallel learning node. 12. The method of claim 1, comprising balancing communication and computation costs in the cluster. 13. The method of claim 12, comprising overlapping communication with computation in the cluster for maximum concurrency. 14. The method of claim 1, comprising choosing ‘k’ nodes such that a time to communicate k updates is equal to a time to compute model updates for next iteration. 15. A machine learning system, comprising: a plurality of parallel networked nodes each including a processor; a network card coupling the processor to the other learning machine nodes; and computer readable code executable by the processor for: determining k nodes as a subset of nodes to update model updates with 1<=k<=N, wherein k is chosen to disseminate the updates across all nodes directly or indirectly and to optimize predetermined goals of the system including freshness, balanced communication and computation ratio in the cluster; and sending model updates to fewer nodes reduces communication costs but the overall model still converges; sending reduced model updates and ensuring that models send/receive model updates in a uniform fashion. 16. The system of claim 15, wherein the parallel network nodes communicate in a peer-peer fashion with per-sender receive queues. 17. The system of claim 15, wherein the nodes perform uniform dissemination of network node updates with Halton series. 18. The system of claim 17, wherein each learning node is selected in a uniform fashion to ensure that updates are distributed across all nodes. 19. The system of claim 17, wherein each learning node propagates updates to k nodes, comprising picking k node IDs based on a uniform random sequence that generates successive points that uniformly cover an ID range and wherein each network node is marked as 1 . . . N, and node 1 sends updates to a Halton series including N/2, N/4, 3N/4, N/8, 3N/8, 5N/8. 20. The system of claim 15, comprising a vector library to create unit parameters or gradients (updates to parameters) to be synchronized across network nodes and the vectors communicate over a DiSTributed One-sided Remote Memory (dstorm).