The present disclosure provides a new scalable coordinate descent (SCD) algorithm and associated system for generalized linear models whose convergence behavior is always the same, regardless of how much SCD is scaled out and regardless of the computing environment. This makes SCD highly robust and enables it to scale to massive datasets on low-cost commodity servers. According to one aspect, by using a natural partitioning of parameters into blocks, updates can be performed in parallel a block at a time without compromising convergence. Experimental results on a real advertising dataset are used to demonstrate SCD's cost effectiveness and scalability.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A computer-implemented method, comprising: obtaining, by one or more computing machines, a training dataset useful to machine learn a plurality of model parameters, the training dataset comprising a plurality of examples, each example comprising entries for a plurality of features, and wherein the plurality of features respectively correspond to the plurality of model parameters; partitioning, by the one or more computing machines, the plurality of features into a plurality of blocks, wherein at least one of the plurality of blocks contains at least two features. 2. The computer-implemented method of claim 1, wherein partitioning, by the one or more computing machines, the plurality of features into the plurality of blocks comprises partitioning, by the one or more computing machines, the plurality of features into the plurality of blocks, at least one of which comprises a pure block, and wherein, in each pure block, a respective feature subvector for each example has at most one non-zero entry. 3. The computer-implemented method of claim 1, wherein: each of the plurality of features corresponds to one of a plurality of variables; and partitioning, by the one or more computing machines, the plurality of features into the plurality of blocks comprises partitioning, by the one or more computing machines, the plurality of features according to the plurality of variables, such that the number of blocks equals the number of variables, and such that each of the plurality of blocks includes the respective features that correspond to a respective one of the plurality of variables. 4. The computer-implemented method of claim 1, wherein: each of the plurality of features corresponds to one of a plurality of variables, the plurality of variables comprising one or more categorical variables and one or more set-valued variables; and partitioning, by the one or more computing machines, the plurality of features into the plurality of blocks comprises: for each of the one or more categorical variables, partitioning, by the one or more computing machines, the features that correspond to the one or more categorical variables according to the one or more categorical variables, such that each resulting block includes the respective features that correspond to a respective one of the one or more categorical variables; and for each of the one or more set-valued variables, partitioning, by the one or more computing machines, the features that correspond to the one or more set-valued variables into at least two blocks per set-valued variable, such that each resulting block includes, on average, one active feature per example. 5. The computer-implemented method of claim 1, further comprising: learning, by the one or more computing machines, the model parameters from the plurality of blocks, wherein learning the model parameters comprises performing, by the one or more computing machines, a plurality of iterations, and wherein performing the plurality of iterations comprises processing, by the one or more computing machines, only one of the plurality of blocks per iteration. 6. The computer-implemented method of claim 5, wherein learning, by the one or more computing machines, the model parameters from the plurality of blocks further comprises: sharding, by the one or more computing machines, the plurality of examples into a plurality of example shards; and at each iteration, respectively processing in parallel, by a plurality of worker computing machines, the plurality of example shards for one of the plurality of blocks. 7. A distributed computing system to perform large-scale machine learning, the system comprising: a master computing machine that includes at least one processing device; and a plurality of worker computing machines under control of the master computing machine, wherein each worker computing machine includes at least one processing device; wherein the system is configured to: partition a plurality of features of a training dataset into a plurality of blocks, wherein the training dataset comprises a plurality of examples, wherein each example comprises entries for the plurality of features, and wherein at least one of the plurality of blocks comprises at least two features; and perform a plurality of iterations of a machine learning technique to learn a plurality of parameters of a model from the plurality of blocks, wherein the plurality of parameters respectively correspond to the plurality of features, and wherein the system is configured to process only one of the plurality of blocks per iteration. 8. The distributed computing system of claim 7, wherein to perform each iteration, the system is configured to: select, by the master computing machine, one of the plurality of blocks; determine, by the worker computing machines, sufficient statistics for the selected block; aggregate the determined sufficient statistics; update the parameters that correspond to the selected block based on the aggregated sufficient statistics; and update a plurality of predictions respectively for the plurality of updated parameters. 9. The distributed computing system of claim 8, wherein, at each iteration, the master computing machine selects the one of the plurality of blocks based on one or more heuristics that estimate which block will have the largest impact on convergence. 10. The distributed computing system of claim 8, wherein to perform each iteration, the system is further configured to: determine, by the master computing machine, a step size after the sufficient statistics are aggregated but before the parameters of the model are updated. 11. The distributed computing system of claim 8, wherein to determine sufficient statistics for the selected block, the plurality of worker computing machines are configured to: obtain respective shards of the plurality of examples for the selected block; and respectively determine sufficient statistics for the respective shards in parallel. 12. The distributed computing system of claim 11, wherein the master computing machine attempts to assign the same respective shard to each worker computing machine in each iteration so as to improve caching. 13. The distributed computing system of claim 11, wherein each worker computing machine is further configured to prefetch the respective shard for the next iteration so as to reduce the impact of tail latencies. 14. The distributed computing system of claim 8, wherein to determine sufficient statistics for the selected block, at least one of the plurality of worker computing machines is configured to: obtain at least one of the respective shard, one or more labels for the respective shard, and one or more of the plurality of predictions for the respective shard from at least one of a first cache maintained in a first memory of such worker computing machine and a second cache maintained in a second memory of at least one other worker computing machine. 15. The distributed computing system of claim 8, wherein to aggregate the determined sufficient statistics, the system is configured to: partition the determined sufficient statistics into a plurality of ranges; assign, by the master computing machine, the plurality of ranges to respective ones of the plurality of worker computing machines; fetch, by each worker computing machine to which one of the ranges was assigned, the sufficient statistics for the corresponding range from the other worker computing machines; aggregate, by each worker computing machine to which one of the ranges was assigned, the sufficient statistics for the corresponding range; and transmit, by each worker computing machine to which one of the ranges was assigned, the aggregated sufficient statistics for the corresponding range to the master computing machine. 16. The distributed computing system of claim 8, wherein: each worker computing machine is configured to implement a plurality of worker threads that execute work items in parallel; and each worker computing machine is configured to: maintain at least a two-level memory store that comprises a single worker-level store and a plurality of thread-level stores respectively for the plurality of worker threads that such worker computing machine implements; store at each thread-level store a first number of the sufficient statistics that correspond to the most frequently observed features; and store at the worker-level store a second number of the sufficient statistics that correspond to infrequently observed features and that are not stored at the thread-level stores; wherein the two-level memory store enables lock-free updates of the sufficient statistics. 17. A structured numerical representation that represents at least a plurality of parameters of a linear model, the structured numerical representation produced by a process comprising: obtaining, by a distributed computing system, a training dataset, the training dataset comprising a plurality of examples, each example comprising entries for a plurality of features, and wherein the plurality of features respectively correspond to the plurality of parameters of the linear model; partitioning, by the distributed computing system, the plurality of features into a plurality of blocks, wherein at least one of the plurality of blocks contains at least two features; and performing, by the distributed computing system, a plurality of iterations of a machine learning technique to learn values for the plurality of parameters of the linear model from the plurality of blocks; wherein performing the plurality of iterations comprises processing, by the distributed computing system, only one of the plurality of blocks per iteration. 18. The structured numerical representation of claim 17, wherein partitioning, by the distributed computing system, the plurality of features into the plurality of blocks comprises partitioning, by the distributed computing system, the plurality of features into a plurality of pure blocks in which a respective feature subvector for each example has at most one non-zero entry. 19. The structured numerical representation of claim 17, wherein: each of the plurality of features corresponds to one of a plurality of variables; and partitioning, by the distributed computing system, the plurality of features into the plurality of blocks comprises partitioning, by the distributed computing system, the plurality of features according to the plurality of variables, such that the number of blocks equals the number of variables, and such that each of the plurality of blocks includes the respective features that correspond to a respective one of the plurality of variables. 20. The structured numerical representation of claim 17, wherein performing, by the distributed computing system, the plurality of iterations of the machine learning technique comprises, for each iteration: selecting, by a master computing machine of the distributed computing system, one of the plurality of blocks; obtaining, by a plurality of worker computing machines of the distributed computing system, respective shards of the plurality of examples for the selected block; respectively determining, by the plurality of worker computing machines in parallel, sufficient statistics for the respective shards; aggregating, by the distributed computing system, the determined sufficient statistics; updating, by the distributed computing system, the parameters that correspond to the selected block based on the aggregated sufficient statistics; and updating, by the distributed computing system, a plurality of predictions respectively for the plurality of updated parameters.