The present teaching relates to distributed deep machine learning on a cluster. In one example, a request is received for estimating one or more parameters associated with a machine learning model on a cluster including a plurality of nodes. A set of data is obtained to be used for estimating the one or more parameters. The set of data is divided into a plurality of sub-sets of data, each of which corresponds to one of the plurality of nodes. Each sub-set of data is allocated to a corresponding node for estimating values of the one or more parameters based on the sub-set of data. Estimated values of the one or more parameters obtained based on a corresponding sub-set of data allocated to the node, are received from each of the plurality of nodes. The one or more parameters of the machine learning model are estimated based on the estimated values of the one or more parameters generated by at least some of the plurality of nodes.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method implemented on at least one computing device each of which has at least one processor, storage, and a communication platform connected to a network for estimating one or more parameters on a cluster including a plurality of nodes, the method comprising: receiving a request for estimating one or more parameters associated with a machine learning model; obtaining a set of data to be used for estimating the one or more parameters; dividing the set of data into a plurality of sub-sets of data, each of which corresponds to one of the plurality of nodes; allocating each sub-set of data to a corresponding node for estimating values of the one or more parameters based on the sub-set of data; receiving, from each of the plurality of nodes, estimated values of the one or more parameters obtained based on a corresponding sub-set of data allocated to the node; and estimating the one or more parameters of the machine learning model based on the estimated values of the one or more parameters generated by at least some of the plurality of nodes. 2. The method of claim 1, wherein each of the at least some of the plurality of nodes performs the following: obtaining training values related to the one or more parameters; generating updated training values based on a corresponding sub-set of data; exchanging information related to the updated training values with other nodes in the at least some of the plurality of nodes to generate a state of the one or more parameters that is shared by the at least some of the plurality of nodes; and repeating the steps of generating and exchanging until a predetermined condition is met to generate the estimated values of the one or more parameters. 3. The method of claim 1, wherein each of the plurality of nodes performs the following: allocating one or more processing units in accordance with the request; generating, by each processing unit within the node, a subset of values related to the one or more parameters of the machine learning model; and synchronizing the one or more processing units within the node to generate the estimated values of the one or more parameters of the machine learning model. 4. The method of claim 3, further comprising: detecting a failure at one of the plurality of nodes; and instructing the remaining nodes in the plurality of nodes to continue the estimating. 5. The method of claim 3, further comprising determining a number of the one or more processing units for each node in accordance with the request, wherein the one or more processing units within each node include at least one graphics processing unit (GPU) or central processing unit (CPU) device. 6. The method of claim 1, further comprising: determining a number of the plurality of nodes in accordance with the request; and determining a location of the set of data for estimating the one or more parameters associated with the machine learning model based on the request. 7. The method of claim 1, further comprising instructing one of the at least some of the plurality of nodes to store a snapshot of a state of the one or more parameters of the machine learning model in a storage outside the plurality of nodes. 8. The method of claim 7, further comprising: detecting a failure of the estimating; instructing the plurality of nodes to retrieve a snapshot of a last state of the one or more parameters of the machine learning model from the storage; and instructing the plurality of nodes to continue the estimating based on the last state. 9. The method claim 1, wherein the plurality of nodes are synchronized via MPI AllReduce like API using a network interface implemented on each of the plurality of nodes, wherein the network interface includes at least one of an Ethernet interface and an Infiniband interface. 10. The method of claim 1, wherein: the set of data for estimating the one or more parameters is stored on a Hadoop Distributed File System (HDFS); and the plurality of nodes are implemented in a Spark framework. 11. A system, having at least one processor, storage, and a communication platform connected to a network for estimating one or more parameters on a cluster including a plurality of nodes, the system comprising: a configuration information identifier configured for receiving a request for estimating one or more parameters associated with a machine learning model; a training data locator configured for obtaining a set of data to be used for estimating the one or more parameters; a training data distributor configured for dividing the set of data into a plurality of sub-sets of data, each of which corresponds to one of the plurality of nodes and allocating each sub-set of data to a corresponding node for estimating values of the one or more parameters based on the sub-set of data; and a training model determiner configured for receiving, from each of the plurality of nodes, estimated values of the one or more parameters obtained based on a corresponding sub-set of data allocated to the node and estimating the one or more parameters of the machine learning model based on the estimated values of the one or more parameters generated by at least some of the plurality of nodes. 12. The system of claim 11, wherein each of the at least some of the plurality of nodes performs the following: obtaining training values related to the one or more parameters; generating updated training values based on a corresponding sub-set of data; exchanging information related to the updated training values with other nodes in the at least some of the plurality of nodes to generate a state of the one or more parameters that is shared by the at least some of the plurality of nodes; and repeating the steps of generating and exchanging until a predetermined condition is met to generate the estimated values of the one or more parameters. 13. The system of claim 11, wherein each of the plurality of nodes performs the following: allocating one or more processing units in accordance with the request; generating, by each processing unit within the node, a subset of values related to the one or more parameters of the machine learning model; and synchronizing the one or more processing units within the node to generate the estimated values of the one or more parameters of the machine learning model. 14. The system of claim 13, further comprising: a node failure detector configured for detecting a failure at one of the plurality of nodes; and a training instruction generator configured for instructing the remaining nodes in the plurality of nodes to continue the estimating. 15. The system of claim 13, further comprising an operation node selector configured for determining a number of the one or more processing units for each node in accordance with the request, wherein the one or more processing units within each node include at least one graphics processing unit (GPU). 16. The system of claim 11, further comprising: an operation node selector configured for determining a number of the plurality of nodes in accordance with the request; and the training data locator configured for determining a location of the set of data for estimating the one or more parameters associated with the machine learning model based on the request. 17. The system of claim 11, further comprising a training instruction generator configured for instructing one of the at least some of the plurality of nodes to store a snapshot of a state of the one or more parameters of the machine learning model in a storage outside the plurality of nodes. 18. The system of claim 17, further comprising: a node failure detector configured for detecting a failure of the estimating; a training instruction generator configured for instructing the plurality of nodes to retrieve a snapshot of a last state of the one or more parameters of the machine learning model from the storage and instructing the plurality of nodes to continue the estimating based on the last state. 19. The system of claim 11, wherein the plurality of nodes are synchronized via MPI AllReduce like API using a network interface implemented on each of the plurality of nodes, wherein the network interface includes at least one of an Ethernet interface and an Infiniband interface. 20. A machine-readable tangible and non-transitory medium having information for estimating one or more parameters on a cluster including a plurality of nodes, wherein the information, when read by the machine, causes the machine to perform the following: receiving a request for estimating one or more parameters associated with a machine learning model; obtaining a set of data to be used for estimating the one or more parameters; dividing the set of data into a plurality of sub-sets of data, each of which corresponds to one of the plurality of nodes; allocating each sub-set of data to a corresponding node for estimating values of the one or more parameters based on the sub-set of data; receiving, from each of the plurality of nodes, estimated values of the one or more parameters obtained based on a corresponding sub-set of data allocated to the node; and estimating the one or more parameters of the machine learning model based on the estimated values of the one or more parameters generated by at least some of the plurality of nodes. 21. The method of claim 1, wherein the cluster is a single cluster that enables a deep machine learning to be performed along with a non-deep machine learning and other data processing in the single cluster, with a single program. 22. The method of claim 21, wherein: some data processing is performed in the cluster for producing input datasets for the deep machine learning; the deep machine learning is performed in the cluster for extracting features; the non-deep machine learning is performed in the cluster to generate a classification model based on the features; and the other data processing in the cluster includes applying the classification model against a big data set. 23. The method of claim 1, wherein: the cluster is an Apache Hadoop cluster on top of Apache Spark; and the plurality of nodes in the cluster are capable of communicating with each other via an API similar to MPI AllReduce. 24. The method of claim 1, wherein each of the at least some of the plurality of nodes performs the following: collecting, by one processing unit in the node, the estimated values from other processing units in the node to generate a reduced estimate of the one or more parameters based on the estimated values, wherein the one or more parameters are represented by a vector; dividing the vector into a plurality of fractions, wherein each of the at least some of the plurality of nodes corresponds to one of the plurality of fractions; collecting, by the one processing unit in the node, reduced estimates of a corresponding fraction from other nodes in the at least some of the plurality of nodes to generate an all reduced estimate of the corresponding fraction based on the reduced estimates; broadcasting, by the one processing unit in the node, the all reduced estimate of the corresponding fraction to the other nodes in the at least some of the plurality of nodes; obtaining, by the one processing unit in the node, an all reduced estimate of the vector representing the one or more parameters; and broadcasting, by the one processing unit in the node, the all reduced estimate of the vector to the other processing units in the node.