A novel entity resolution approach for the organization entity domain can be implemented in the MapReduce framework with low memory requirements so that it may scale to large scale datasets. A new clustering approach, sClust, significantly improves the recall of the pairwise classifier.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A machine learning system comprising: a distributed memory arrangement that provides sorted edge lists for each connected component; and a distributed processing arrangement coupled to the distributed memory arrangement, the distributed processing arrangement being configured to generate node structures and conflict sets in a first pass, and in a second pass to agglomeratively construct clusters for scores exceeding a match threshold. 2. The machine learning system of claim 1 wherein the distributed processing arrangement comprises a name node, a job submission node and plural of slave nodes. 3. The machine learning system of claim 2 wherein the plural slave nodes each include a processor that executes at least one mapper and at least one reducer. 4. The machine learning system of claim 3 wherein the plural slave nodes each include local memory coupled to the processor. 5. The machine learning system of claim 1 wherein the distributed processing arrangement is further configured to perform graph-based blocking. 6. The machine learning system of claim 1 wherein for each connected component CiεC, there exists a path in G between any two vertices vk and vl where (vk,vl)εCi. 7. The machine learning system of claim 8 wherein for any distinct connected component (Ci,Cf)εC, there is no path between any pair vk and vl where vkεCi,vlεCj. 8. The machine learning system of claim 1 wherein the distributed processing arrangement is further configured to recursively partition oversized blocks into sub-blocks by sub-blocking properties that the records they contain share, and append the properties to a key. 9. The machine learning system of claim 8 wherein the recursive partitioning uses mapping to extract tokens from organization input records, and emits combined organization identifiers and extracted tokens or sublevel tokens and current blocking keys to a reducer. 10. The machine learning system of claim 1 wherein the distributed processing system is further configured to perform binomial tree blocking where separate mapper processes work on different nodes in a level of the binomial tree without creating redundant sub-blocks.