Technologies for a human computation framework suitable for answering common sense questions that are difficult for computers to answer but easy for humans to answer. The technologies support solving general common sense problems without a priori knowledge of the problems; support for determining whether an answer is from a bot or human so as to screen out spurious answers from bots; support for distilling answers collected from human users to ensure high quality solutions to the questions asked; and support for preventing malicious elements in or out of the system from attacking other system elements or contaminating the solutions produced by the system, and preventing users from being compensated without contributing answers.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method performed on a computing device, the method comprising: selecting, by the computing device, a common-sense problem from a first source; receiving, by the computing device, answers to the common-sense problem from a second source; identifying, by the computing device, any of the received answers that are arbitrary answers; removing, by the computing device, the identified arbitrary answers from the received answers; and designating, by the computing device in response to the removing, as final answers any remaining received answers. 2. The method of claim 1 further comprising sending, in response to the designating, the final answers to the first source. 3. The method of claim 1 where the computing device is configured for performing the method without a priori knowledge of the common-sense problem. 4. The method of claim 1 further comprising inhibiting compensation to a source that does not contribute an answer to the common-sense problem. 5. The method of claim 1 where the first source and the second source are the same source. 6. The method of claim 1 where the second source comprises at least one human. 7. The method of claim 1 where the identifying the arbitrary answers is based on modeling the arbitrary answers as a uniform distribution. 8. A computing device comprising: memory; a processor coupled to the memory and via which the computing device: orders answers according to their frequency of occurrence; determines a relative difference for each neighboring pair of the ordered answers, the relative distance based on the frequency of occurrence of each ordered answer of the each neighboring pair; and designates as final answers any of the ordered answers that have a frequency of occurrence that is greater than a frequency of occurrence of an ordered answer of a neighboring pair that has a greatest relative distance of the neighboring pairs. 9. The computing device of claim 8 where the relative distance is determined based on calculating a slope. 10. The computing device of claim 8 where the answers are directed to labeling an image. 11. The computing device of claim 10 where the labeling comprises a process including a plurality of refining stages. 12. The computing device of claim 11 where the plurality of refining stages comprise collecting candidate labels. 13. The computing device of claim 12 where the plurality of refining stages comprise further refining the candidate labels based on multiple choices. 14. The computing device of claim 12 where the plurality of refining stages comprise further refining based on locating an object in the image that corresponds to at least one of the refined candidate labels 15. At least one computer storage device that comprises computer-executable instructions that, based on execution by a computing device, configure the computing device to perform actions comprising: ordering, by a computing device, answers according to their frequency of occurrence; determining, by a computing device, a relative difference for each neighboring pair of the ordered answers, the relative distance based on the frequency of occurrence of each ordered answer of the each neighboring pair; and designating, by a computing device, as final answers any of the ordered answers that have a frequency of occurrence that is greater than a frequency of occurrence of an ordered answer of a neighboring pair that has a greatest relative distance of the neighboring pairs. 16. The at least one computer storage device of claim 15 where the determining the relative distance comprises calculating a slope. 17. The at least one computer storage device of claim 15 where the answers are directed to labeling an image. 18. The at least one computer storage device of claim 17 where the labeling comprises a process including a plurality of refining stages. 19. The at least one computer storage device of claim 18 where the plurality of refining stages comprise collecting candidate labels. 20. The at least one computer storage device of claim 19 where the plurality of refining stages comprise further refining the candidate labels based on multiple choices, and further refining based on locating an object in the image that corresponds to at least one of the refined candidate labels.