high recall, 8 workers were asked to label each image. A category is considered present if any worker indicated the category; false positives are handled in subsequent stages. A detailed analysis of performance is presented in §\[sec:annotation-performance\]. This stage took $\sim$20k worker hours to complete.

Instance Spotting
-----------------

\[sec:instance-spotting\] In the next stage all instances of the object categories in an image were labeled, Fig. \[fig:pipeline\](b). In the previous stage each worker labeled one instance of a category, but multiple object instances may exist. Therefore, for each image, a worker was asked to place a cross on top of each instance of a specific category found in the previous stage. To boost recall, the location of the instance found by a worker in the previous stage was shown to the current worker. Such priming helped workers quickly find an initial instance upon first seeing the image. The workers could also use a magnifying glass to find small instances. Each worker was asked to label at most 10 instances of a given category per image. Each image was labeled by 8 workers for a total of $\sim$10k worker hours.

Instance Segmentation
---------------------

\[sec:instance-segmentation\] Our final stage is the laborious task of 