instances (wall: 20,213, window: 16,080, chair: 7,971) while most have a relatively modest number of instances (boat: 349, airplane: 179, floor lamp: 276). In our dataset, we ensure that each object category has a significant number of instances, Fig. \[fig:dataanalysis\].

[**Other vision datasets** ]{} Datasets have spurred the advancement of numerous fields in computer vision. Some notable datasets include the Middlebury datasets for stereo vision [@scharstein2002taxonomy], multi-view stereo [@seitz2006comparison] and optical flow [@baker2011database]. The Berkeley Segmentation Data Set (BSDS500) [@amfm_pami2011] has been used extensively to evaluate both segmentation and edge detection algorithms. Datasets have also been created to recognize both scene [@Patterson2012SunAttributes] and object attributes [@farhadi2009describing; @lampert2009learning]. Indeed, numerous areas of vision have benefited from challenging datasets that helped catalyze progress.

![image](figs/ui_pipeline_summary){width="\textwidth"}

Image Collection {#sec:image_collection}
================

We next describe how the object categories and candidate images are selected.

Common Object Categories
------------------------

The selection of object categories is a non-trivial exercise. The categories must form a representative set of all categories, be relevant to practical applications and occur with high enough frequency to enable the collection of a large dataset. Other important decisions are whether to include both “thing” and “stuff” categories [@heitz2008learning] and whether fine-grained [@Birds200; @Imagenet] and object-part 