a category (in terms of relationships to other categories), and the error is back-propagated through the relationship graph to lower layers. Note that, this extra layer is similar to [@sukhbaatar2014learning], in which $R_{ij}$ is used to characterize the label-flip noise. Different from them, we do not assume all the categories are *mutually exclusive*, but instead *inter related*. For example, “cat” is a hyper-class of “Siamese cat”, and it is reasonable if the model believes some examples of “Siamese cat” are more close to the average image of a “cat”. Please see Section \[sec:exp\] for our empirical validation of this assumption. For fear of semantic drift, in this paper we keep the initially learned graph structure fixed, but it would be interesting to see how updating the relationship graph performs (like [@chen2013neil]).

![[Our pipeline of object localization (for “countryman”). E-LDA detectors [@hariharan2012discriminative] trained on $fc7$ features of the seed images are fired on EdgeBox proposals (purple boxes) from other images for nearest neighbors (red boxes), which are then merged to form subcategories. Noisy subcategories are purged with density estimation [@chen_cvpr14]. \[fig:loc\]]{}](loc){width="0.97\linewidth"}

\[sec:loc\]Localizing Objects
-----------------------------

Until now, we have focused on learning a webly-supervised CNN representation based on classification loss. In order 