first, followed by a more extensive and comprehensive learning procedure with similarity constraints to learn visual representations. We demonstrate that visual representations learned by our algorithm performs very competitively as compared to ImageNet trained CNNs.

Finally, our paper is also related to learning from weak or noisy labels [@crandall2006weakly; @pandey2011scene; @deselaers2012weakly; @song2014learning; @wang2014weakly]. There are some recent works showcasing that CNNs trained in a weakly supervised setting can also develop detailed information about the object intrinsically [@simonyan2013deep; @Oquab13; @pathak2014fully; @bergamo2014self; @papandreou2015weakly]. However, different from the assumptions in most weakly-supervised approaches, here our model is deprived of clean human supervision altogether (instead of only removing the location or segmentation). Most recently, novel loss layers have also been introduced in CNNs to deal with noisy labels [@sukhbaatar2014learning; @reed2014training]. On the other hand, we assume a vanilla CNN is robust to noise when trained with simple examples, from which a relationship graph can be learned, and this relationship graph provides powerful constraints when the network is faced with more challenging and noisier data.

Approach
========

Our goal is to learn deep representations directly from the massive amount of data online. While it seems that CNNs are designed for big data - small datasets 