images [^2].

For fine-tuning the network with hard Flickr images, we again feed these images as-is for training, with the tags as class labels. While we are getting more realistic images, we did notice that the data becomes noisier. Powerful as CNNs, they are still likely to be diluted by the noisy examples over the fine-tuning process[^3]. In an noisy open-domain environment, mistakes are unavoidable. But humans are more intelligent: we not just learn to recognize concepts independently, but also build up interconnections and develop theories to help better understand the world [@carruthers1996theories]. Inspired by this, we want to train CNNs with such relationships - with their simplest form being pair-wise look-alike ones [@chen2013neil; @divvala2014learning]. Such a relationship graph can provide more information of the class and regularize/constrain the network training. A motivating example is “iphone”. While Google mostly returns images of the product, on Flickr it is often used to specify the device a photo is taken with - as a result, virtually any image can be tagged as “iphone”. Knowing similar-looking categories to “iphone” can intuitively help here.

One way to obtain relationships is through extra knowledge sources like WordNet [@miller1995wordnet]. However, they are not necessarily developed for 