<p>I suggest having a look at</p>

<blockquote>
  <p><a href="https://arxiv.org/abs/1503.03832" rel="nofollow noreferrer">FaceNet: A Unified Embedding for Face Recognition and Clustering</a></p>
</blockquote>

<p>My <a href="http://www.shortscience.org/paper?bibtexKey=journals/corr/1503.03832" rel="nofollow noreferrer">shortscience summary</a> (go there if you want to see the Math parts rendered correctly):</p>

<p>FaceNet directly maps face images to $\mathbb{R}^{128}$ where distances directly correspond to a measure of face similarity. They use a triplet loss function. The triplet is (face of person A, other face of person A, face of person which is not A). Later, this is called (anchor, positive, negative).</p>

<p>The loss function is learned and inspired by LMNN. The idea is to minimize the distance between the two images of the same person and maximize the distance to the other persons image.</p>

<h3>LMNN</h3>

<p>Large Margin Nearest Neighbor (LMNN) is learning a pseudo-metric</p>

<p>$$d(x, y) = (x -y) M  (x -y)^T$$</p>

<p>where $M$ is a positive-definite matrix. The only difference between a pseudo-metric and a metric is that $d(x, y) = 0 \Leftrightarrow x = y$ does not hold.</p>

<h2>Curriculum Learning: Triplet selection</h2>

<p>Show simple examples first, then increase the difficulty. This is done by selecting the triplets.</p>

<p>They use the triplets which are <em>hard</em>. For the positive example, this means the distance between the anchor and the positive example is high. For the negative example this means the distance between the anchor and the negative example is low.</p>

<p>They want to have</p>

<p>$$||f(x_i^a) - f(x_i^p)||_2^2 + \alpha &lt; ||f(x_i^a) - f(x_i^n)||_2^2$$</p>

<p>where $\alpha$ is a margin and $x_i^a$ is the anchor, $x_i^p$ is the positive face example and $x_i^n$ is the negative example. They increase $\alpha$ over time. It is crucial that $f$ maps the images not in the complete $\mathbb{R}^{128}$, but on the unit sphere. Otherwise one could double $\alpha$ by simply making $f' = 2 \cdot f$.</p>

<h3>Tasks</h3>

<ul>
<li><strong>Face verification</strong>: Is this the same person?</li>
<li><strong>Face recognition</strong>: Who is this person?</li>
</ul>

<h3>Datasets</h3>

<ul>
<li>99.63% accuracy on Labeled FAces in the Wild (LFW)</li>
<li>95.12% accuracy on YouTube Faces DB</li>
</ul>

<h3>Network</h3>

<p>Two models are evaluated: The <a href="http://www.shortscience.org/paper?bibtexKey=journals/corr/ZeilerF13" rel="nofollow noreferrer">Zeiler &amp; Fergus model</a>  and an architecture based on the <a href="http://www.shortscience.org/paper?bibtexKey=journals/corr/SzegedyLJSRAEVR14" rel="nofollow noreferrer">Inception model</a>.</p>

<h2>See also</h2>

<ul>
<li><a href="http://www.shortscience.org/paper?bibtexKey=conf/cvpr/TaigmanYRW14#martinthoma" rel="nofollow noreferrer">DeepFace</a></li>
</ul>

<h2>See also</h2>

<ul>
<li><a href="https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf" rel="nofollow noreferrer">DeepFace: Closing the Gap to Human-Level Performance in Face Verification</a></li>
</ul>
Clustering human faces from a video
<p>I have run the face detection algorithm inbuilt in opencv to extract faces in each frame of a video(sampled at 1 fps). I have also resized each face image to be of same size and I have cropped some fraction of image to remove background noise and hair. Now the problem is that I have to cluster these images of faces - Each cluster corresponding to a person. I implemented the algorithm described here <a href="http://bitsearch.blogspot.in/2013/02/unsupervised-face-clustering-with-opencv.html" rel="nofollow">http://bitsearch.blogspot.in/2013/02/unsupervised-face-clustering-with-opencv.html</a></p>

<p>Basically the above algorithm, uses LBPH face recognizer of OpenCV iteratively to cluster the images. In the description on that page itself the results are not satisfactory. In my implementation the results are worse. Can anyone suggest a better way to cluster faces? May be using some other feature and some other clustering algorithm. The number of clusters are unknown. </p>