title: How to build a Face recognition system from scratch?
body: <p>I am building a prototype for a <strong>face recognition system</strong> and while writing the algorithm, I had a few questions.</p>
  
  <p><strong>Algorithm:</strong></p>
  
  <ol>
  <li><p>Collect pair of (A(i),P(i),N(i)) -set of the anchor, positive, negative images of employees working at XYX company. </p></li>
  <li><p>Using gradient descent train the <strong>Triplet loss function</strong> to learn CNN parameters. Actually, here I am training a <a href="https://en.wikipedia.org/wiki/Siamese_network" rel="nofollow noreferrer">Siamese network</a>(Idea of running two identical CNNs' on 2 different inputs[one time on A(i)-P(i) and next A(i)-N(i)] and then comparing them). </p>
  
  <blockquote>
    <p><a href="https://i.stack.imgur.com/lGNdH.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/lGNdH.png" alt="enter image description here"></a>
    <a href="https://i.stack.imgur.com/nLfK2.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/nLfK2.png" alt="enter image description here"></a>
    These learned parameters will ensure that the distance between the flattened n-dim encoding of the same images would be small and different image would be large.! </p>
  </blockquote></li>
  <li><p>Now, create a database wherein you will store the encoding of each training image of XYX company's employees! </p>
  
  <blockquote>
    <p>Simply make a forward pass through the trained CNN and store the corresponding encoding of each image in the database </p>
  </blockquote></li>
  <li><p>At test time, you have the image of an XYX company's employee and image of an outsider! </p>
  
  <blockquote>
    <ul>
    <li><p>You will pass both of the test images through the CNN and get the corresponding encodings! </p></li>
    <li><p>Now, <strong>The question comes</strong> that how would you find the similarity between the test-picture-encoding and all the training-picture-encoding in the database? </p>
    
    <ul>
    <li><p><strong>First question</strong>, Would you do cosine similarity or I need to do something else? Can you add more clarity on it? </p></li>
    <li><p><strong>Second question</strong>, Also, in terms of efficiency, how would you handle a scenario wherein you have 100,000 employees training-picture-encoding in the database present and for every new person you need to look these 100,000 encodings and compute cosine similarity and give result in &lt;2 secs? Any suggestion on this part?</p></li>
    </ul></li>
    </ul>
  </blockquote>
  
  <ul>
  <li><strong>Third question</strong> usually for face recognition task if we use approach(Image-->CNN-->SoftMax--> output), Each time a new person joins your organization you need to retrain your network, that's why it's a bad approach! </li>
  </ul>
  
  <blockquote>
    <ul>
    <li><p>This problem can be mitigated by using the 2nd approach wherein we are using a learned distance function "d(img1, img2)" over a pair of images of employees as stated above on in <strong>point 1 to 3</strong>. </p></li>
    <li><ul>
    <li><strong>My question</strong> is in case of a new employee joining the organization, How this learned distance function would be able to generalize when it was not been used in the training set at all? Isn't a problem of changed data distribution of test and train set? Any suggestion in this regards </li>
    </ul></li>
    </ul>
  </blockquote></li>
  </ol>
  
  <p>Could anyone help in understanding these conceptual glitches? </p>