Reference to the Andrew Deep Learning Course. It was told by instructor that the recognition system is more challenging than a verification system so following is the transcript of the video I pasted below

"So, the recognition problem is much harder than the verification problem. To see why, let's say, you have a verification system that's 99 percent accurate. So, 99 percent might not be too bad but now suppose that K is equal to 100 in a recognition system. If you apply this system to a recognition task with a 100 people in your database, you now have a hundred times of chance of making a mistake and if the chance of making mistakes on each person is just one percent. So, if you have a database of a 100 persons and if you want an acceptable recognition error, you might actually need a verification system with maybe 99.9 or even higher accuracy before you can run it on a database of 100 persons that have a high chance and still have a high chance of getting incorrect. In fact, if you have a database of 100 persons currently just be even quite a bit higher than 99 percent for that to work well. "

I did not get as to what the instructor was trying to say? Can somebody explain that with some simple example?

  • 1
    $\begingroup$ Is he talking about basic probability? The chance of an outcome is 1 %, but if you repeat the experiment 100 times then the chance is not equal to 1 % anymore but is higher? $\endgroup$ – user2974951 Sep 18 '18 at 11:23

I tried to watch the video to understand exactly what he means by "verification" and "recognition" but unfortunately the video only shows the preview (only first half of the video). So I would give my own understanding of the two systems.

Given an image of a face A, a "verification" system tries to answer the question: does the image of the face A match the person A, who is known to the system? On average, a verification system with 99% accuracy (or 1% error rate) would make mistake once out of every 100 predictions.

On the other hand, "recognition" system tries to answer: who among the K=100 people matches the image of the face A? (None of them is also a possible answer). A simple program may just repeat the "verification" task on K=100 known people, and return the answer for those persons that the verification task says yes. Ideally you would have one correct answer, but due to imperfection in the system, sometimes you may end up with the wrong answer whose face is that. Sometimes it returns even two or more faces. If you repeat a not-so-good verification system above (with error rate 1%) for 100 times, on average, the error rate on the recognition system is 100x1%=1.

With that understanding, I think the difficulty of recognition system boils down to the difference in number of prediction classes. Verification problem is a binary classification with two classes whereas recognition problem is a classification problem with K labels, with K as the number of the possible identities to return. K could be very large (imagine an company of 10000 employees).

  • $\begingroup$ I got your point but given that we have 99% accuracy in the verification system you did not explain as to why we need more than 99% accuracy in recognition system? $\endgroup$ – Naseer Ahmed Sep 19 '18 at 2:35
  • $\begingroup$ 99% accuracy is assumed here the "acceptable error rate" in the text, be it recognition or verification. With a recognition system built on top of a 99%-accuracy (1% error) verification system would have 100x1%=100% error rate. You need a 0.01% error (or more than 99% accuracy) verification to achieve 1% error rate (or 99% accuracy) in recognition. $\endgroup$ – Nelson Dinh Sep 19 '18 at 8:05

Your Answer

