O slideshow foi denunciado.

Deep Learning for Computer Vision: Face Recognition (UPC 2016)

3.082 visualizações

Publicada em


Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.

Publicada em: Dados e análise
  • How can I get a flat tummy naturally? ◆◆◆ http://ishbv.com/bkfitness3/pdf
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui

Deep Learning for Computer Vision: Face Recognition (UPC 2016)

  1. 1. Day 3 Lecture 5 Face Recognition Acknowledments: Prof. Ramon Morros Students: Gerard Martí (Ms CV), Carlos Roig (Bs Tel) and Alessandro Vilardi (Ms EE) Elisa Sayrol [course site]
  2. 2. Face Recognition •Face Detection •Face Alignment/ Frontalization •Face Recognition •Face Identification (Classification) •Face Verification(Binary Decision) •People Recognition in videos (Camomile Project at UPC) •Speech •Face •Text 2
  3. 3. Face Recognition •Databases •Well-Known Systems •Deep Face (FaceBook) •FaceNet (Google) •Deep ID • Some experiments at UPC 3
  4. 4. Databases YouTube Faces: [http://www.cs.tau.ac.il/~wolf/ytfaces/] 621126 pictures, 1595 identities (celebrities). Images come from videos so there is not a lot of variability between them. May overlap with other celebrity databases Available info: Original frames, cropped faces, aligned faces. Head-pose angles for all the faces faceScrub: [http://vintage.winklerbros.net/facescrub.html] [http://megaface.cs.washington.edu/participate/challenge.html] 106863 photos of 530 celebrities, 265 whom are male (55306 images), and 265 female (51557 images). Face bounding boxes provided. Full frame and cropped version available. MegaFace: [http://megaface.cs.washington.edu/] 1 milion faces, 690572 unique people MSRA-CFW [http://research.microsoft.com/en-us/projects/msra-cfw/] 202792 faces, 1583 people (celebrities). May overlap with other celebrity databases. Links, has to be downloaded (downloading!). 4
  5. 5. Databases Labeled Faces in the Wild (LFW) [http://vis-www.cs.umass.edu/lfw/ ] 13,000 images of faces collected from the web, 1680 of the people pictured have two or more distinct photos in the data set. CelebFaces [http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html ] 202599 face images of 10177 identities (celebrities). People in LFW and CelebFaces+ are mutually exclusive. 10k US Adult Faces [http://www.wilmabainbridge.com/facememorability2.html] ~10000 images, ?? people (celebrities excluded manually). /work/morros/faces/facedatabase/ CASIA [http://www.cbsr.ia.ac.cn/english/CASIA-WebFace-Database.html] 494,414 images, 10,575 subjects. GoogleUPC !!! 5
  6. 6. DeepFace Architecture Yaniv Taigman, etc (Facebook) . DeepFace: Closing the Gap to Human-Level Performance in Face Verification, CVPR 2014 6
  7. 7. DeepFace, Verification *S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity met-ric discriminatively, with application to face verification, CVPR,2005. B) Use of Siamese Networks inspired in Chopra et al* A) Weighted χ2 distance where f1 and f2 are the DeepFace Representations. The weights parameters wi are learned using a linear SVM In DeepFace: αi are the trainable parameters with standard cross-entropy loss and backward propagation 7
  8. 8. FaceNet Florian Schroff et al. (Google) FaceNet: A Unified Embedding for Face Recognition and Clustering, CVPR 2015 This Face recognition/verification/clustering model learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Triplet Loss function: Where f is the embedding 8
  9. 9. Deep ID2 Parameters of the Network: But you compute parameters From a Verification loss function and an Identification loss Function Yi Sun, etc. Deep Learning Face Representation by Joint Identification-Verification. NIPS 2014 9
  10. 10. Deep ID2 When you backprop you backprop gradients of verification and identification parameters and you also update the weight of the convolutional layers 10
  11. 11. Deep ID2 DeepID2 Uses a Joint Bayesian model in top of the network for face verification. If we model a face as Verification is achieved through Log-Likelihood Ratio Test: Interpersonal variations Intrapersonal variations Both Gaussian Distributed, estimated during Training Chen, et al. Bayesian Face Revisited: A Joint Formulation, ECCV 2012 11
  12. 12. Comparing Face recognition Deep Face results Deep Face CNN 4,4 million images 4030 people 93% accuracy DeepID2 202,599 images 10,177 people 98,9% accuracy Imagenet Fine-Tuning 10.422 images 520 people 97,3% accuracy DeepID2 results Imagenet Fine-Tuning results (Msc Sergi Delgado)12 Experiments at UPC Face recognition (2015) 12
  13. 13. Essex Dataset Crops from TV show videos Our own database to be used in the Camomile EU Project - 520 instances composed by 10.422 images. - 8858 images were used in training stage, and 1.564 for testing. Experiments at UPC Face recognition (2015) 13
  14. 14. Face Recognition using Very Deep Neural Networks • VGG • GoogleNet • ResNet • Ensenble VGG+GoogleNet Pre-trained Networks with VGG-Imagenet or VGG-Faces. Google Net and ResNet pretrained over Imagenet. Experiments with YouTube Faces, FaceScrub and Google UPC Faces Ongoing experiments at UPC Face recognition (2016) Ramon Morros Students Carlos Roig (Bs in Tel), Alessandro Vilardi (Ms in EE), Gerard Martí (Ms in CV) 14
  15. 15. Experiments with a compound DataBase with YouTube Faces, FaceScrub and LFW 3.500 identities 100.000 images With VGG Ongoing experiments at UPC Face recognition (2016) Ramon Morros Students Carlos Roig (Bs in Tel), Alessandro Vilardi (Ms in EE), Gerard Martí (Ms in CV) 15
  16. 16. Verification with: VGG+Autoencoder with 8 hidden layers to reduce dimensionality, from 4096 to 256 vector+ Joint Bayesian Results with DataBase 1 (the previous one without YTF, and a Test set with FaceScrub and LFW) Precision Recall f1-score support (pairs of the dataset) 0.97 0.95 0.96 2288 Results with DataBase 2 (the previous one without YTF, and a Validation set of LFW) Precision Recall f1-score support(pairs of the dataset) 0.80 0.80 0.79 998 Undergoing Experiments also with Advanced Joint Bayesian, Siamese networks, Triplets…. Ongoing experiments at UPC Face recognition (2016) Ramon Morros Students Carlos Roig (Bs in Tel), Alessandro Vilardi (Ms in EE), Gerard Martí (Ms in CV) 16