본문 바로가기
KERD
논문KCI 등재

A Comprehensive Survey of Lightweight Neural Networks for Face Recognition

양재경|장영립
한국산업경영시스템학회 (KOREAN SOCIETY OF INDUSTRIAL AND SYSTEMS ENGINEERING)|2023.03|한국산업경영시스템학회지 (Journal of Korean Society of Industrial and Systems Engineering)|제46권 제1호|pp. 55-67

0

조회

0

다운로드

0

인용

0

참고문헌

키워드

Face RecognitionMobile DeviceLightweight NetworksEfficient Face Models
주제분류산업공학

초록

Lightweight face recognition models, as one of the most popular and long-standing topics in the field of computer vision, has achieved vigorous development and has been widely used in many real-world applications due to fewer number of parameters, lower floating-point operations, and smaller model size. However, few surveys reviewed lightweight models and reimplemented these lightweight models by using the same calculating resource and training dataset. In this survey article, we present a compre- hensive review about the recent research advances on the end-to-end efficient lightweight face recognition models and reimplement several of the most popular models. To start with, we introduce the overview of face recognition with lightweight models. Then, based on the construction of models, we categorize the lightweight models into: (1) artificially designing lightweight FR models,(2) pruned models to face recognition, (3) efficient automatic neural network architecture design based on neural architecture searching, (4) Knowledge distillation and (5) low-rank decomposition. As an example, we also introduce the SqueezeFaceNetand EfficientFaceNet by pruning SqueezeNet and EfficientNet. Additionally, we reimplement and present a detailed performance comparison of different lightweight models on the nine different test benchmarks. At last, the challenges and future works areprovided. There are three main contributions in our survey: firstly, the categorized lightweight models can be conveniently identifiedso that we can explore new lightweight models for face recognition; secondly, the comprehensive performance comparisons are carried out so that ones can choose models when a state-of-the-art end-to-end face recognition system is deployed on mobile devices; thirdly, the challenges and future trends are stated to inspire our future works.

Abstract

Lightweight face recognition models, as one of the most popular and long-standing topics in the field of computer vision, has achieved vigorous development and has been widely used in many real-world applications due to fewer number of parameters, lower floating-point operations, and smaller model size. However, few surveys reviewed lightweight models and reimplemented these lightweight models by using the same calculating resource and training dataset. In this survey article, we present a compre- hensive review about the recent research advances on the end-to-end efficient lightweight face recognition models and reimplement several of the most popular models. To start with, we introduce the overview of face recognition with lightweight models. Then, based on the construction of models, we categorize the lightweight models into: (1) artificially designing lightweight FR models,(2) pruned models to face recognition, (3) efficient automatic neural network architecture design based on neural architecture searching, (4) Knowledge distillation and (5) low-rank decomposition. As an example, we also introduce the SqueezeFaceNetand EfficientFaceNet by pruning SqueezeNet and EfficientNet. Additionally, we reimplement and present a detailed performance comparison of different lightweight models on the nine different test benchmarks. At last, the challenges and future works areprovided. There are three main contributions in our survey: firstly, the categorized lightweight models can be conveniently identifiedso that we can explore new lightweight models for face recognition; secondly, the comprehensive performance comparisons are carried out so that ones can choose models when a state-of-the-art end-to-end face recognition system is deployed on mobile devices; thirdly, the challenges and future trends are stated to inspire our future works.

참고문헌 (90)

  1. Abate, A. F., "2D and 3D face recognition : A survey", Pattern Recognition Letters, 28(14), pp.1885~1906 (2007)
  2. Adjabi, I., "and future of face recognition : A review", Electronics, 9(8), pp.1188~ (2020)
  3. Alonso-Fernandez, F., "SqueezeFacePoseNet: Lightweight Face Verification Across Different Poses for Mobile Platforms", International Conference on Pattern Recognition, pp.139~153 (2021)
  4. Belhumeur, P. N., "Eigenfaces vs. fisherfaces: Recognition using class spe-cific linear projection", IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), pp.711~720 (1997)
  5. Boutros F., "QuantFace : Towards lightweight face recognition by synthetic data low-bit quantization", arXiv preprint arXiv:2206.10526, 855~862 (2022)
  6. Boutros, F., "Mixfacenets: Extremely efficient face recognition networks", 2021 IEEE International Joint Conference on Biometrics (IJCB), pp.1~8 (2021)
  7. Boutros, F., "PocketNet: extreme lightweight face recognition network using neural archi-tecture search and multistep knowledge distillation", IEEEAccess, 46823~46833 (2022)
  8. Cai, H., "Proxylessnas : Direct neural architecture search on target task and hardware", arXiv preprint arXiv : 1812. 00332, ~ (2018)
  9. Cao, Q., "Vggface2: A dataset for recognising faces across pose and age", 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) (2018)
  10. Cao, Z., "Face recognition with learning-based descriptor", 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.2707~2714 (2010)
  11. Chen S., "Mobilefacenets:Efficient cnns for accurate real-time face verification on mobile devices", Chinese Conference on Biometric Recognition, pp.428~438 (2018)
  12. Chen, K., "LightQNet : Lightweight Deep Face Quality Assessment for Risk-Controlled Face Recognition", IEEE Signal Processing Letters, 28, pp.1878~1882 (2021)
  13. Deng, J., "Imagenet : A large-scale hierarchical image database", 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.248~255 (2009)
  14. Deng, J., "Arcface:Additive angular margin loss for deep face recognition", Proceedings of the IEEE, 2019, 4690~4699
  15. Deng, J., "Lightweight face recognition challenge", Proceedings of the IEEE, 2019, ~
  16. Deng, W., "Extended SRC : Undersampled face recognition via intraclass variant dic-tionary", IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(9), pp.1864~1870 (2012)
  17. Du, H., "The elements of end-to-end deep face recognition: A survey of recent advances", ACM Computing Surveys (CSUR), 54(10s), pp.1~42 (2022)
  18. Duong, C. N., "Shrinkteanet : Million-scale lightweight face recognition via shrinking teacher-student networks", arXiv preprint arXiv : 1905. 10620, ~ (2019)
  19. Duong, C. N., "Mobiface : A lightweight deep learning face recog-nition on mobile devices", 2019 IEEE 10th International Conference on Biometrics Theory, pp.1~6 (2019)
  20. Feng, Y., "Triplet distillation for deep face recognition", 2020 IEEE International Conference on Image Processing (ICIP), pp.808~812 (2020)
  21. Fu, C., "Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence (2022)
  22. Ge, S., "Low-resolution face recognition in the wild via selective knowledge dis-tillation", IEEE Transactions on Image Processing, 28(4), pp.2051~2062 (2018)
  23. Ge, S., "Efficient low-resolution face recognition via bridge distillation", IEEE Transactions on Image Processing, 29, pp.6898~6908 (2020)
  24. Guo, L., "A lightweight and robust face recognition network on noisy condition", 2019Asia-Pacific Signal and Information Processing Associa-tion Annual Summit and Conference (APSIPA ASC), pp.1964~1969 (2019)
  25. Guo, Y., "Ms-cel-eb-1m: A dataset and benchmark for large-scale face recognition", European Conference on Computer Vision, pp.87~102 (2016)
  26. He, K., "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
  27. He, X., "Face recognition using laplacianfaces", IEEE transactions on Pattern Analysis and Machine Intelligence, 27(3), pp.328~340 (2005)
  28. Hinton, G., "Distilling the knowl-edge in a neural network", arXiv preprint arXiv:1503. 02531 2.7, ~ (2015)
  29. Hoo, S. C., "ConvFaceNeXt:Lightweight Networks for Face Recognition", Mathematics, 10(19) (2022)
  30. Howard, A. G., "Mobilenets : Efficient convolutional neural networks for mobile vision applications", arXiv preprint arXiv : 1704. 04861, ~ (2017)
  31. Huang, G, "Densely connected convolutional networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.4700~ (2017)
  32. Huang, G. B., "Labeled faces in the wild: A database forstudying face recognition in unconstrained environments", Workshop on faces in'Real-Life'Images: detection, align-ment, and recognition (2008)
  33. Iandola, F. N., "SqueezeNet : AlexNet-level accuracy with 50x fewer parameters and< 0. 5 MB model size", arXiv preprint arXiv : 1602. 07360, ~ (2016)
  34. Rabia Jafri, "A Survey of Face Recognition Techniques", JIPS(Journal of Information Processing Systems), 5(2), pp.41~68 (2009), 한국정보처리학회
  35. Kim, M., "AdaFace: Quality Adaptive Margin for Face Recognition", Proceedings of the IEEE, 2022, 18750~18759
  36. Krizhevsky, A., "Imagenet classification with deep convolutional neural networks", Communications of the ACM, 60(6), pp.84~90 (2017)
  37. LeCun, Y., "Handwritten digit recog-nition with a back-propagation network", Advances in Neural Information Processing Systems, 2, pp.396~404 (1989)
  38. Lei, Z., "Learning discrim-inant face descriptor", IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2), pp.289~302 (2013)
  39. Li, X., "Airface : Lightweight and efficient model for face recognition", Proceedings of the IEEE, 2019, ~
  40. Li, Z., "Gp-nas: Gaussian process based neural architecture search", Proceedings of the IEEE, 2020, 11933~11942
  41. Lin, M., "Network in network", arXiv preprint arXiv : 1312. 4400, ~ (2013)
  42. Liu, C., "Gabor feature based classi-fication using the enhanced fisher linear discriminant model for face recognition", IEEE Transactions on Image processing, 11(4), pp.467~476 (2002)
  43. Liu, W., "Sphereface: Deep hypersphere embedding for face recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.212~220 (2017)
  44. Liu, W., "Face recognition based on lightweight convolutional neural networks", Information, 12(5) (2021)[DOI]
  45. Lyu, Y., "Factorizing and reconstituting large-kernel MBConv for lightweight face recognition", Proceedings of the IEEE, 2019, ~
  46. Ma, N., "Shufflenet v2: Practical guidelines for efficient cnn architecture de-sign", Proceedings of the European conference on com-puter vision (ECCV), pp.116~131 (2018)
  47. Ma, Y., "Effective methods for lightweight image-based and video-based face recognition", Proceedings of the IEEE, 2019, ~
  48. Martindez-Diaz, Y., "Shufflefacenet : A lightweight face architecture for efficient and highly-accurate face recognition", Proceed-ings of the IEEE, 2019, ~
  49. Martinez-Diaz, Y., "Benchmarking lightweight face archi-tectures on specific face recognition scenarios", Artificial Intelligence Review, pp.1~44 (2021)
  50. Masi, I., "Deep face recognition: A survey", 2018 31st SIBGRAPI Conference on Graphics, pp.471~478 (2018)
  51. Maze, B., "Iarpa janus benchmark-c:Face dataset and protocol", 2018 International Conference on Biometrics, pp.158~165 (2018)
  52. Micikevicius P., "Mixed precision training", arXiv preprint arXiv : 1710. 03740, ~ (2017)
  53. Moghaddam, B., "Bayesian face recognition", Pattern recognition, 33(11), pp.1771~1782 (2000)
  54. Moschoglou, S., "Agedb: the first manually collected, in-thewild age database", 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, pp.51~59 (2017)
  55. Parkhi, O. M., "Deep face recognition", ~ (2015)
  56. Paszke, A., "Automatic differentiation in pytorch", NeurIPS workshop (2017)
  57. Romero, A., "Fitnets : Hints for thin deep nets", arXiv preprint arXiv : 1412. 6550, ~ (2014)
  58. Russakovsky, O., "Imagenet large scale visual recognition challenge", International Journal of Computer Vision, 115, pp.211~252 (2015)
  59. Sandler, M., "Mobilenetv2:Inverted residuals and linear bottlenecks", Proceedings of the IEEE conference on com-puter vision and pattern recognition, pp.4510~4520 (2018)
  60. Schroff, F., "Facenet : A unified embedding for face recognition and clustering", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)
  61. Sengupta, S., "Frontal to profile face verification in the wild", 2016 IEEE Winter Conference on Applications of Computer Vision(WACV), pp.1~9 (2016)
  62. Simonyan, K., "Very deep convolu-tional networks for large-scale image recognition", arXiv preprint arXiv : 1409. 1556, ~ (2014)
  63. Sun, , Y., "Deeply learned face representations are sparse, selective, and robust", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2892~2900 (2015)
  64. Sun, Y., "Deep learning face representation by joint identification-verification", Advances in Neural Information Processing Systems, 27 (2014)
  65. Sun, Y., "Deepid3 : Face recognition with very deep neural networks", arXiv preprint arXiv : 1502. 00873, ~ (2015)
  66. Sun, Y., "Deep learning face representation from predicting10,000 classes", Proceed-ings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1891~1898 (1891)
  67. Szegedy, C., "Going deeper with convolutions", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1~9 (2015)
  68. Taigman, Y., "Deepface: Closing the gap to human-level performance in face verification", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1701~1708 (2014)
  69. Tan, M., "Mixconv : Mixed depthwise convolu-tional kernels", arXiv preprint arXiv : 1907. 09595, ~ (2019)
  70. Tan, M., "Efficientnet: Rethinking model scaling for convolutional neural networks", International Conference on Machine Learning, pp.6105~6114 (2019)
  71. Turk, M., "Eigenfaces for recognition", Journal of cognitive neuroscience, 3(1), pp.71~86 (1991)
  72. Wang, M., "Deep face recognition : A survey", Neurocomputing, 429, pp.215~244 (2021)
  73. Wang, H., "Cosface: Large margin cosine loss for deep face recognition", Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.5265~5274 (2018)
  74. Wang, X., "Exclusivity-consistency regularized knowledge dis-tillation for face recognition", European Conference on Computer Vision, pp.325~342 (2020)
  75. Wang, X., "Teacher guided neural architecture search for face recognition", Proceedings of the AAAI Conference on Artificial Intelligence, 35(4), pp.2817~2825 (2021)
  76. Whitelam, C., "Iarpa janus benchmark-b face dataset", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.90~98 (2017)
  77. Wright, J., "Robust face recognition via sparse representation", IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2), pp.210~227 (2008)
  78. Wu, X., "A light CNN for deep face representation with noisy labels", IEEE Transactions on Information Forensics and Security, 13(11), pp.2884~2896 (2018)
  79. Xiao, J., "A Lightweight Face Recognition Model based on MobileFaceNet for Limited Computation Environment", EAI Endorsed Transactions on Internet of Things, 7(27), pp.1~9 (2021)
  80. Yan, M., "Vargfacenet : An efficient variable group convolu-tional neural network for lightweight face recognition", Proceedings of the IEEE, 2019, ~
  81. Yang, S., "Sparse Individual Low-Rank Component Repre-sentation for Face Recognition in the IoT-Based System", IEEE Internet of Things Journal, 8(24), pp.17320~17332 (2021)
  82. Zagoruyko, S., "Paying more attention to attention : Improving the performance of convolutional neural networks via attention transfer", arXiv preprint arXiv : 1612. 03928, ~ (2016)
  83. Zhang, L., "Sparse representation or collaborative representation: Which helps face recog-nition?", 2011 International Conference on Computer Vision, pp.471~478 (2011)
  84. Zhang, P., "Efficient LightweightAttention Network for Face Recognition", IEEEAccess, 10, pp.31740~31750 (2022)
  85. Zhang, Q., "Vargnet : Variable group convolutional neural network for efficient em-bedded computing", arXiv preprint arXiv : 1907. 05653, ~ (2019)
  86. Zhao, J., "A novel deep learning algorithm for incomplete face recognition : Low-rank-recovery network", Neural Networks, 94, pp.115~124 (2017)
  87. Zhao, W., "Face recognition : A literature survey", ACM Computing Surveys(CSUR), 35(4), pp.399~458 (2003)
  88. Zheng, T., "Cross-age LFW: A database for studying cross-age face recognition in uncon-strained environments", arXiv preprint arXiv:1708.08197, ~ (2017)
  89. Zhu, N., "A new deep neural archi-tecture search pipeline for face recognition", IEEE Access, 8, pp.91303~91310 (2020)
  90. Zoph, B., "Neural architecture search with reinforcement learning", arXiv preprint arXiv : 1611. 01578, ~ (2016)

관련 논문AI추천

Small Sample Face Recognition Algorithm based on Novel Siamese NetworkArun Kumar Sangaiah, Jianming Zhang, Jin Wang, Xiaokang Jin, Yukai Liu한국정보처리학회 · JIPS(Journal of Information Processing Systems)2018 · 제14권 제6호A Multi-Scale Parallel Convolutional Neural Network Based Intelligent Human Identification Using Face InformationChen Li, Ke Xiao, Mengti Liang, Wei Song한국정보처리학회 · JIPS(Journal of Information Processing Systems)2018 · 제14권 제6호A Survey of Face Recognition TechniquesHamid R. Arabnia, Rabia Jafri한국정보처리학회 · JIPS(Journal of Information Processing Systems)2009 · 제5권 제2호A Facial Expression Recognition Method Using Two-Stream Convolutional Networks in Natural ScenesLixin Zhao한국정보처리학회 · JIPS(Journal of Information Processing Systems)2021 · 제17권 제2호Improvement of Face Recognition Algorithm for Residential Area Surveillance System Based on Graph Convolution Network담하의, 민병원한국사물인터넷학회 · 사물인터넷융복합논문지2024 · 제10권 제2호Facial Expression Recognition Method Based on Residual Masking Reconstruction NetworkHongmei Li, Jianing Shen한국정보처리학회 · JIPS(Journal of Information Processing Systems)2023 · 제19권 제3호

이 간행물의 다른 논문 (11건)

내외부 압력이 ESG 활동과 경영성과에 미치는 영향에 관한 연구박태양, 김종대pp. 1-14다변수 순회 판매원 문제를 위한 퍼지 로직 개미집단 최적화 알고리즘전규범, 이종환, 이병길pp. 15-22머신러닝을 통한 잉크 필요량 예측 알고리즘태현철, 현영주, 권세욱pp. 23-31재무분야 감성사전 구축을 위한 자동화된 감성학습 알고리즘 개발조수지, 이기광, 양철원pp. 32-41Fractal 차원을 이용한 워게임에서의 도시화조정계수 추정권오정, 김재오, 조남석, 김동철pp. 42-47
원문 보기