Behind the Mask: Detection and Recognition Based-on Deep Learning

https://doi.org/10.22146/ijccs.72075

Ade Nurhopipah(1*), Irfan Rifai Azziz(2), Jali Suhaman(3)

(1) Department of Informatics, Universitas Amikom Purwokerto, Purwokerto
(2) Department of Informatics, Universitas Amikom Purwokerto, Purwokerto
(3) Department of Informatics, Universitas Amikom Purwokerto, Purwokerto
(*) Corresponding Author

Abstract


COVID-19 prevention procedures are executed to support public services and business continuity in a pandemic situation. Manual mask use monitoring is not efficient as it requires resources to monitor people at all times. Therefore, this task can be supported by automated surveillance systems based on Deep Learning. We performed mask detection and face recognition for a real-environment dataset. YOLOV3 as a one-stage detector was implemented to simultaneously generate a bounding box of the face area and class prediction. In face recognition, we compared the performance of three pre-trained models, namely ResNet152V2, InceptionV3, and Xception. The mask detection showed promising results with MAP=0.8960 on training and MAP=0.8957 on validation. We chose the Xception model for face recognition because it has equal quality as ResNet152V2 but has fewer parameters. Xception achieved a minimal loss value in the validation of 0.09157 with perfect accuracy on facial images larger than 100 pixels. Overall the system delivers promising results and can identify faces, even those behind the mask.

Keywords


Deep Learning; face recognition; mask detection; pre-trained model; YOLO

Full Text:

PDF


References

E. Noyes, J. P. Davis, N. Petrov, K. L. H. Gray, and K. L. Ritchie, “The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers,” R. Soc. Open Sci., vol. 8, no. 3, pp. 1–18, 2021. https://doi.org/10.1098/rsos.201169

E. Freud, A. Stajduhar, R. S. Rosenbaum, G. Avidan, and T. Ganel, “The COVID-19 pandemic masks the way people perceive faces,” Sci. Rep., vol. 10, no. 22344, pp. 1–8, 2020. https://doi.org/10.1038/s41598-020-78986-9

M. Marini, A. Ansani, F. Paglieri, F. Caruana, and M. Viola, “The impact of facemasks on emotion recognition, trust attribution and re-identification,” Sci. Rep., vol. 11, no. 5577, pp. 1–14, 2021. https://doi.org/10.1038/s41598-021-84806-5

C. Gupta and Nasib Singh Gill, “Coronamask: A Face Mask Detector for Real-Time Data,” Int. J. Adv. Trends Comput. Sci. Eng., vol. 9, no. 4, pp. 5624–5630, 2020. https://doi.org/10.30534/ijatcse/2020/212942020

M. Sujaritha, S. Kabilan, M. Manikandan, and S. N. Kisore, “Real Time Face Mask Identification Using Deep Learning,” J. Phys. Conf. Ser., vol. 1916, no. 012077, pp. 1–10, 2021. https://doi.org/10.1088/1742-6596/1916/1/012077

S. Ge, J. Li, Q. Ye, and Z. Luo, “Detecting Masked Faces in The Wild with LLE-CNNs,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 426–434, 2017. https://doi.org/10.1109/CVPR.2017.53

M. Loey, G. Manogaran, M. H. N. Taha, and N. E. M.Khalifa, “A Hybrid Deep Transfer Learning Model with Machine Learning Methods for Face Mask Detection in The Era of The COVID-19 Pandemic,” Measurement, vol. 167, 2020. https://doi.org/10.1155/2021/4529107

R. Katari, Sreekar Kaza, B. RamyaSree, V. Divyavani, and M. AbubakarJ, “A Comparative Analysis of Variant Deep Learning Models for COVID-19 Protective Face Mask Detection,” Turkish J. Comput. Math. Educ., vol. 12, no. 6, pp. 2841–2848, 2021. https://doi.org/10.17762/turcomat.v12i6.5791

W. Hariri, “Efficient masked face recognition method during the COVID-19 pandemic,” Signal, Image Video Process., 2021. https://doi.org/10.1007/s11760-021-02050-w

A. S. Joshi, S. S. Joshi, G. Kanahasabai, R. Kapil, and S. Gupta, “Deep Learning Framework to Detect Face Masks from Video Footage,” in Proceedings - 2020 12th International Conference on Computational Intelligence and Communication Networks, CICN 2020, 2020, pp. 435–440. https://doi.org/10.1109/CICN49253.2020.9242625

S. Sethi, M. Kathuria, and T. Kaushik, “Face mask detection using deep learning: An approach to reduce risk of Coronavirus spread,” J. Biomed. Inform., vol. 120, no. 103848, 2021. https://doi.org/10.1016/j.jbi.2021.103848

M. S. Ejaz and M. R. Islam, “Masked face recognition using convolutional neural network,” in 2019 International Conference on Sustainable Technologies for Industry 4.0, STI 2019, 2019, no. December 2019. https://doi.org/10.1109/STI47673.2019.9068044

J. S. Talahua, J. Buele, P. Calvopina, and J. Varela-Aldas, “Facial recognition system for people with and without face mask in times of the covid-19 pandemic,” Sustain., vol. 13, no. 12, pp. 1–19, 2021. https://doi.org/10.3390/su13126900

Y. Said, “Pynq-YOLO-Net: An embedded quantized convolutional neural network for face mask detection in COVID-19 pandemic era,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 9, pp. 100–106, 2020. https://doi.org/10.14569/IJACSA.2020.0110912

S. Abbasi, H. Abdi, and A. Ahmadi, “A Face-Mask Detection Approach based on YOLO Applied for a New Collected Dataset,” in 26th International Computer Conference, Computer Society of Iran, CSICC 2021, 2021, pp. 1–6. https://doi.org/10.1109/CSICC52343.2021.9420599

B. Roy, S. Nandy, D. Ghosh, D. Dutta, P. Biswas, and T. Das, “MOXA: A Deep Learning Based Unmanned Approach For Real-Time Monitoring of People Wearing Medical Masks,” Trans. Indian Natl. Acad. Eng., vol. 5, no. 3, pp. 509–518, 2020. https://doi.org/10.1007/s41403-020-00157-z

A. Kumar, A. Kalia, A. Sharma, and M. Kaushal, “A hybrid tiny YOLO v4-SPP module based improved face mask detection vision system,” Journal of Ambient Intelligence and Humanized Computing. 2021. https://doi.org/10.1007/s12652-021-03541-x

J. Yu and W. Zhang, “Face Mask Wearing Detection Algorithm Based on Improved YOLO-v4,” Sensors, vol. 21, no. 3263, 2021. https://doi.org/10.3390/ s21093263

N. M. Aszemi and P. D. D. Dominic, “Hyperparameter Optimization in Convolutional Neural Network using Genetic Algorithms,” Int. J. Adv. Comput. Sci. Appl., vol. 10, no. 6, pp. 269–278, 2019. https://doi.org/10.14569/ijacsa.2019.0100638

S. Loussaief and A. Abdelkrim, “Convolutional Neural Network Hyper-Parameters Optimization based on Genetic Algorithms,” Int. J. Adv. Comput. Sci. Appl., vol. 9, no. 10, pp. 252–266, 2018. https://doi.org/10.14569/IJACSA.2018.091031

H. Choi, “CNN Output Optimization for More Balanced Classification,” Int. J. Fuzzy Log. Intell. Syst., vol. 17, no. 2, pp. 98–106, 2017. https://doi.org/10.5391/IJFIS.2017.17.2.98

Aurélien Géron, “Deep Computer Vision Using Convolutional Neural Network,” in Hands-On Machine Learning with Scikit-Learn, Keras & tensorFlow, 2nd ed., Sebastopol: O’Reilly Media, Inc., 2019, pp. 445–496.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. December, pp. 770–778, 2016. https://doi.org/10.1109/CVPR.2016.90

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016, vol. December, pp. 2818–2826. https://doi.org/10.1109/CVPR.2016.308

F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017, vol. 2017-Janua, pp. 1800–1807. https://doi.org/10.1109/CVPR.2017.195

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 779–788, 2016. https://doi.org/10.1021/je00029a022

J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement,” arXiv:1804.02767v1, pp. 1–6, 2018. https://pjreddie.com/media/files/papers/YOLOv3.pdf

Tzutalin, “LabelImg (Git code ).” 2015.

N. Nguyen, T. Do, T. D. Ngo, and D. Le, “An Evaluation of Deep Learning Methods for Small Object Detection,” Hindawi, 2020. https://doi.org/10.1155/2020/3189691

Y. Ding, Z. Li, and D. Yastremsky, “Real-time Face Mask Detection in Video Data,” arXiv:2105.01816, 2021. http://arxiv.org/abs/2105.01816



DOI: https://doi.org/10.22146/ijccs.72075

Article Metrics

Abstract views : 3662 | views : 2560

Refbacks

  • There are currently no refbacks.


Bookmark and Share


Copyright (c) 2022 IJCCS (Indonesian Journal of Computing and Cybernetics Systems)

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.


Copyright of :
IJCCS (Indonesian Journal of Computing and Cybernetics Systems)
ISSN 1978-1520 (print); ISSN 2460-7258 (online)
is a scientific journal the results of Computing
and Cybernetics Systems
A publication of IndoCEISS.
Gedung S1 Ruang 416 FMIPA UGM, Sekip Utara, Yogyakarta 55281
Fax: +62274 555133
email:ijccs.mipa@ugm.ac.id | http://jurnal.ugm.ac.id/ijccs


View My Stats1
View My Stats2