AI Researchers From University of Hong Kong Propose Unified Python Framework, Named Face Feature Rectification Network (FFR-Net), For Masked And Mask-Free Face Recognition

Facial recognition is an area of research that has been extensively studied in recent years. It has been very successful and has achieved satisfactory performance. However, with the covid19 pandemic, wearing a mask has been a real challenge for facial recognition networks. The mask hides a large part of the face and causes a problem of occlusion.

We can distinguish two main methods in the literature to solve the occlusion problem caused by wearing a mask. The first approach completely ignores the features extracted from the mask area. This technique indeed avoids corrupted features, but it also causes a loss of global facial information. The second approach tries to restore the face’s hidden part by usually using generative adversarial networks. However, this strategy does not conserve the identity information in the spatial space. 

A recently published paper introduced a new approach to recognising masked and mask-free faces. The novel network is a unified framework named Face Feature Rectification Network (FFR-Net), which uses rectification blocks intending to rectify features extracted by an encoder (backbone) in both channel and spatial dimensions. 

The authors introduced two parallel rectification blocks, ChnRec and SpcRec, which can be added to any state-of-the-art encoder. The two blocks are implemented on top of extracted features. ChnRec deals with the channel dimension, while SpcRec deals with the spatial dimension. The output of each block is the multiplication of the features extracted by the encoder (original features), and a rectification matrice denoted M. The outputs obtained from each block are then merged with the original features. 

Since it is essential to predict the matrix M well, two techniques for carrying out this task have been proposed: Self-Similarity and Spacial Symmetry.  Self-Similarity is proposed to compute the cosine similarity matrices between feature vectors in both channel and spatial dimensions in order to overcome the loss of facial structure caused by mask occlusion. On the other hand, ChnRec seeks to choose channels that extract features from mask-free regions while excluding those from masked areas, and it can focus on only one side of the face and ignore the similarity. Spacial Semmutry, a flipped fusion block, is thus added as the last step of ChnRec to push it to conserve spatial symmetry.

The loss functions were elected to achieve three goals: identity consistency, mask robustness and accurate classification. The identity loss aims to achieve two targets: keeping the features extracted from mask-free faces uncorrupted and rectifying the features extracted from masked faces. In addition, the triplet loss is used to make the distance between the rectified representation (anchor) and the representation of the mask-free counterpart smaller and the distance between the anchor and the original representation of the same masked face larger. Finally, the CosFace function was elected as a classification loss.

To evaluate FFR-Net, the authors carried out an experimental study over LFW dataset for the task of face verification and MegaFace for the task of face identification. The accuracy of the network on mask-free data, the accuracy on masked, and the average accuracy were used as metrics. Results demonstrate that the novel unified framework proposed in this paper achieves competitive performance, especially considering the average accuracy. In addition, an ablation study shows that the two rectification blocks (channel and spatial) are efficient and enhance the network’s overall performance.

In this article, the researchers proposed to unify the face recognition problem for datasets containing images of faces without masks and others made by images of faces with masks. They introduced a novel channel and space rectification block aiming to rectify the extracted features in both spatial and channel dimensions to achieve their goal. The evaluation of this approach has proven its effectiveness.

This Article is written as a summary article by Marktechpost Staff based on the research paper 'A UNIFIED FRAMEWORK FOR MASKED AND MASK-FREE FACE RECOGNITION VIA FEATURE RECTIFICATION'. All Credit For This Research Goes To Researchers on This Project. Checkout the paper and github link.

Please Don't Forget To Join Our ML Subreddit