Research Team Develops AI Technique for 3D Facial Expression Detection

A joint research team led by Professors Ki-Hun Jeong and Doheon Lee from the Korea Advanced Institute of Science and Technology (KAIST) has developed a new technique for facial expression detection by combining near-infrared light-field camera techniques with artificial intelligence (AI).

The research was published in Advanced Intelligent Systems.

Light-Field Cameras

Light-field cameras contain micro-lens arrays in front of the image sensor, and this enables them to fit into a smart phone. At the same time, they can still acquire the spatial and directional information of the light with a single shot. 

This imaging technique is used to reconstruct images in many different ways, such as multiviews, refocusing, and 3D image acquisition. 

With that said, the technique has some limitations. Existing light-field cameras have struggled to provide accurate image contrast and 3D reconstruction at times due to the shadows caused by external light sources in the environment. 

The research team was able to stabilize the accuracy of the 3D image reconstruction that depended on environmental light, and the technique allowed them to overcome the limitations of existing light-field cameras. They developed a new camera that was optimized for the 3D image reconstruction of facial expressions, and they used it to acquire high-quality 3D reconstruction images of facial expressions of various emotions. They could achieve this regardless of the lighting conditions of the environment.

Machine Learning to Distinguish Expressions

The team then used machine learning to distinguish the facial expressions in the acquired 3D images, which achieved an 85% accuracy rate. They also calculated the interdependence of distance information, which varies with facial expression in 3D images, to identify the information a light-field camera uses to distinguish human expressions. 

“The sub-miniature light-field camera developed by the research team has the potential to become the new platform to quantitatively analyze the facial expressions and emotions of humans,” Professor Ki-Hun Jeong said. 

This research could have a big impact on a wide range of industries. 

 “It could be applied in various fields including mobile healthcare, field diagnosis, social cognition, and human-machine interactions,” he said.