Course: Biomedical Engineering
Date of thesis discussion: 10/2020
Automatic 3D facial expression recognition with ecologically valid 3D data
This project aimed to conceptualize a database of human faces intended for automatic facial expression recognition (FER). For the first time in the facial expression recognition context, the study provided specific images to arouse emotions in the subjects summoned in the experiment to realize an ecologically valid database. The ecological validity has been achieved by eliciting spontaneous facial expressions without asking the participants to act, but only to react spontaneously to the images administered.
The images have been chosen carefully, considering that the neural network used can distinguish, over the neutral expression, the six basic Paul Ekman’s emotions (i.e., anger, disgust, enjoyment, fear, sadness, and surprise). The interdisciplinary team has also included the psychologist’s figure among the engineering ones to make valid choices.
After visualizing the image, each person was asked to rate, in term of arousal and valence through the Self-Assessment Manikin (SAM) scale, the images seen and to label the emotion felt, to better understand if the pictures have yielded the emotion intended, and how the ratings can be distributed in the affective space. Before joining the experimentation, each person was required to compile an empathy and alexithymia test that provides his personality profile. Each frame was captured from a sequence recorded simultaneously by an RGB camera and a coded-light depth sensor integrated into an innovative instrument providing a 3D map of the acquisition.
This eliciting method’s efficiency is discussed, and the confusion matrix resulting from the neural network is analyzed.
The thesis was realized in collaboration with the Politecnico di Milano, and the experimental sessions were conducted in the 3D Lab of the Politecnico di Torino.
