IDEAS home Printed from https://ideas.repec.org/a/gam/jftint/v14y2021i1p5-d709020.html
   My bibliography  Save this article

Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition

Author

Listed:
  • Peter A. Gloor

    (MIT Center for Collective Intelligence, Cambridge, MA 02142, USA)

  • Andrea Fronzetti Colladon

    (Department of Engineering, University of Perugia, 06123 Perugia, Italy)

  • Erkin Altuntas

    (Galaxyadvisors AG, 5000 Aarau, Switzerland)

  • Cengiz Cetinkaya

    (Department of Data Science, Lucerne University of Applied Sciences and Arts, 6002 Lucerne, Switzerland)

  • Maximilian F. Kaiser

    (Department of Information Systems, University of Cologne, 50923 Cologne, Germany)

  • Lukas Ripperger

    (Department of Information Systems, University of Cologne, 50923 Cologne, Germany)

  • Tim Schaefer

    (Department of Information Systems, University of Cologne, 50923 Cologne, Germany)

Abstract

Can we really “read the mind in the eyes”? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of the individual’s face through facial emotion recognition (FER) while watching a series of 15 short videos of different genres. To calibrate the system, we invited 85 people to watch the videos, while their emotional responses were analyzed through their facial expression. At the same time, these individuals also took four well-validated surveys of personality characteristics and moral values: the revised NEO FFI personality inventory, the Haidt moral foundations test, the Schwartz personal value system, and the domain-specific risk-taking scale (DOSPERT). We found that personality characteristics and moral values of an individual can be predicted through their emotional response to the videos as shown in their face, with an accuracy of up to 86% using gradient-boosted trees. We also found that different personality characteristics are better predicted by different videos, in other words, there is no single video that will provide accurate predictions for all personality characteristics, but it is the response to the mix of different videos that allows for accurate prediction.

Suggested Citation

  • Peter A. Gloor & Andrea Fronzetti Colladon & Erkin Altuntas & Cengiz Cetinkaya & Maximilian F. Kaiser & Lukas Ripperger & Tim Schaefer, 2021. "Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition," Future Internet, MDPI, vol. 14(1), pages 1-18, December.
    • Handle: RePEc:gam:jftint:v:14:y:2021:i:1:p:5-:d:709020
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1999-5903/14/1/5/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1999-5903/14/1/5/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jannik Rößler & Jiachen Sun & Peter Gloor, 2021. "Reducing Videoconferencing Fatigue through Facial Emotion Recognition," Future Internet, MDPI, vol. 13(5), pages 1-15, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cèlia Llurba & Gabriela Fretes & Ramon Palau, 2024. "Classroom Emotion Monitoring Based on Image Processing," Sustainability, MDPI, vol. 16(2), pages 1-14, January.
    2. Josep Petchamé & Ignasi Iriondo & Garazi Azanza, 2022. "“Seeing and Being Seen” or Just “Seeing” in a Smart Classroom Context When Videoconferencing: A User Experience-Based Qualitative Research on the Use of Cameras," IJERPH, MDPI, vol. 19(15), pages 1-13, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jftint:v:14:y:2021:i:1:p:5-:d:709020. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.