Differences in facial expressions of four universal emotions
Introduction
Facial expressions are used in humans and animals for communication, in particular to convey one's emotional state (Darwin, 1965). This communication can be reflexive, as situations may evoke emotions that are spontaneously expressed on the face. In other instances, particularly in humans, facial expressions may be volitional signals intended for communication and not reflect the true emotional state of the person (Ekman and Friesen, 1975). Impairment in emotional processing, specifically emotion recognition, has been described in psychiatric disorders including schizophrenia, depression and bipolar disorder, and neurological disorders (review: Kohler et al., 2004). Since the earliest descriptions of schizophrenia, decreased and muted facial expressions of emotions has been reported as a hallmark of the illness, however, there have been few attempts to investigate this impairment further in schizophrenia and other psychiatric disorders.
Six basic emotions—happiness, sadness, anger, fear, disgust and surprise—and their corresponding facial expressions are recognized across different cultures (Huber, 1931, Eibl-Eibesfeldt, 1970, Izard, 1971, Ekman and Friesen, 1975). Descriptions have been made about which facial muscles are involved in the formation of each of the basic emotions (Huber, 1931, Plutchik, 1962, Ekman and Friesen, 1975, Gosselin et al., 1997). For happy expressions, Ekman and Friesen (1975) described facial expressions of tense lower eyelids, raised cheeks and lip corners pulled up; for sad expressions, inner eyebrows raised and drawn together, and lip corners pulled down; for anger expressions, lowered eyebrows drawn together, tense lower eyelids, pressed lips or lips parted in a square shape; for fear expressions, eyebrows raised and drawn together, wide open eyes with tense lower eyelids and stretched lips. Based on facial muscle movement, Ekman and Friesen (1978) developed the Facial Action Coding System (FACS) by identifying the presence of specific actions of facial muscles called Action Units (AUs). Gosselin et al. (1997) tested Ekman and Friesen's predictions about facial expressions of six emotions in two conditions—posed or unfelt and evoked or felt. In that study, actors used two different methods of displaying facial expressions of six emotions—trying to experience the target emotion according to the Stanislawski technique, while expressing the emotion (evoked emotion) or merely displaying the emotion without the emotional experience (posed emotion). FACS analyses of facial expressions by a single rater revealed that AUs for each emotion were concordant with Ekman and Friesen's descriptions. Occurrence rates of AUs for evoked and posed facial expressions showed considerable overlap, in particular for happy and surprise expressions. Other studies that investigated facial landmark changes associated with emotional expression focused on measurement of muscle activity with electromyography (EMG). Limitations of this methodology include that only select muscle groups, such as corrugator supercilii orbicularis oculi and zygomaticus major have been measured, showing the corrugator to be associated with sad and the zygomaticus with happy emotions (Schwartz et al., 1976). Tassinary and Cacioppo (1989) elucidated that expressions of action units involving brow and cheek regions are associated with discrete facial muscle activity as measured by surface EMG. More recently, considerable overlap has been shown between surface and intramuscular recordings of facial EMG during happy, sad and angry expressions (Aniss and Sachdev, 1996).
In our study, three FACS certified raters examined 128 images of extremely happy, sad, angry and fearful faces that were selected for use in a functional imaging study and piloted for recognition in a group of healthy subjects. Disgust was not included because of our assumption that it may not present a pure emotion, but rather a mixture of other universal emotions (Kohler et al., 2003). Surprise was not included because its valence depends entirely on the triggering event and it can be therefore any of the other emotions, with a rapid onset. The purpose of our study was to investigate which facial changes are most frequent in happy, sad, angry and fearful expressions, and which facial changes are essential for accurate recognition of the particular emotion. The study included the following specific aims: (1) Which action units characterize the different emotions? We hypothesized that each emotion can be defined by the presence of action units common to faces with the particular emotion. (2) Which action units distinguish different emotions from each other? We hypothesized that facial expressions of each emotion consist of unique action units that are distinct from other emotions. (3) How do posed and evoked emotions differ with respect to action units? We hypothesized that different action units are used for the expression of posed and evoked emotions. (4) Do men and women utilize different action units for the expression of emotions? We hypothesized that certain emotions, in particular anger, are expressed differently by men and women. (5) Which action units are associated with recognition of each emotion? We hypothesized that the presence of characteristic action units are associated with proper detection of the particular emotion.
We propose that findings based on accurate descriptions of facial muscle groups in people without psychiatric disorders will facilitate investigations into the effects of psychiatric illness on facial emotion expression in persons with psychiatric disorders. This knowledge will lead to better understanding of how interpersonal nonverbal communication is affected in psychiatric disorders. In particular, this will give us information whether disorders, such as schizophrenia or affective disorders, are associated with muted, but appropriate facial muscle movement or recruitment of different muscle groups.
Section snippets
Task construction
Color slide photographs acquired during a study of facial displays of emotion (Gur et al., 2002a), which totaled over 5000 pictures were digitally scanned, then processed in PhotoShop© to remove background features and reduce distinguishing hair and clothing cues. Evoked and posed images from eight male and eight female actors were selected for each emotion (Happy, Sad, Anger, Fear). Based on our finding that disgust expressions, in particular of extreme intensity, were poorly recognized (
Characterization of emotional expressions
FACS ratings revealed separate profiles for Happy, Sad, Anger and Fear expressions (Fig. 1, Table 1). Characteristic, uniquely absent and present AUs were found for each emotion. Expressions of Anger and Sad shared the most characteristic AUs (5), while Fear and Happy shared the fewest (2). Characteristic AUs for Happy included in descending order of frequency 12, 7, 26, 6, 10, 1 and 25. AUs 6 and 12 were uniquely present, while AUs 4 and 20 were uniquely absent in Happy expressions.
Discussion
This study examined the presence of action units (AUs) in four universal emotions, how these AUs differ according to gender of poser and condition, and the relationship between presence or absence of AUs and recognition of the expressed emotion.
In Happy faces, we found characteristic expressions to consist of raised inner eyebrows, tightened lower eyelid, raised cheeks, upper lip raised and lip corners turned upward. Recognition of Happy faces was associated with characteristic AUs, such as
Acknowledgments
This work was supported by NIMH MH01839 and MH43880
References (24)
- et al.
Facial emotion discrimination: I. Task construction and behavioral findings in normal subjects
Psychiatry Research
(1992) - et al.
A method for obtaining 3-dimensional facial expressions and its standardization for use in neurocognitive studies
Journal of Neuroscience Methods
(2002) - et al.
Brain activation during facial emotion processing
NeuroImage
(2002) - et al.
Concordance between surface and intra-muscular recordings of facial emg during emotional expression
Electromyography and Clinical Neurophysiology
(1996) - et al.
Measuring facial expressions by computer image analysis
Psychophysiology
(1999) - et al.
Automated face analysis by feature point tracking has high concurrent validity with manual facs coding
Psychophysiology
(1999) The Expression of Emotions in Man and Animals
(1965)Ethology, The Biology of Behavior
(1970)- et al.
Unmasking the Face
(1975) - et al.
Manual of the Facial Action Coding System (FACS)
(1978)
Felt, false, and miserable smiles
Journal of Nonverbal Behavior
Components and recognition of facial expression in the communication of emotion by actors
Cited by (184)
An empirical user-study of text-based nonverbal annotation systems for human–human conversations
2023, International Journal of Human Computer StudiesThe effect of pedestrian lighting on facial expression recognition with 3D models: A lab experiment
2023, Building and Environment