Facial trustworthiness dampens own-gender bias in emotion recognition

Faces are one of the most salient stimuli in social communication, as they provide information useful for social inference and shape efficient interactions (Jack & Schyns, 2015). This information is mainly based on the perception of invariant (e.g., facial structure, eye shape) and variant (e.g., facial expression) features of a face (Haxby et al., 2000; Quinn & Macrae, 2011). The literature suggests that social categories derived from invariant facial features, such as gender, interact with emotional expressions derived from variant facial features to influence interpersonal and intergroup behaviors (Bagnis et al., 2019; Craig & Lipp, 2017; Freeman & Johnson, 2016; Herlitz & Lovén, 2013; Hewstone et al., 2002; Macrae & Bodenhausen, 2000; Mason et al., 2006; Wacker et al., 2017).

Several studies reported an own-gender bias, especially in women that seem to be more accurate at recognizing female (i.e., ingroup stimuli) faces than male (i.e., outgroup stimuli) faces (Herlitz & Lovén, 2013; Lovén et al., 2011; Man & Hills, 2016; Rehnman & Herlitz, 2007). It has also been shown that happy facial expressions were recognized more quickly when shown by females, while anger was recognized more quickly when shown by males, regardless the gender of observers (Hess et al., 2004; Kret et al., 2011). This interaction between gender and discrete emotional expressions can be explained by two mutual accounts (Craig & Lee, 2020). The visual-structural account (bottom-up) suggests that the interaction is facilitated by an overlap between men's and women's facial features (e.g., square jaw, thicker eyebrows, a round face with large eyes) and angry and happy expressions, respectively (Becker et al., 2007). The stereotype-based account (top-down) suggests instead that this may be due to cultural stereotypes associating men with aggressiveness and women with more positive evaluations (Harris & Ciaramitaro, 2016; Hugenberg & Sczesny, 2006).

In social interactions, another important process led by invariant features from faces (i.e., facial appearance) is the automatic inference of a person's social traits, such as trustworthiness (Todorov et al., 2015). When personal information is not accessible, these inferences guide behavior, such as approaching and remembering trustworthy-looking faces more than untrustworthy-looking faces (Mattarozzi et al., 2015, 2017; Oosterhof & Todorov, 2008). Previous research has established that emotionally neutral faces rated as trustworthy are perceived as expressing happiness, while neutral faces rated as untrustworthy are perceived as expressing anger (Oosterhof & Todorov, 2008, 2009) because of an overgeneralization of adaptive mechanisms underlying the processing of emotional faces (Montepare & Dobish, 2003; Oosterhof & Todorov, 2008, 2009; Said et al., 2009). Recent studies investigating the relationship between facial trustworthiness and emotion recognition highlighted that trustworthy-looking faces enhance a general emotion recognition compared to untrustworthy-looking faces (Colonnello et al., 2019a, 2019b; Colonnello et al., 2019a, 2019b). This effect resonates with the fact that positive and negative inferences differentially activate motivational systems and behavioral responses, i.e., appetitive/approach and defensive/avoidance, leading to an advantage or a disadvantage on emotion recognition (Lang & Bradley, 2010).

Building upon extant research, we can hypothesize that gender would interact with facial appearance-based inferences of trustworthiness in emotion recognition. To date, no studies have investigated the intersection of social categories from different invariant facial features (i.e., gender and facial trustworthiness) influencing emotion recognition. Accordingly, we aimed to examine whether gender would affect emotion recognition as a function of facial trustworthiness. Specifically, we tested this hypothesis by measuring the accuracy and reaction times during an emotion recognition task while female participants watched dynamic emotional expressions (i.e., happiness and anger) displayed by female and male faces varying in level of facial trustworthiness (i.e., trustworthy- vs. untrustworthy-looking faces).

Since trusting someone is a crucial aspect that drives how people behave toward each other, we might expect facial trustworthiness to modulate gender bias in emotion recognition. Following Colonnello et al., (2019a, 2019b), it can be hypothesized that (H1a) trustworthy-looking faces would lead to an overall improvement in the recognition of emotional expressions by activating the appetitive motivational system, dampening the own- gender bias. Conversely, untrustworthy-looking faces would reveal an own-gender bias (H1b). As ingroup members makes people more inclined to approach them (Paladino & Castelli, 2008), even if they are untrustworthy, they would favor emotion recognition compared to untrustworthy outgroup members (i.e., untrustworthy male faces observed by women).

In addition, in line with the visual structural and stereotype-based accounts (Craig & Lee, 2020), we might expect the results to differ according to emotional expressions, namely happiness and anger. Also, it is important to note that global face characteristics, such as a masculine/feminine appearance, influence inferences of trustworthiness, as feminine faces are usually judged as more trustworthy than masculine faces and vice versa (Hess et al., 2009; Oosterhof & Todorov, 2008; Todorov et al., 2015). These perceptual overlaps between the facial configuration of trustworthy-looking and feminine faces may contribute to the happy-female advantage (H2a), whereas the advantage in perceiving anger in males would be stronger for untrustworthy-looking targets than for trustworthy-looking targets (H2b), by reinforcing the well-known physical overlap between female/male facial traits and smiling/angry faces (Craig & Lee, 2020). See Fig. 1 for a visual summary of the hypotheses.

During social interactions, people tend to adapt their behavior to social categories, such as gender, or their evaluation of others in terms of trustworthy-looking appearance (Bagnis et al., 2019; Todorov et al., 2015). Here, our aim was to investigate the intersection of multiple social category dimensions derived by invariant facial features (i.e., gender and facial trustworthiness) in affecting face emotion recognition.

In keeping with our hypothesis, the present results showed that facial trustworthiness can modulate the own-gender bias on emotion recognition. In particular, the own-gender bias was specific for untrustworthy-looking faces (H1b), where emotional expressions were recognized less accurately when displayed by an outgroup member (i.e., an untrustworthy man) than by an ingroup member (i.e., an untrustworthy woman). On the contrary, facial features that convey trustworthiness seems to be able to reduce the own-gender bias (H1a), attenuating differences in recognizing female and male emotional expressions.

These results may be mutually explained by a differential activation of the motivational systems by untrustworthy-looking and trustworthy-looking faces and by the involvement of a different level of attentional resources to the stimuli (Bradley et al., 2001; Oosterhof & Todorov, 2008; Pessoa, 2009; Schupp et al., 2004). Specifically, it has been shown that untrustworthy-looking faces represent social stimuli perceived as potentially threatening and, thus, activate the defensive motivational system associated with evolutionarily preserved avoidance responses (Colonnello et al., 2019a, 2019b; Colonnello et al., 2019a, 2019b). Consistent with an attentional negativity-bias (i.e., an adaptive evolutionary bias that foster negative stimuli detection to avoid threat and danger), it is also possible that untrustworthy-looking faces have captured higher attentional resources, diverting processing away from the main task with effects on emotion recognition (Eastwood et al., 2003; Öhman et al., 2001; Schupp et al., 2004). Consistently, here, emotion recognition was generally worst for untrustworthy-looking faces compared to trustworthy-looking faces. However, the interaction between gender and facial trustworthiness revealed that recognition accuracy was higher when the emotion was expressed by untrustworthy-looking woman (i.e., ingroup) compared to untrustworthy-looking man (i.e., outgroup). This finding suggests that the ingroup category is less susceptible to the sense of threat led by the untrustworthy facial appearance and, thus, less likely to undergo a worsening in the emotion recognition performance, revealing the own-gender bias. In fact, the categorization among humans between “us”, ingroups, and “them”, outgroups, is an adaptive mechanism that allows to maximize our behavioral responses (Brewer, 1999; Paladino & Castelli, 2008). The ability to recognize quickly and accurately the emotional expressions of others have fundamental consequences on social interactions, and, here, the avoidance behavior usually activated by untrustworthy-looking faces seems to be weaken when emotions were displayed by the ingroup category increasing the emotion recognition, in favor of “our” compared to “their” emotional expressions. Moreover, when the emotion is displayed by an untrustworthy-looking face, the differences in familiar facial features and experience level (i.e., greater familiarity with female faces) may have contributed to the more accurate recognition of own-gender emotional expressions. Consistently, own-gender bias in females is thought to be an advantage based on an early perceptual expertise for female faces that is reinforced during social development through reciprocal interactions (Herlitz & Lovén, 2013).

On the contrary, no evidence for a response bias toward ingroup and outgroup emotions expressed by trustworthy-looking faces was found suggesting that positive inferences on trustworthiness may reduce intergroup differences in emotion recognition, probably due to the activation of the appetitive motivational system and, thus, of approach behavior (Todorov et al., 2008; Lang & Bradley, 2010).

Finally, when we took into account the discrete emotional expressions, namely happiness and anger, we found that facial appearance is able to influence the readiness to recognize female and male facial expressions, in function of discrete emotional expressions. Contrary to our hypothesis of a happy-female (H2a) and anger-male (H2b) advantage facilitated by perceptual overlaps between the facial configuration of trustworthy-looking and feminine faces, and untrustworthy-looking and masculine faces, we did not find a recognition advantage for happiness on trustworthy- looking female faces and anger on untrustworthy-looking male faces. Specifically, in line with the visual structural and stereotype-based accounts (Craig & Lee, 2020), the intersection between female and happiness and between male and anger was found to influence emotion recognition speed, but only when faces looked trustworthy, as happiness was recognized more quickly in females and anger in males. In untrustworthy-looking faces, no differences in recognition speed between female and male emotional expressions were found. A possible explanation for this might be that, when positive inferences on trustworthiness are activated, the appetitive motivational system promotes an adaptive evolutionary mechanism about the readiness to perceive anger in males to avoid potential physical threat and happiness in female to gain social support (Becker et al., 2007; Craig & Lee, 2020; Tay, 2015). Untrustworthiness, on the other hand, seems to interfere with this facilitation, again potentially reflecting a higher capture of attentional resources by negative stimuli (i.e., untrustworthy-looking faces), with a resulting more elaborated cognitive processing than positive ones which may have delayed the emotion categorization (Todorov et al., 2008; Lang & Bradley, 2010; Colonnello et al., 2019a, 2019b).

Some limitations of this work must be addressed. First, our sample was limited to female participants, reducing the generalizability of the results and the interpretation of the mechanisms underlying them. Although additional research is needed to address the same pattern of results with male participants, this work represents a first step to understand whether the effect of gender on emotion recognition is moderated by facial trustworthiness. In addition, it is worth to note that it has already been shown that the own-gender bias is more robust and reliable in women (Herlitz & Lovén, 2013; Lovén et al., 2011; Man & Hills, 2016; Rehnman & Herlitz, 2007). Another limitation lies in the lack of implicit or explicit measures of biases and stereotypes (Amodio & Devine, 2006). Accordingly, future studies need to be carried out using measures to characterize personality individual differences in perceiving intergroup members to disambiguate the underlying mechanisms explained by the visual structural and stereotype-based accounts (Craig & Lee, 2020). Finally, our sample consisted of Caucasian participants. Since it has been shown that people have considerable difficulty perceiving emotional expressions from people of different ethnic backgrounds, resulting in lower recognition accuracy (e.g., Chiao & Ambady, 2007), future studies should investigate the effects of ethnic and cultural differences on the recognition of emotions in trustworthy- and untrustworthy-looking faces.

In conclusion, the present work adds to intergroup literatures by examining the extent to which emotion recognition is influenced by the intersection between social categories led by invariant facial features, such as gender and facial trustworthiness. Results suggested that people are indeed influenced by making automatic inferences about trustworthiness during intergroup interactions. This finding is not only theoretically important, but also holds real-world implications, as trustworthiness inferences from faces as well as gender biases have been demonstrated to have an impact on several social contexts (e.g., Bagnis et al., 2021; Bagnis, et al., 2020; Mattarozzi et al., 2017, 2020; Pireddu et al., 2022; Todorov, 2005; Wilson & Rule, 2015). We showed that negative inferences from untrustworthy-looking faces reveal the own-gender bias and thus may contribute, for example, to systemic gender-based disparities in healthcare (Fitzgerald et al., 2019). Similarly, further work should carry out to extend this line of research to other intergroup bias, such as racial bias or age bias. As such, future studies should take into account potential interaction effects between inferences from facial appearance and intergroup biases when investigating their influence both in experimental and ecological settings.