Abstract
Human social interaction is enriched with synchronous movement which is said to be essential to establish interactional flow. One commonly investigated phenomenon in this regard is facial mimicry, the tendency of humans to mirror facial expressions. Because studies investigating facial mimicry in face-to-face interactions are lacking, the temporal dynamics of facial mimicry remain unclear. We therefore developed and tested the suitability of a novel approach to quantifying facial expression synchrony in face-to-face interactions: windowed cross-lagged correlation analysis (WCLC) for electromyography signals. We recorded muscle activations related to smiling (Zygomaticus Major) and frowning (Corrugator Supercilii) of two interaction partners simultaneously in 30 dyadic affiliative interactions. We expected WCLC to reliably detect facial expression synchrony above chance level and, based on previous research, expected the occurrence of rapid synchronization of smiles within 200 ms. WCLC significantly detected synchrony of smiling but not frowning compared to a control condition of chance level synchrony in six different interactional phases (smiling: d z s = .85–1.11; frowning: d z s = .01–.30). Synchronizations of smiles between interaction partners predominantly occurred within 1000 ms, with a significant amount occurring within 200 ms. This rapid synchronization of smiles supports the notion of the existence of an anticipated mimicry response for smiles. We conclude that WCLC is suited to quantify the temporal dynamics of facial expression synchrony in dyadic interactions and discuss implications for different psychological research areas.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
For this article, we assume that behavioral mimicry is a form of nonverbal synchrony that is restricted to matching movements and that the timing component of the mimicry response can be accounted for by using measures of nonverbal synchrony for matching behaviors. For a discussion of the overlap of behavioral mimicry and nonverbal synchrony see Chartrand and Lakin (2013).
We use the term ‘synchronization’ to refer to the active process of synchronizing a movement to match another movement. Contrarily, we use the term ‘synchrony’ to refer to the state of movements being synchronous.
By setting the time constant of the moving average to a value of .5 s, we approximated the approach taken by Ramseyer and Tschacher (2011).
For an optimal temporal resolution of the WCLC output, this increment should be set equal to the time interval between single observations in the time series (e.g., .1 s when the data was sampled at 10 Hz).
Practical considerations may also play a role in the analysis. For example, choosing smaller window sizes as well as choosing larger maximum time lags increases the number of correlations calculated by WCLC and thus processing time and family wise error rate.
For analyses across muscle sites, we averaged the synchrony emerging for the two combinations of muscle sites (i.e. Zygomaticus person 1 » Corrugator person 2 and Corrugator person 1 » Zygomaticus person 2).
In cross-lagged correlation analyses (cave: not windowed cross-lagged correlation), the time lag corresponding to the peak value is a commonly used outcome to determine the time lag of the association of two analyzed time series (cf. Boker et al. 2002). Because in WCLC each row (= each time window) represents a single cross-lagged correlation, our peak counting approach is analogous to the established approach.
References
Bailey, P. E., & Henry, J. D. (2009). Subconscious facial expression mimicry is preserved in older adulthood. Psychology and Aging, 24(4), 995–1000. doi:10.1037/a0015789.
Bernieri, F. J. (1988). Coordinated movement and rapport in teacher–student interactions. Journal of Nonverbal Behavior, 12(2), 120–138. doi:10.1007/BF00986930.
Boker, S. M., Cohn, J. F., Theobald, B.-J., Matthews, I., Brick, T. R., & Spies, J. R. (2009). Effects of damping head movement and facial expression in dyadic conversation using real-time facial expression tracking and synthesized avatars. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences, 364(1535), 3485–3495. doi:10.1098/rstb.2009.0152.
Boker, S. M., Rotondo, J. L., Xu, M., & King, K. (2002). Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychological Methods, 7(3), 338–355. doi:10.1037/1082-989X.7.3.338.
Chartrand, T. L., & Bargh, J. A. (1999). The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology, 76(6), 893–910. doi:10.1037/0022-3514.76.6.893.
Chartrand, T. L., & Lakin, J. L. (2013). The antecedents and consequences of human behavioral mimicry. Annual Review of Psychology, 64(1), 285–308. doi:10.1146/annurev-psych-113011-143754.
Cohen, A. S., Morrison, S. C., & Callaway, D. A. (2013). Computerized facial analysis for understanding constricted/blunted affect: Initial feasibility, reliability, and validity data. Schizophrenia Research, 148(1–3), 111–116. doi:10.1016/j.schres.2013.05.003.
Dimberg, U. (1982). Facial reactions to facial expressions. Psychophysiology, 19(6), 643–647. doi:10.1111/j.1469-8986.1982.tb02516.x.
Dimberg, U., Thunberg, M., & Elmehed, K. (2000). Unconscious facial reactions to emotional facial expressions. Psychological Science, 11(1), 86–89. doi:10.1111/1467-9280.00221.
Epstein, L. H. (1990). Perception of activity in the Zygomaticus Major and Corrugator Supercilii muscle regions. Psychophysiology, 27(1), 68–72. doi:10.1111/j.1469-8986.1990.tb02181.x.
Fischer, A. H., & van Kleef, G. A. (2010). Where have all the people gone? A plea for including social interaction in emotion research. Emotion Review, 2(3), 208–211. doi:10.1177/1754073910361980.
Fridlund, A. J., & Cacioppo, J. T. (1986). Guidelines for human electromyographic research. Psychophysiology, 23(5), 567–589. doi:10.1111/j.1469-8986.1986.tb00676.x.
Heerey, E. A. (2015). Decoding the dyad: Challenges in the study of individual differences in social behavior. Current Directions in Psychological Science, 24(4), 285–291. doi:10.1177/0963721415570731.
Heerey, E. A., & Crossley, H. M. (2013). Predictive and reactive mechanisms in smile reciprocity. Psychological Science, 24(8), 1446–1455. doi:10.1177/0956797612472203.
Hess, U., & Blairy, S. (2001). Facial mimicry and emotional contagion to dynamic emotional facial expressions and their influence on decoding accuracy. International Journal of Psychophysiology, 40(2), 129–141. doi:10.1016/S0167-8760(00)00161-6.
Hess, U., Blairy, S., & Kleck, R. E. (2000). The influence of facial emotion displays, gender, and ethnicity on judgments of dominance and affiliation. Journal of Nonverbal Behavior, 24(4), 265–283. doi:10.1023/A:1006623213355.
Hess, U., & Bourgeois, P. (2010). You smile—I smile: emotion expression in social interaction. Biological Psychology, 84(3), 514–520. doi:10.1016/j.biopsycho.2009.11.001.
Hess, U., & Fischer, A. H. (2013). Emotional mimicry as social regulation. Personality and Social Psychology Review, 17(2), 142–157. doi:10.1177/1088868312472607.
Hühnel, I., Fölster, M., Werheid, K., & Hess, U. (2014). Empathic reactions of younger and older adults: No age related decline in affective responding. Journal of Experimental Social Psychology, 50(1), 136–143. doi:10.1016/j.jesp.2013.09.011.
Kupper, Z., Ramseyer, F., Hoffmann, H., & Tschacher, W. (2015). Nonverbal synchrony in social interactions of patients with schizophrenia indicates socio-communicative deficits. PLoS ONE, 10(12), e0145882. doi:10.1371/journal.pone.0145882.
Lakin, J. L., & Chartrand, T. L. (2003). Using nonconscious behavioral mimicry to create affiliation and rapport. Psychological Science, 14(4), 334–339. doi:10.1111/1467-9280.14481.
Lanzetta, J. T., & Englis, B. G. (1989). Expectations of cooperation and competition and their effects on observers’ vicarious emotional responses. Journal of Personality and Social Psychology, 56(4), 543–554. doi:10.1037/0022-3514.56.4.543.
Laroche, J., Berardi, A. M., & Brangier, E. (2014). Embodiment of intersubjective time: Relational dynamics as attractors in the temporal coordination of interpersonal behaviors and experiences. Frontiers in Psychology, 5(October), 1–17. doi:10.3389/fpsyg.2014.01180.
Larsen, J. T., Norris, C. J., & Cacioppo, J. T. (2003). Effects of positive and negative affect on electromyographic activity over Zygomaticus Major and Corrugator Supercilii. Psychophysiology, 40, 776–785.
Lehrl, S. (1999). Mehrfachwahl-Wortschatz-Intelligenztest: MWT-B. Balingen: Spitta.
Likowski, K. U., Mühlberger, A., Seibt, B., Pauli, P., & Weyers, P. (2008). Modulation of facial mimicry by attitudes. Journal of Experimental Social Psychology, 44(4), 1065–1072. doi:10.1016/j.jesp.2007.10.007.
McIntosh, D. N., Reichmann-Decker, A., Winkielman, P., & Wilbarger, J. L. (2006). When the social mirror breaks: Deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. Developmental Science, 9(3), 295–302. doi:10.1111/j.1467-7687.2006.00492.x.
Messinger, D. S., Mahoor, M. H., Chow, S.-M., & Cohn, J. F. (2009). Automated measurement of facial expression in infant–mother interaction: A pilot study. Infancy, 14(3), 285–305. doi:10.1080/15250000902839963.
Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated body movement reflects relationship quality and outcome. Journal of Consulting and Clinical Psychology, 79(3), 284–295. doi:10.1037/a0023419.
Stel, M., & Vonk, R. (2010). Mimicry in social interaction: Benefits for mimickers, mimickees, and their interaction. British Journal of Psychology, 101(Pt 2), 311–323. doi:10.1348/000712609X465424.
Tassinary, L. G., Cacioppo, J. T., & Vanman, E. J. (2007). The skeletomotor system: Surface electromyography. In J. T. Cacioppo, L. G. Tassinary, & G. Bernston (Eds.), Handbook of psychophysiology (3rd ed., pp. 267–299). Cambridge: Cambridge University Press.
Tickle-Degnen, L., & Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. Psychological Inquiry, 1(4), 285–293. doi:10.1207/s15327965pli0104_1.
Tschacher, W., Rees, G. M., & Ramseyer, F. (2014). Nonverbal synchrony and affect in dyadic interactions. Frontiers in Psychology, 5, 1323. doi:10.3389/fpsyg.2014.01323.
van der Schalk, J., Fischer, A. H., Doosje, B., Wigboldus, D., Hawk, S. T., Rotteveel, M., et al. (2011). Convergent and divergent responses to emotional displays of ingroup and outgroup. Emotion, 11(2), 286–298. doi:10.1037/a0022582.
Varcin, K. J., Bailey, P. E., & Henry, J. D. (2010). Empathic deficits in schizophrenia: The potential role of rapid facial mimicry. Journal of the International Neuropsychological Society, 16(4), 621–629. doi:10.1017/S1355617710000329.
Varlet, M., Marin, L., Raffard, S., Schmidt, R. C., Capdevielle, D., Boulenger, J.-P., et al. (2012). Impairments of social motor coordination in schizophrenia. PLoS ONE, 7(1), e29772. doi:10.1371/journal.pone.0029772.
Wexler, B. E., Levenson, L., Warrenburg, S., & Price, L. H. (1994). Decreased perceptual sensitivity to emotion-evoking stimuli in depression. Psychiatry Research, 51(2), 127–138. doi:10.1016/0165-1781(94)90032-9.
Wittchen, H.-U., Fydrich, T., & Zaudig, M. (1997). SKID: Strukturiertes Klinisches Interview für DSM-IV; Achse I und II. Achse I: psychische Störungen. SKID-I. Göttingen: Hogrefe.
Zhang, Z. (2012). Microsoft kinect sensor and its effect. IEEE Multimedia, 19(2), 4–10. doi:10.1109/MMUL.2012.24.
Acknowledgements
We wish to thank all participants for their participation and Rukiye Köysürenbars, Mathias Osterried, Beatrice Salewski, Olga Schulz, and Katrin Zilliken, who helped with the data collection.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures were approved by a local ethics committee (Psychotherapeutenkammer Hamburg) and performed in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Riehle, M., Kempkensteffen, J. & Lincoln, T.M. Quantifying Facial Expression Synchrony in Face-To-Face Dyadic Interactions: Temporal Dynamics of Simultaneously Recorded Facial EMG Signals. J Nonverbal Behav 41, 85–102 (2017). https://doi.org/10.1007/s10919-016-0246-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10919-016-0246-8