Abstract
This study examined the validity and possible utility of a new procedure for the extraction of nonverbal behaviors from dyadic conversation. Three methods were used to extract nonverbal behaviors (i.e., hand gestures, adaptors, and utterances). A novel automated method employing video images and speech-signal analysis software programs was compared to the more traditional coding and behavioral rating methods. The automated and coding methods provided an objective count of how many times a target behavior occurred, while behavioral ratings were based on more subjective impressions. Although there was no difference between the automated and coding methods for hand gestures, the coding method using an event recorder yielded marginally significantly more instances of adaptors and utterances as compared to the software programs. Measures of each nonverbal behavior were positively correlated across the different methods. In addition, interpersonal impressions of each speaker were rated by both observers and conversational partners. Although R 2 was lower than for the coding/behavioral rating methods, nonverbal behaviors extracted using the software programs significantly predicted familiarity and activeness ratings from both observer and partner points of view. These results support the validity and possible utility of the software-based automated extraction procedure.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Twelve speakers each were assigned to coders A and B, coders A and C, coders A and D, coders B and C, and coders B and D. The remaining 10 speakers were assigned to coders C and D.
Reliability was calculated as the correlation between the two coders. Correlation coefficients for each pair of coders were all significant (ps < .01): coders A and B (hand gestures r = 0.96, adaptors r = 0.87, utterances r = 0.93), coders A and C (hand gestures r = 0.88, adaptors r = 0.97, utterances r = 0.95), coders A and D (hand gestures r = 0.99, adaptors r = 0.98, utterances r = 0.96), coders B and C (hand gestures r = 0.82, adaptors r = 0.97, utterances r = 0.92), coders B and D (hand gestures r = 0.98, adaptors r = 0.94, utterances r = 0.83), and coders C and D (hand gestures r = 0.88, adaptors r = 0.89, utterances r = 0.99).
As in the case of the coding method, reliability was calculated as the correlation between two raters. Correlation coefficients were all significant (ps < .01): hand gestures r = 0.74, adaptors r = 0.78, utterances r = 0.73.
References
Ambady, N., Koo, J., Rosenthal, R., & Winograd, C. H. (2002). Physical therapists’ nonverbal communication predicts geriatric patient’s health outcomes. Psychology and Aging, 17, 443–452.
Ambady, N., & Rosenthal, R. (1992). Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. Psychological Bulletin, 111, 256–274.
Ambady, N., & Rosenthal, R. (1993). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology, 64, 431–441.
Andersen, J. F. (1985). Nonverbal immediacy in interpersonal communication. In A. W. Siegman & S. Feldstein (Eds.), Multichannel integrations of nonverbal behavior (pp. 1–36). Hillsdale, NJ: Lawrence Erlbaum.
Arakawa, A., & Suzuki, N. (2004). Hand movements and feeling. Japanese Journal of Research on Emotions, 10, 56–64.
Baesler, E. J., & Burgoon, J. K. (1987). Measurement and reliability of nonverbal behavior. Journal of Nonverbal Behavior, 11, 205–233.
Bernieri, J. F., Gills, J. S., Davis, J. M., & Grahe, J. E. (1996). Dyad rapport and the accuracy of its judgment across situations: A lens model analysis. Journal of Personality and Social Psychology, 71, 110–129.
Daibo, I. (1990). Expression of intimacy in interpersonal relationships: Development and dissolution of relationships in communication. Japanese Psychological Review, 33, 322–352.
Dunber, N. E., & Burgoon, J. K. (2005). Perception of power and interactional dominance in interpersonal relationships. Journal of Social and Personal Relationships, 22, 207–233.
Ekman, P., & Friesen, W. V. (1972). Hand movements. Journal of Communication, 22, 353–374.
Floyd, K. (2006). Communicating affection: Interpersonal behavior and social context. Cambridge, UK: Cambridge University Press.
Fujiwara, K., & Daibo, I. (2010). The function of positive affect in a communication context: Satisfaction with conversation and hand movement. Japanese Journal of Research on Emotions, 17, 180–188.
Guerrero, L. K. (1997). Nonverbal involvement across interactions with same-sex friends, opposite-sex friends, and romantic partners: Consistency or change? Journal of Social and Personal Relationships, 14, 31–58.
Guerrero, L. K. (2005). Observer ratings of nonverbal involvement and immediacy. In V. Manusov (Ed.), The sourcebook of nonverbal measures: Going beyond words (pp. 221–235). Mahwah, NJ: Lawrence Erlbaum.
Hayashi, F. (1978). The fundamental dimensions of interpersonal cognitive structure. Nagoya University School of Education, Department of Education Psychology Bulletin, 25, 233–247.
Ito, T. (1991). The characteristics of unit nonverbal behaviors in face-to-face interaction. The Japanese Journal of Experimental Social Psychology, 31, 85–93.
Kimura, M., Daibo, I., & Yogo, M. (2010). Investigation into the mechanism of interpersonal communication-cognition as a social skill. Japanese Journal of Social Psychology, 26, 13–24.
Krauss, R. M., Morrel-Samuels, P., & Colasante, C. (1991). Do conversational hand gestures communicate? Journal of Personality and Social Psychology, 61, 743–754.
Murphy, N. A. (2005). Using thin slices for behavioral coding. Journal of Nonverbal Behavior, 29, 235–246.
Ogawa, K. (2003a). The effects of utterance balance in a conversation on a neutral observer’s impressions of the conversants and the conversational process. The Japanese Journal of Experimental Social Psychology, 43, 63–74.
Ogawa, K. (2003b). The effects of balance in the amount of conversational utterance on the impressions of the partner and the conversation. IECICE Technical Report, 103, 63–74.
Osgood, C. E., & Suci, G. J. (1955). Factor analysis of meaning. Journal of Experimental Psychology, 50, 325–338.
Riggio, R. E., & Friedman, H. S. (1986). Impression formation: The role of expressive behavior. Journal of Personality and Social Psychology, 50, 421–427.
Rosenberg, S., Nelson, C., & Vivekananthan, P. S. (1968). A multidimensional approach to the structure of personality impressions. Journal of Personality and Social Psychology, 9, 283–294.
Tickle-Degnen, L., & Rosenthal, R. (1987). Group rapport and nonverbal behavior. Review of Personality and Social Psychology, 9, 285–293.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fujiwara, K., Daibo, I. The Extraction of Nonverbal Behaviors: Using Video Images and Speech-Signal Analysis in Dyadic Conversation. J Nonverbal Behav 38, 377–388 (2014). https://doi.org/10.1007/s10919-014-0183-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10919-014-0183-3