Emotional intelligence: A meta-analytic investigation of predictive validity and nomological net

Abstract

This study used meta-analytic techniques to examine the relationship between emotional intelligence (EI) and performance outcomes. A total of 69 independent studies were located that reported correlations between EI and performance or other variables such as general mental ability (GMA) and the Big Five factors of personality. Results indicated that, across criteria, EI had an operational validity of .23 (k=59, N=9522). Various moderating influences such as the EI measure used, dimensions of EI, scoring method and criterion were evaluated. EI correlated .22 with general mental ability (k=19, N=4158) and .23 (Agreeableness and Openness to Experience; k=14, N=3306) to .34 (Extraversion; k=19, N=3718) with the Big Five factors of personality. Results of various subgroup analyses are presented and implications and future directions are provided.

Introduction

Research on emotional intelligence is gaining momentum (cf. Davies, Stankov, & Roberts, 1998; Goleman, 1995; *Mayer, Caruso, & Salovey, 1999) and becoming one of the most topical areas in organizational research. To some extent, this recent emphasis on emotional intelligence arises from the renewed interest in personality research (cf. Barrick & Mount, 1991; Hough & Ones, 2001). This article discusses the current state of affairs in emotional intelligence research and then applies meta-analytic procedures to provide the first comprehensive understanding of the power of EI to predict performance outcomes. The article also explores the nomological net of EI with other individual difference variables that psychologists traditionally use to predict behavior: General Mental Ability (GMA) and the Big Five factors of personality.

It is difficult to provide an operational definition of EI that is accepted by all. This is not surprising as decades of research on stress, for example, still has not resulted in a universally accepted definition of what constitutes stress; the same can be said of the research on GMA as well as the definition of personality variables (cf. Goldstein, Zedeck, & Goldstein, 2002; Ones, 1993; Ones & Anderson, 2002). The EI concept is typically credited to Salovey and Mayer (1990) who coined the term emotional intelligence but Thorndike (1920) first proposed the idea of social intelligence that some consider akin to EI (but see Mayer & Salovey, 1997). In actuality, EI can best be thought of as a subset of social intelligence (Salovey & Mayer, 1990). However, since the construct of social intelligence is broader than EI, it has often been difficult to discriminate between it and GMA. As a more isolated construct, EI may be more distinct from GMA and personality. In addition, it is possible that measures of EI, especially ability based, are less susceptible to faking than other measures that are more transparent. Nonetheless, others (Kaufman & Kaufman, 2001) argue that the origins of EI can be traced to Binet’s early notions of intelligence. Gardner (1993) has postulated that social intelligence consists of a person’s inter- and intra personal intelligences. Still, EI was not truly popularized until the recent renaissance of the dispositional approach in the workplace (cf. Hough & Ones, 2001).

The differing names given to emotional intelligence are part of the reason that it has been difficult for researchers to agree on an all-encompassing definition. Emotional intelligence has also been referred to as emotional literacy, the emotional quotient, personal intelligence, social intelligence, and interpersonal intelligence (Dulewicz & Higgs, 2000). The definitions are so varied, and the field is growing so rapidly, that researchers are constantly amending even their own definitions of the construct. Nonetheless, although the definitions of EI vary, they tend to be complementary rather than contradictory (*Ciarrochi, Chan, & Caputi, 2000). Based on the many definitions that have already been proffered, this article conceptualizes EI as the set of abilities (verbal and nonverbal) that enable a person to generate, recognize, express, understand, and evaluate their own, and others, emotions in order to guide thinking and action that successfully cope with environmental demands and pressures.

The emotional intelligence concept is generally divided into four dimensions (Salovey & Mayer, 1990), although other models such as a five-component model (Bar-On, 1997) are also widely accepted. George (2000) and others (e.g., Mayer, Salovey, & Caruso, 1999) summarize the four dimensions of emotional intelligence postulated by Salovey and Mayer: the perception of emotion, the integration and assimilation of emotion, knowledge about emotions, and management of emotions. The four dimensions of emotional intelligence have been proposed as a sequential set of steps with the perception of emotion occurring at the first step and the other three then following in the order they are listed above (Mayer, Salovey, Caruso, & Sitarenios, 2001).

Briefly, the first dimension can be thought of as the ability to perceive emotion. This involves the accuracy with which a person can identify emotions in themselves and others. The next dimension consists of the ability to use or assimilate emotions to facilitate thought. This allows people to use emotions to guide their thinking, and new emotions can also be generated during this stage. The third dimension involves how people understand their emotions. Knowledge of emotions allows a person to understand how emotions change and the relationship between these states. The final dimension involves the management of not only ones own mood and emotions, but also the emotions of others.

Emotional intelligence is appealing not only because of our general fascination with the way people feel but, more importantly, because the traditional cognitive predictors leave a considerable amount of variance unexplained. Indeed, cognitive ability accounts only for approximately 25% of the variance in job performance (Goldstein et al., 2002; Hunter & Hunter, 1984). The appeal of EI therefore lies in the possibility of tapping into and explaining another portion of the remaining variance. Although models (Arvey, Renz, & Watson, 1998) have specified the role of general emotionality in the workplace, as of yet, however, no theory specifically discusses the role of EI and its relationship to work outcomes (Wong & Law, 2002).

If EI and GMA were strongly correlated, the influence of EI would not be as important. Some studies have shown, however, that EI and GMA tend to be orthogonal constructs that denote different competencies (e.g., Ciarrochi et al., 2000; Davies et al., 1998; Fox & Spector, 2000). The inability of GMA to account for sufficient variance in success criteria in an organizational and educational context has therefore led to increased research seeking to remedy this deficiency (Dulewicz & Higgs, 2000). However, EI may add to the predictive power of GMA but it certainly does not trump it. Indeed, it may be that EI is not a strong predictor of performance in and of itself. Surgeons may still be successful, and many would argue more skilled since they will be detached, if they have a low level of EI. A person will most likely never even become a surgeon, though, if they have high EI that is not accompanied with high GMA. Accordingly, the most successful employee will most likely possess a high level of both emotional and general intelligence. In addition to the potential for incremental validity, the use of EI may result in reduced adverse impact for minority group members. It is a well-established fact that GMA results in adverse impact due to the large group differences as compared to the smaller group differences found in personality-based variables (cf. Ones, 1993; Ones & Anderson, 2002).

When Goleman (1995) published his book on EI there was a relative dearth of empirical studies examining the link between organizational performance and EI. This led many researchers (e.g., Abraham, 1999) to simply put forth a hypothesis, or hypotheses, without actually testing it. Almost unanimously, these researchers proposed a positive relationship between EI and performance. It is still to be determined if EI influences performance consistently or if it differs according to type of job (e.g., academic vs. work) and other potential variables. Accordingly, the current study examined various subgroups that may affect the relationship between EI and performance. Some of the subgroup analyses studied here may be described as exploratory in nature but they provide valuable insights about many of the speculative hypotheses in the EI literature.

Although the distinctiveness of EI and GMA has been shown in many studies, the same cannot be said of EI and measures of personality. Indeed, Ciarrochi et al. (2000) found it difficult to distinguish between the two constructs. This is similar to the finding reported by Davies et al. (1998) who concluded that emotional intelligence is generally indistinguishable from established personality traits. The above researchers go on to say that it is thus not surprising that EI and GMA are not correlated as few self-report measures of personality are correlated with general intelligence (see also, Ackerman & Heggestad, 1997; Ones, 1993). Davies and colleagues concluded that little uniqueness is left in EI after accounting for personality. A conflicting, albeit promising, finding was demonstrated in a measurement validation study by Schutte et al. (1998) who constructed a measure that was not significantly related to four of the Big Five personality dimensions. The study by Schutte et al. suggests that EI is in fact a construct distinct from other personality constructs. The current study statistically analyzed the above inconsistencies in order to examine more thoroughly the relationship between EI and the Big Five personality dimensions.

Many of the claims that have been put forth in relation to EI have not been substantiated by empirical research, especially on replication. Additionally, studies have not used the same, or even a few of the same, measures of EI. Of the existing measures of EI, the Bar-On Emotional Quotient Inventory (EQ-i) is one of the most widely used measures. The EQ-i is a 133-item questionnaire, rated on a five-point scale, which measures abilities and the potential for performance rather than performance itself (Bar-On, Brown, Kirkcaldy, & Thome, 2000). The EQ-i consists of five components: intrapersonal, interpersonal, adaptability, general mood and stress management. The EQ-i has previously been shown to demonstrate sufficient test-retest reliability (.85 after one month and .75 after 4 months; Bar-On, 1997).

However, the generally accepted view appears to suggest that it is difficult to measure EI and no truly robust measure currently exist (Dulewicz & Higgs, 2000). Appropriately, researchers (e.g., Sala, 2002; Wong & Law, 2002) are working to devise better measures of EI and often prefer to construct their own measure of EI when conducting studies on the topic. This is reasonable as there is still a serious lack of research examining the predictive validity of existing measures of EI; even less is known about its predictive validity in work situations (Cherniss, 2000). As Mayer et al. (2001) recognize, it took decades to construct measures of GMA. Thus, the measurement of EI is still in its infancy and numerous measures can only benefit the advancement of the construct until more is known about EI.

Recent research has increasingly used a new measure, the Multifactor Emotional Intelligence Scale (MEIS; Mayer et al., 1999). The MEIS is notable, because unlike other measures of EI, it is a performance-based test. The scale, which has shown adequate psychometric properties, consists of 12 subset tasks that are divided into four dimensions of emotional intelligence, as mentioned above. Continued examination of this scale, however, is still needed to confirm the psychometric results that have thus far been evidenced. Accordingly, Mayer et al. (1999) have recently amended the MEIS into the MSCEIT, and subsequently into the MSCEIT V2.0 (Mayer, Salovey, Caruso, & Sitarenios, in press) which has also shown promising results, in order to provide a more psychometrically sound measure of EI. In yet another attempt, Wong and Law (2002) constructed and validated a short 16-item measure of EI. Although it was an exploratory study, results indicate that the measure shows discriminant validity with personality as measured by the Big Five. This measure, along with the above-mentioned measure by Schutte et al. (1998), and various other measures, could provide more reliable and shorter measures of EI.

A primary concern with existing measures of EI is the shortage of evidence for their psychometric measurement properties. The manuals for measures of EI have indicated sufficient reliability but other studies have not consistently produced the same result. This led Davies et al. (1998), for instance, to argue that measures of EI suffer in terms of reliability. Furthermore, the available information is scattered across manuals, theses, dissertations, and technical reports and the reporting of the reliability assessments vary. Gathering all this information in a standard comparative format will facilitate a better understanding of the construct as well guide future measurement efforts. Finally, by taking into account the reliability of EI assessments, the current meta-analysis allowed for a closer approximation of the true predictive validity of emotional intelligence as well as a better estimate of the nomological net of emotional intelligence with other traditional variables (cf. Ones, 1993).

Like most predictors of performance, the influence of emotional intelligence may best be understood by examining the influence of potential moderators. Since the current meta-analysis did not test for moderation using multiple regression interaction techniques, the term subgroup will instead be used in this article to avoid confusion. The current study assessed the overall influence of EI on performance and it then examined the effects of five hypothesized subgroups that are briefly discussed below. Additionally, the analyses explored the relationship of EI with the Big Five factors of personality and GMA.

Typically, studies examining EI and performance focus on either academic or work related outcomes. At the present time, enough studies have not yet been conducted that explored the EI link with training performance so the current analysis focused on academic, work related, and a group of other outcome variables (e.g., life success) that did not fall within the first two categories. Various outcome measures include performance in actual jobs, performance in laboratory tasks, success in school (typically measured by GPA), and sports such as hockey and baseball. This provided information on relationships between EI and other additional work-relevant factors as has been previously recommended (Janovics & Christiansen, 2001). An assessment of this subgroup will show us if EI is more or less effective depending on the criterion it is to be used with. For instance, it could be that measures of EI are more valuable for predicting work success, where it could be argued emotions are more prevalent or important, than academic success. Thus, it is possible that tests of EI may have different utility depending on the criterion. In addition to examining whether the mean predictive validity differs across domains, there is also need to investigate whether there is more variability in the workplace settings than in academic settings. The demands of the workplace settings are more varied compared to academic settings, which could result in greater variability of the predictive validity of EI in workplace settings.

The studies in the current meta-analysis did not use a single measure of emotional intelligence. It was therefore appropriate to treat the measure of EI employed in each study as another subgroup variable since the tests vary in conceptual underpinnings and format. The two most common measures used were the Bar-On (1997) EQ-i and the 33-item Emotional Intelligence Scale (EIS; Schutte et al., 1998). The MEIS (Mayer & Salovey, 1997) was used in fewer studies than anticipated, considering the amount of attention that has been given to the four dimensions of EI proposed by its authors, but this was most likely a function of the short lived nature of the MEIS (i.e., it was revised into a new instrument). Other common measures include the Trait Meta Mood Scale (TMMS; Salovey, Mayer, Goldman, Turvey, & Palfai, 1995), and the Emotional Competence Inventory (ECI; Sala, 2002). In all, five different measures of EI were explored in this subgroup analysis: the MEIS, TMMS, EIS, Bar-On EQ-I, and ECI. Additionally, a category of miscellaneous EI measures included all measures that were not used by multiple studies. The measures vary considerably on characteristics such as length and reliability as well as in their specific conceptualizations. Accordingly, this may be one of the most important moderators of EI since little is known about the differences between tests. To date, few studies have pitted the measures against each other thereby making the analysis of this subgroup more important.

Studies that use emotional intelligence as a predictor typically report one of three types of possible correlations; (1) the correlation between a total EI score and performance; (2) the correlation between EI subsets and performance; and, (3) the correlation between the total EI score and performance as well as the correlations of the EI subsets and performance. The current meta-analysis will use both the four-dimension classification of Salovey and Mayer (1990), the five-dimension classification of Bar-On (1997), and total score when examining the differing influence of the dimensions used. In instances where dimensions other than the above nine were used, a panel of five independent raters were used to classify each dimension employed in a focal study into one of the four dimensions postulated by Salovey and Mayer and also into one of Bar-On’s five dimensions. The analysis of EI subscales was important to determine if all of the subscales are actually necessary for an accurate prediction of EI and also to determine if the nomological net differs across dimensions.

Studies also vary according to type of response format used to measure EI. Most of the measures (e.g., Bar-On EQ-i) use a subjective self-report format; other methods involve having someone else rate the person on EI. The current analysis will determine if other ratings of EI are more powerful than self-report measures. Kaufman and Kaufman (2001, p. 263) state that the future of EI assessment is surely based on the continued refinement of performance-based measures and not in self-report inventories. In this subgroup analysis we first separated self-reports from other ratings and then self-reports were further classified according to how the self-reports were scored. Self-reports can be scored by comparing self-ratings to consensus ratings or to expert ratings of the different options. These latter self-report methods are more objective. Indeed, Davies et al. (1998) stress the importance of objective measures of EI. Additionally, subjective measures of EI are likely to be more prone to faking than objective measures. All three methods of EI scoring were examined separately.

Studies that assess emotional intelligence and performance may also have results that are influenced by the source of the criterion data. The criterion could be objective records of performance (e.g., units produced, GPA in school) or subjective ratings. Criterion ratings can be obtained from many people including supervisors, teachers, peers, and self. The source of the rating in the included studies varied according to the type of criterion that was used. This is another subgroup that needs to be examined further.

Enough studies have now been conducted, using many different measures, that it is now necessary to use meta-analytic techniques to combine the studies in order to paint a clearer picture of the general influence of EI on various performance outcomes. This will provide a platform for subsequent research that examines the construct and predictive validity of emotional intelligence. The following hypotheses are therefore proposed:

• Hypothesis 1: A positive correlation exists between emotional intelligence and measures of performance.

• Hypothesis 2: The predictive validity of EI varies according to the setting (i.e., academic, work, or other domain) in which performance is assessed.

• Hypothesis 3: The validity of EI for predicting performance is influenced by the measure of EI used.

• Hypothesis 4: The validity of EI for predicting various criteria differs according to the dimension of EI analyzed.

• Hypothesis 5: The validity of EI differs according to the scoring method used (i.e., expert ratings vs. self reports, and consensus scored self-reports versus expert scored self-reports).

• Hypothesis 6: The validity of EI is influenced by the criterion measurement method (i.e., ratings versus organizational records, and by the source of the ratings-supervisors, self, etc.).

• Hypothesis 7: Emotional Intelligence is not correlated significantly with measures of GMA.

• Hypothesis 8: Emotional intelligence is not correlated significantly with the Big Five personality dimensions.