A latent profile analysis of college students’ achievement goal orientation

https://doi.org/10.1016/j.cedpsych.2006.10.003Get rights and content

Abstract

Achievement goal research has grown increasingly complex with the number of proposed goal orientations that motivate students. As the number of proposed goal constructs proliferates, a variety of data analytic challenges have emerged, such as profiling students on different types of goal pursuit as well as evaluating the relationships of multiple goal pursuit with different educational outcomes. The purpose of the current article is to showcase the advantages of using latent profile analysis (LPA) over other traditional techniques (such as multiple regression and cluster analysis) when analyzing multidimensional data like achievement goals. Specifically, we review the advantages of LPA over traditional person- and variable-centered analyses and then provide a critical look at three different conceptualizations of goal orientation (2-, 3-, and 4-factor) using LPA.

Introduction

In recent years, many researchers have proposed increasingly complex models to describe the construct of achievement goal orientation (Elliot, 1999, Elliot and McGregor, 2001, Pintrich, 2000). There is much debate as to whether more complex definitions are needed to fully capture the breadth of the construct, or whether more parsimonious conceptualizations can suffice (Harackiewicz et al., 2002, Midgley et al., 2001). In addition to the idea of more complex conceptualizations, a recently purported notion by several goal orientation theorists is that a person may be benefited by endorsing multiple goals (i.e., the multiple goal perspective). When a researcher uses more complex models of goal orientation along with a multiple goal perspective, another level of complexity is added to the already difficult tasks of: (1) describing the typical goal orientation profiles of students in a sample and (2) analyzing the relationships of goal orientation with other variables.

Our article has two purposes. The first is to discuss the disadvantages of using traditional data analytic techniques (e.g., multiple regression, cluster analysis) when more complex conceptualizations of goal orientation are being utilized and to demonstrate latent profile analysis (LPA), a technique which offers many advantages over traditional methods. The second aim of our article is to provide evidence as to whether more complex models of goal orientation are needed to justify their use over more simplistic models. As noted by Elliot (1999), one way in which such evidence can be collected is by showing that the more complex models are better able to predict achievement-related outcomes. To accomplish this, we use LPA with 2-, 3-, and 4-factor conceptualizations of goal orientation to better understand if more complex conceptualizations are needed to better differentiate among individuals’ goal orientations and to more accurately predict achievement-related outcomes. Before discussing the data analytic techniques that are traditionally used in goal orientation research and speculating as to the utility of more complex models, we first describe the development of achievement goal orientation theory.

For the past two decades, achievement goal orientation has been one of the primary constructs used in the study of achievement motivation (see Elliot, 2005 for a review). An individual’s achievement goal orientation represents one’s purpose for engaging in achievement-related behavior, as well as one’s orientation towards evaluating his or her competence in the achievement activity. For instance, individuals who pursue achievement-related behavior for the purpose of developing their skills and who evaluate their competence by the extent to which they have mastered the task or shown self-improvement would be labeled as having a mastery goal orientation. Alternatively, individuals who pursue achievement-related behavior for the purpose of demonstrating their skills and who evaluate their competence in relation to others would be labeled as having a performance goal orientation.

In addition to distinguishing goal orientations based on a mastery or performance focus, goal orientations also can be differentiated by whether an individual is guided by the notion of attaining positive outcomes (an approach focus) or by the notion of avoiding negative outcomes (an avoidance focus). Elliot and McGregor (2001) argued that such distinctions in achievement goals can be nicely represented along two dimensions: how competence is defined (mastery vs. performance) and how competence is valenced (approach vs. avoid). Fully crossing these two dimensions leads to four potential goal orientations: mastery-approach, mastery-avoidance, performance-approach, and performance-avoidance. Students with a mastery-approach or a mastery-avoidance goal orientation are alike in that they both are focused on mastering the material and developing their skills. The mastery-approach student, however, seeks to gain as much knowledge and skills as possible, whereas the mastery-avoidance student is focused on not losing the knowledge and skills they already have or misunderstanding the material. Similarly, students with a performance-approach or a performance-avoidance goal orientation are alike in that both are concerned about their performance in relation to their peers. The performance-approach student, however, is focused on performing better than other students, whereas the performance-avoidance student is focused on not performing worse than other students.

Early goal orientation theorists focused predominantly on distinguishing goal orientations by how competence is defined (e.g., Dweck, 1986, Nicholls, 1984). When using some form of a competency-defined framework, some would conceptualize mastery and performance goals as being a mix of both approach and avoidance strivings, whereas others would conceptualize the goals as being reflective of only approach strivings (see Elliot, 2005; for a review). However, interest in the particular conceptualization being used in a study increased when it was speculated that differences in how the goals were defined and valenced resulted in differences in the relationships found between goals and other variables. For example, researchers (Elliot and Church, 1997, Elliot and Harackiewicz, 1996) noted that the inconsistent patterns of positive, negative, and null relationships between performance goal orientations and other variables across different studies could be explained by reclassifying past research studies along valence of performance goals, with approach strivings in a performance goal orientation being most associated with positive processes and outcomes1 (e.g., intrinsic motivation, high self-esteem, effort, persistence, and performance) and avoidance strivings being most associated with negative processes and outcomes (e.g., reduced intrinsic motivation, low self-esteem, anxiety, procrastination, and poor performance).

Theoretical and historical arguments were also provided in support for the separation of performance goals into approach and avoidance components. As noted by Elliot, 1999, Elliot, 2005, the approach-avoidance distinction not only had a rich tradition in early theories of motivation, but has had a prevalent role across all major fields of psychology. These arguments, along with empirical evidence showing more explanatory power when performance goals were distinguished by valence, led many theorists to utilize a 3-factor conceptualization of goal orientation that left the mastery factor intact but separated the performance factor into the factors of performance-approach and performance-avoidance (Elliot and Church, 1997, Elliot and Harackiewicz, 1996, Middleton and Midgley, 1997, Midgley et al., 1998).

Unlike performance goals, the findings for mastery goals (typically defined as being approach-valenced) have been consistent across studies and associated with a range of positive outcomes. Thus, it was not empirical evidence that drove the division of the mastery orientation into approach and avoidance valences, but the same theoretical and historical arguments made above for the split of the performance factor. A call was made for a 4-factor conceptualization (Elliot, 1999, Pintrich, 2000) and Elliot and McGregor (2001) were among the first to offer empirical evidence supporting the utility of adding a mastery-avoidance goal orientation.

For the remainder of the article, we will use the following terminology to refer to the different conceptualizations of goal orientation. We term the conceptualization involving only mastery-approach and performance-approach orientations as the 2-factor model; the conceptualization involving mastery-approach, performance-approach, and performance-avoidance as the 3-factor model; and the same conceptualization with the addition of the mastery-avoidance orientation as the 4-factor model2.

Several researchers have chosen to demonstrate support for the more complex 3- and 4-factor goal orientation models by showing how the separation of goal orientations by valence yields a larger number of distinct factors that predict (and are predicted by) different variables (e.g., Conroy et al., 2003, Elliot and Church, 1997, Elliot and McGregor, 2001, Middleton and Midgley, 1997, Skaalvik, 1997). Although these studies are important in the investigation of the utility of more complex models, they oftentimes utilize variable-centered as opposed to person-centered analyses. In a variable-centered analysis (such as regression), process and outcome variables are typically related to each goal orientation separately. In a person-centered analysis (such as cluster analysis), the differences in process and outcome variables are examined for various subgroups of students, with subgroups consisting of students who have similar profiles across the various dimensions of goal orientation.

The use of person-centered analytic techniques is particularly important for goal orientation researchers interested in an increasingly popular notion in goal orientation theory known as the multiple goal perspective. The multiple goal perspective states that an individual is optimally motivated by endorsing more than one goal orientation. The idea of adopting multiple goals simultaneously is not new to the field of goal orientation research (Barron and Harackiewicz, 2000, Harackiewicz et al., 2002, Midgley et al., 2001, Wentzel, 1992). However, there is still debate regarding which combination of goals leads to the most adaptive outcomes, and how the effects of multiple goals are best revealed. For researchers who favor the multiple-goal perspective, a person-centered approach to investigating the utility of more complex goal orientation theories is warranted particularly since, as noted by Bråten and Olaussen (2005), “…persons move through instructional environments, not variables…” (p. 360). In the following section, we describe the traditional analyses that have been used by researchers to capture the typical levels of goal orientation in their sample and to relate the different levels to processes and outcomes. The section begins with a review of variable-centered methods, including descriptive statistics, correlations, and multiple regression, and ends with a review of person-centered methods, including median split techniques and cluster analysis. When describing different techniques that have been used, we have highlighted a number of example studies that have adopted a particular approach. Our goal is to provide the readers with examples, however it is not our intent to identify or single out a particular researcher or article. In fact, many of the researchers noted for adopting a less optimal approach (including past achievement goal work of the second author of the current article) now adopt and use more sophisticated approaches.

There are a variety of different methods researchers could employ to describe the levels of goal adoption in a sample. Descriptive statistics would indicate the typical level and variability of each goal, but fail to capture the relationships among goals. To obtain this information, early goal investigators calculated bivariate correlations between different goal measures. Bivariate correlations revealed that measures of mastery and performance goals generally shared null (or slightly positive) relationships (see Harackiewicz, Barron, & Elliot, 1998 for a review). Thus, rather than being motivated by one goal or the other, any combination of mastery and performance goals appears possible for any given person. If any combination is possible, it is of interest to ask whether there are certain combinations of goals that are more common than others.

In addition to describing the goal orientation profiles in a sample, researchers must decide whether to examine goals separately or together when examining the relationships of goals with other variables. For example, a number of early investigations were limited to simple, correlational approaches (Miller et al., 1993, Nolen, 1988) that just evaluated the bivariate correlations of each goal with different types of educational outcomes. However, it may be more informative to study how certain combinations of mastery and performance goals relate to other variables rather than how each goal relates separately. One popular method researchers have adopted to examine the relationship of multiple goals with other variables is multiple regression. Investigations by numerous researchers have used multiple regression (e.g., Elliot and Church, 1997, Harackiewicz et al., 1997, Kaplan and Midgley, 1997, Middleton and Midgley, 1997, Skaalvik, 1997).

The use of regression models to study the relationship of goals with other variables becomes increasingly complicated as the conceptualization of the construct becomes more complex. For instance, if studying interactive relationships of the goals with other variables, use of the 4-factor conceptualization in a regression model would entail including the four-way interaction, all three- and two-way interactions, and all main effects (e.g., see Elliot & McGregor, 2001). Accurate estimation of all parameters would require a large sample size and interpretation of the results might be difficult and possibly complicated by multicollinearity. Even if a researcher were able to overcome such problems, multiple regression techniques are limited in that they only allow the researcher to describe the relationships of goals with other variables, not to characterize the common goal orientation profiles in their sample.

Median split techniques are a seductively easy way to: first, identify the most common goal patterns in the sample and then second, examine the relationships of such patterns or “profiles” with other variables. Using median split procedures, participants are first categorized as “high” if their score falls above the median on a goal factor or “low” if their score falls below the median. When using a 2-factor conceptualization of goals, median split procedures have the possibility of capturing four distinct profiles3. Once participants are classified into one of these four groups, differences among profiles in outcome variables can be examined using analysis of variance (ANOVA) techniques.

Although easy to implement, there are a number of known problems with median split procedures (Maxwell & Delaney, 1993). In fact, many of the goal orientation studies that have utilized the technique often recognize its limitations and supplement their median-splits with additional analyses (e.g., Kaplan and Midgley, 1997, Meece and Holt, 1993). Perhaps most problematic is the dependency of the procedure on the sample median being used. Because the median may vary in value, comparison of the results across studies is complicated.

As noted by MacCallum, Zhang, Preacher, and Rucker (2002) in their review of the troubles associated with dichotomizing continuous variables, a problem with median split procedures has to do with the questionable homogeneity of the cases classified in each profile as well as the problematic use of labels such as “low” and “high” to characterize cases falling below and above the median. A solution to the above problem may be to split each goal factor into more than two categories. This would help create more homogeneous groups, but with an increase in the number of categories, the number of possible profiles increases exponentially (# possible profiles = # of categories# factors). The number of possible profiles becomes larger, and therefore less parsimonious, when using more complex conceptualizations of goal orientation.

When the purpose is to divide persons into homogeneous subgroups, cluster analysis can be used as an alternative to median split techniques. Cluster analysis is a statistical technique for finding “clusters” of observations that have similar values on a set of variables. In this exploratory technique, clusters are created such that the differences within clusters on a set of measures are minimized and the differences between clusters are maximized. Readers interested in cluster analytic techniques should consult: Aldenderfer and Blashfield, 1984, Everitt et al., 2001, Hair et al., 1998, Kaufman and Rousseeuw, 2005.

There are a variety of different cluster analysis methods to choose from. Some goal orientation researchers, such as Bråten and Olaussen (2005), have employed agglomerative hierarchical techniques which start with each observation in its own cluster and proceed by combining clusters with similar values on the cluster indicators, which are the variables used as input into the cluster analysis. A common method for combining clusters in a hierarchical analysis is Ward’s method, which creates clusters so that the within cluster variance across all variables is as small as possible. It is largely the decision of the researcher as to which of the solutions, between N clusters and one cluster, to interpret. Although there are statistics that can be used for such an endeavor (e.g., pseudo F-statistic; Calinksi & Harabasz, 1974), many of these statistics are of questionable utility or may only be appropriate for use with particular kinds of data (Milligan & Cooper, 1985). Instead of relying upon statistics, researchers typically examine a variety of different cluster solutions and use theory and oftentimes, their own judgment to decide upon a solution. The lack of rigorous guidelines to aid in the selection of a solution is an often cited weakness of hierarchical cluster analysis.

The subjectivity associated with this method can be overcome somewhat by showing that one’s solution replicates well when employing a different method of clustering on a separate sample. To this end, some researchers choose to supplement their hierarchical analysis by executing a non-hierarchical cluster analysis (a.k.a. optimization clustering) with a separate sample. In non-hierarchical cluster analysis the number of clusters is specified in advance and some initial partition, often based on the results of a hierarchical analysis, is used to assign observations to clusters. Observations are then reassigned to different clusters until a criterion is met. As with hierarchical clustering, a commonly used criterion is to create clusters so that the within-group variance across all cluster indicators is a minimum.

A number of studies using either hierarchical, non-hierarchical, or some combination of the two cluster analytic methods in the motivation and achievement goal orientation literature do exist (e.g., Ainley, 1993, Bembenutty, 1999, Bråten and Olaussen, 2005, Etnier et al., 2004, Hodge and Petlichkoff, 2000, Kaplan and Bos, 1995, Meece and Holt, 1993, Ntoumanis, 2002, Salisbury-Glennon et al., 1999, Turner et al., 1998, Urdan and Midgley, 1994). Trying to summarize the results of these cluster analytic studies is quite difficult for a variety of reasons. First, synthesis of the results is complicated by the fact that a variety of different populations are the focus of the studies employing cluster analytic techniques. Another complication is the variety of different variables that are used as cluster indicators. This is quite problematic because it is well known that the results of a cluster analysis are quite dependent on the variables that are used as cluster indicators. Most importantly, trying to synthesize the results of different cluster analytic studies is difficult because of the subjective nature of cluster analysis. Because of the subjectivity associated with traditional cluster analytic techniques, researchers are turning more towards model-based cluster analytic techniques, such as LPA, which offer more rigorous criteria for determining the number of clusters to retain in addition to several other advantages.

LPA is a latent variable modeling technique that is known in the literature by a variety of names, including latent class cluster analysis (Vermunt & Magidson, 2002) and finite mixture modeling (McLachlan & Peel, 2000). A good introduction to this technique and to latent variable modeling in general can be found in Magidson and Vermunt, 2002, Magidson and Vermunt, 2004, Muthén, 2001, Muthén, 2004, Muthén and Muthén, 2000, Vermunt and Magidson, 2002. A more technical but very thorough treatment is given in McLachlan and Peel (2000). The goal of LPA is the same as that of cluster analysis: to identify clusters of observations that have similar values on cluster indicators. The main difference between LPA and traditional cluster analytic techniques is that LPA is model-based, whereas hierarchical and most non-hierarchical applications of cluster analysis are not.

LPA is a type of latent variable mixture model. The term latent variable in this situation is referring to the latent categorical variable of cluster membership. This latent categorical variable has K number of categories or clusters. A person’s value on this variable is thought to cause his or her levels on the observed cluster indicators, which in our situation would be the different measures of goal orientation. The term mixture is referring to the notion that the data are not being sampled from a population that can be described by a single probability distribution. Instead, the data are conceived as being sampled from a population composed of a mix of distributions, one for each cluster, with each cluster distribution characterized by its own unique set of parameters.

When latent variable mixture modeling is used with only continuous cluster indicators, it is often called LPA. When only categorical variables are used, the technique is often called latent class analysis (LCA). This distinction is not necessary because it is the same model, a latent variable mixture model, which is being used in both situations. In fact, the distinction made between LPA and LCA seems even more unnecessary when one considers the fact that both categorical and continuous cluster indicators can be used simultaneously in latent variable mixture models.

Although standard clustering techniques can also be used with both categorical and continuous cluster indicators, the use of latent variable mixture modeling for such a purpose is relatively less difficult. Mixture modeling is also advantageous because indicators on different scales do not need to be transformed prior to their input into the analysis. With traditional clustering techniques, it is recommended that variables on different scales or with widely divergent variances be standardized prior to the analysis. With latent variable mixture modeling, no such transformation is necessary.

To further illustrate the notion of LPA, consider an example where a continuous variable yi is used as a single indicator of cluster membership for person i in our sample of size N (i = 1,  ,N). To make the example more concrete, one could consider the use of just a single goal orientation factor (e.g., mastery-approach) as the continuous variable. Although the number of clusters, K, is not typically known a priori, suppose there are two different clusters of persons (K = 2) in our population. In mixture modeling this would translate into the presence of two different distributions, typically assumed to be normal, from which our data were sampled. Note that although the population distribution is assumed to be a mixture of two normal distributions in this example, the population distribution itself need not be normal.

In LPA, it is possible for a unique set of parameters to be estimated for each cluster. For instance, parameters μ1 and σ12 could be estimated for Cluster 1 and parameters μ2 and σ22 could be estimated for Cluster 2. This is the most complex model that could be estimated for this example and more parsimonious models could be specified by constraining some of the parameters to be equal across clusters. For example, one could allow the means for each distribution to remain unique but constrain the variances to be equal across clusters; or one could allow the variances to remain unique across clusters and constrain the means to be equal.

In addition to the parameters of each cluster’s distribution, LPA also provides estimates for the mixing proportion or the weight given to each cluster in the population. The model for this example can be represented using the following equation:f(yi|θ)=π1f1(yi|μ1,σ12)+π2f2(yi|μ2,σ22),which shows that the distribution of our cluster indicator, yi, given the model parameters (θ=π1,μ1,σ12,π2,μ2,σ22) is a weighted mixture of two separate distributions, each characterized by a unique set of parameters. The weights in a mixture model are non-negative and must sum to one. If the weights in our example were estimated to be π1 = .60 and π2 = .40, it would imply that 60% of our population can be described by the parameters of Cluster 1 and 40% of our population by the parameters of Cluster 2.

When more than one continuous cluster indicator is used in LPA, the multivariate distribution of the r cluster indicators, contained in vector yi for person i, is conceived of as a weighted mixture of K different distributions, typically assumed to be multivariate normal. For instance, if the subscales associated with either the 2-, 3-, or 4-factor conceptualization of goal orientation were used as cluster indicators, a multivariate LPA model would need to be utilized. The multivariate representation of Eq. (1) with r indicators and K clusters is,f(yi|θ)=k=1Kπkfk(yi|μk,Σk).As with the univariate model, the weights in Eq. (2) are constrained to be non-negative and must sum to one. In the univariate example shown in Eq. (1), the distribution for each cluster was defined by only two parameters, a mean and a variance. In the multivariate case, the distribution for each cluster k is now defined by a mean vector μk and covariance matrix Σk.

In the univariate example using a single cluster indicator, we discussed how a variety of different models could be fit to the data by allowing the means and/or the variances of the single cluster indicator to freely vary or be constrained across clusters. When dealing with multiple cluster indicators, the number of possible models that can be specified increases substantially. Consider a multivariate example using two clusters (K = 2) and four cluster indicators (r = 4). Numerous models are possible when just considering the mean vectors. For instance, the most complex model for the mean vectors would allow the means of all four indicators to vary across the two clusters, resulting in eight means to be estimated. A more simplistic model would constrain the mean vector to be equal across clusters, resulting in only four means to be estimated. These are just two of a variety of different models that could be specified for the mean vectors with other examples including those that allow the means of only certain indicators to remain constant across clusters.

Of course, above we only consider the mean vectors for each cluster, which typically are specified to freely vary across both indicators and clusters. When considering the covariance matrix for a given cluster (Σk), the focus is no longer on the average levels of the indicators in each cluster, but on the extent to which each indicator varies and how the indicators relate to one another. A variety of different specifications for Σk are shown in Table 1. We will describe each of the specifications below, using the multivariate example that includes two clusters (K = 2) and four cluster indicators (r = 4).

A parsimonious form of Σk is shown for Model A, where the variances are allowed to differ across indicators within a cluster (thus the different subscripts for the variances), but are constrained to be equal across clusters. Additionally, the indicators are constrained to be unrelated to one another both within and across clusters (e.g., all covariances are zero). In our example, Model A would result in the estimation of four variances and no covariances. A more complex version of this model is shown with Model C, where the variances are now allowed to differ across clusters (thus the additional k subscript for each variance). In our example, Model C would result in the estimation of eight variances and no covariances.

Model A can also be made more complex by allowing the covariances among the indicators to be freely estimated within a cluster, but with both the variances and covariances constrained to be the same across clusters. The resulting model is Model B and four variances and six covariances would be estimated for our example. Model B can be made more complex by allowing the variances to differ across indicators, but constraining the covariances to remain the same across clusters. This specification results in Model D and in our example eight variances and six covariances would be estimated. The most complex model is Model E which allows both the variances and covariances to vary across clusters. In our example, we would be estimating eight variances and 12 covariances using Model E. Note that the models in Table 1 are nested; for instance, Model A is obtained from Model B by setting all covariances in Model B to zero.

None of the specifications shown in Table 1 can be used in LPA to obtain the results from traditional cluster analysis because the former is a model-based procedure while the latter is not4. However, one of the forms of Σk can be altered to convey the data for which traditional cluster analytic procedures are most appropriate. Traditional cluster analytic results are most appropriate to use when Σk follows the Model C specification, with the additional restriction of constraining the variances of cluster indicators to be equal within a cluster. Thus, an additional advantage of LPA over traditional methods is that it can accommodate data having a variety of different forms of Σk.

Another advantage of LPA is the availability of more rigorous criteria to use in deciding upon one’s final model. In order to understand some of the criteria, it is first important to understand the estimation methods used in LPA. Although there are several different estimation methods to choose from in LPA, model parameters are commonly estimated using maximum likelihood (ML) estimation via the EM algorithm. Readers interested in ML estimation should consult the primer by Enders, 2005, McLachlan and Peel, 2000 if interested in how ML estimation using the EM algorithm is employed in mixture modeling. Conceptually in ML, several different sets of model parameter estimates are “tried out” with the data. Each set is associated with a likelihood value, which is the probability of observing the sample data assuming that set of parameter estimates. Because it is the intent of ML estimation to find the parameter estimates most likely to have given rise to the sample data, the final parameter estimates chosen are those associated with the highest likelihood value.

The logarithmic value of the likelihood (the log-likelihood or LL) is often used because it is more mathematically tractable. The LL of the final parameter estimates is used as a measure of model fit with higher values (e.g., closer to 0) indicating better fit than lower values. For models specifying the same number of clusters, more complex models (e.g., Model B) will always fit the data better than more simplistic models (e.g., Model A) and thus will always have LL values that are higher. A χ2 difference test can be used to determine if the more complex model fits significantly better than the more simplistic model. This test entails taking two times the difference of the log-likelihoods of nested models and comparing the value against a χ2 distribution with degrees of freedom equal to the difference in the number of parameters being estimated. The χ2 difference test only can be used to decide among the models in Table 1 when the models specify the same number of clusters.

Another significance test, the Lo–Mendell–Rubin likelihood ratio test (LMR; Lo, Mendell, & Rubin, 2001) offered in the output of Mplus version 3.01 (Muthén & Muthén, 2004), can be used to compare the fit of models that specify different number of clusters, but that utilize the same parameterization. When estimating a model with K clusters, the null hypothesis of this test is that the data have been generated by a model with K  1 clusters, with the researcher typically specifying the omitted cluster in the K  1 solution as being the smallest cluster in the K solution. A small p-value associated with the LMR test supports the retention of a more complex solution with at least K clusters.

Other fit statistics are employed in LPA that can aid the researcher in deciding upon the number of clusters to retain. The Bayesian information criterion (BIC; Schwartz, 1978) can be used to compare models with different numbers of clusters and/or specifying different parameterizations. The BIC is simply a form of the log-likelihood, specificallyBIC=-2LL+plnN,where p is the number of parameters being estimated and N is the sample size. A sample-size adjusted BIC can also be consulted, which uses N (where N = (N + 2)/24) as opposed to N in Eq. (3). Lower values of both the BIC and the sample-size adjusted BIC are indicative of better model fit.

The BIC may seem to be a more favorable index over the χ2 difference test or LMR because it can be used to compare the fit of any model, regardless of the parameterization being used or the number of clusters specified. However, the BIC does not provide a significance test to assess the fit of competing models. It is for this reason that both the χ2 difference test and the LMR should be employed since both can be used to examine whether the fit of a model is significantly better than the fit of another model. While the χ2 difference test is used with models having different parameterizations but specifying the same number of clusters, the LMR is used with models having the same parameterization but specifying different numbers of clusters.

In addition, tests of multivariate skewness and kurtosis (SK) described in Muthén (2004) can be used to assess the fit of the model to the data. For a given model, multiple data sets are generated according to the estimated model parameters. Values of multivariate skewness and kurtosis are then calculated for each data set and used to create a distribution. The values of multivariate skewness and kurtosis in the observed sample are then compared to these distributions with the resulting two-sided probability value (i.e., p-value) indicating how likely the observed values are given the values estimated by the model-generated data. High p-values associated with the SK tests are indicative of model fit; low p-values indicate that the model does not fit the data. Muthén (2004) illustrated the use of the SK tests, but also noted that these tests need further investigation.

A researcher decides upon the final model by consulting the BICs, LMRs, χ2 difference tests, and SK tests. As with traditional methods, it is also recommended that the cluster profiles be inspected with consideration of theory, sample size, and the uniqueness of the profiles. After the decision regarding the final model has been made persons are classified into clusters. In order to classify a given person, the probabilities of belonging in each cluster are first calculated. These K posterior probabilities are calculated for each person using the following equation:πk|yi=πkfk(yi|μk,Σk)k=1Kπkfk(yi|μk,Σk).

Eq. (4) utilizes the final model parameters as well as the individual’s values on the cluster indicators. A commonly used method of assigning persons to clusters after the posterior probabilities are calculated is modal assignment, where assignment is made to the cluster associated with the largest of the posterior probabilities.

At this point, the sample statistics for each cluster are examined to ensure that the values conform to the population parameters estimated by the model. For example, confidence in the fit of Model A to the data is increased when the sample covariances or correlations among indicators are null in all clusters. There are two different means by which the sample statistics for a given cluster can be computed. The first uses only those observations assigned to Cluster k to compute the sample statistics for Cluster k. The second uses all observations to compute the sample statistics for Cluster k with observations weighted by the posterior probabilities associated with Cluster k.

Posterior probabilities are also used to calculate the classification table and entropy statistics, both of which are used in assessing the classification utility of the model. There are as many rows and columns in the classification table as there are clusters. The kth row of the classification table contains K posterior probabilities, averaged across only those persons assigned to the kth cluster. In the kth row, the largest average posterior probability will be associated with Cluster k with all other averages in that row being lower.

To illustrate, a hypothetical classification table for a four cluster solution is shown in Table 2. The first row contains the averages based on only the 308 persons that were assigned to the first cluster. Because these persons were assigned to the first cluster using modal assignment, as anticipated the average posterior probability for these persons is highest for Cluster 1. Similarly, the highest average for those 285 persons assigned to the second cluster is associated with Cluster 2. Note that these averages, the averages associated with the clusters to which persons were assigned, are captured in the main diagonal of the classification table. For this reason, higher averages on the main diagonal of the classification table reflect greater accuracy in the assignment of persons to clusters.

The remaining averages can be examined to determine which particular clusters may not be distinct from one another. For instance, the third row of the table contains the average posterior probabilities for persons assigned to the third cluster. As expected, the highest average (.709) is associated with the cluster to which these persons were assigned, Cluster 3. The second highest average for persons assigned to the third cluster is associated with Cluster 4 (.274). Because the value of this average is sizeable, it indicates some overlap between Clusters 3 and 4.

Although the classification table is more informative, its information can be captured using a single statistic. This statistic is known as the entropy statistic and it ranges from 0 to 1 with higher values indicative of higher classification utility. As of yet, there is no cutoff value for the entropy statistic; that is, there is no set value that needs to be exceeded in order for researchers to deem their model as having adequate model classification utility. The statistic is best used to compare the classification utility of different models fit to the same sample or of the same model fit to different samples. The entropy statistic E is calculated using the posterior probabilities from Eq. (4), the overall sample size N and number of clusters K.E=1-i=1Nk=1K(-πk|yilnπk|yi)NlnK

In traditional clustering techniques persons are assigned to clusters on an all-or-none basis. In contrast, LPA allows membership of a person to each cluster to a certain degree, allowing for fractional cluster membership as captured in the posterior probabilities. Although the modal assignment of persons to clusters results in a person being classified in only one cluster in LPA, the entropy statistic and classification table in LPA can be used to examine the degree to which this classification is accurate.

When creating the classification table and computing the entropy statistic, the posterior probabilities are calculated using the same sample used to estimate the model parameters. An advantage of LPA over traditional clustering methods is the ease with which the model parameter estimates of one sample can be used to compute the posterior probabilities and assign cluster membership to persons in a second sample. For instance, if the same cluster indicators have been collected from a second sample in the same population, Eq. (4) can easily be used to classify persons into clusters. For cross-validation purposes, the entropy statistic and classification table for this second sample can then be examined to assess the utility of the model to classify persons in another sample.

After a researcher decides upon the final model, the typical next step is to examine the relation between cluster membership and external variables, variables that were not used to determine cluster membership. This is often executed to offer validity evidence for the cluster solution. There are two ways in which the relationship between clusters resulting from LPA and external variables can be examined. The first is to use ANOVA with each external variable serving as the dependent variable and cluster membership as the independent variable. Similarly, multiple regression can be used with each external variable serving again as the dependent variable, but with the posterior probabilities of cluster membership serving as the independent variables. The advantage of using the posterior probabilities to represent cluster membership is that the accuracy of classifying persons into clusters can be incorporated into the analysis.

A second approach to exploring the relationships between external variables and cluster membership is to include the external variables directly in the LPA model. In this approach, external variables can be specified to have relationships with the latent categorical variable of cluster membership and/or the cluster indicators in a variety of different ways. For instance, some variables may be specified as background variables (a.k.a., covariates) that can be used to predict cluster membership, while other variables may be specified as outcomes of cluster membership. Use of this approach requires a solid understanding of one’s variables as well as the relationships that would be anticipated based on theory. Consult Muthén (2004) for further information and examples using this integrated strategy.

To date, there are only a small number of studies that have used person-centered analytic methods to examine the goal orientation profiles of college students. Furthermore, our review of the literature revealed that the use of small samples and cluster analysis procedures further complicated our ability to generalize the results of these studies. Thus, the purpose of the present study was to add to the existing literature by using both a large sample (N = 1868) and a more advanced statistical technique, LPA, to identify the typical goal orientation profiles of college students. In addition, we compared the three latent profile analysis solutions obtained when using as cluster indicators the factors in the different conceptualizations of goal orientation. Specifically, we examined the LPA solutions obtained using a 2-, 3-, and 4-factor model of goal orientation to better understand if more complex conceptualizations were needed to: (1) better differentiate among individuals’ goal orientations and (2) to more accurately predict achievement-related outcomes.

We used three steps to accomplish the purposes of our study. In Step 1, LPA was used with data from 1868 college students (Sample 1) to classify students into clusters with separate sets of analyses being conducted, one for each conceptualization of goal orientation. Step 2 was used to examine the classification accuracy of our three final solutions obtained in Step 1. In this step, the classification accuracy of the model in Sample 1 was examined by using the resulting model parameters to classify college students from a second sample (N = 2290) into goal orientation clusters.

In Step 3 of our study, we used multiple regression procedures to examine the extent to which the cluster membership related to measures of motivational disposition and academic achievement. By using the estimated posterior probabilities of cluster membership as predictors in our regression models, we were able to incorporate the classification accuracy of our LPA models when examining the differences among the clusters in measures of motivational disposition and academic achievement. Examination of the relation of cluster membership to motivational disposition was undertaken to offer validity evidence for the cluster solutions found in Step 1. The extent to which the cluster membership related to academic achievement was pursued to determine if the cluster solutions based on the more complex conceptualizations of goal orientation could more accurately predict an achievement-related outcome.

Section snippets

Samples

We utilized two samples of college sophomores from a mid-sized, Southeastern university who completed achievement goal measures during a semi-annual institution-wide Assessment Day. Data for the first sample (Sample 1) were collected in February 2003, and data for the second sample (Sample 2) were collected in February 2004. Multivariate outliers were detected by calculating Mahalanobis distance; 11 outliers in Sample 1 and 28 outliers in Sample 2 were deleted, yielding a final sample size of

Step 1: Identifying the cluster solutions for the 2-, 3-, and 4-factor conceptualizations of goal orientation in Sample 1

The BICs and sample-size adjusted BICs for a selection of the Step 1 models are shown in Table 5. Values bolded in the table indicate models where the performance-approach mean for a single cluster had to be fixed at a particular value in order to achieve convergence

Discussion

One of the main purposes of the present study was to compare the different cluster solutions that resulted when the factors of different conceptualizations of goal orientation were used as cluster indicators. To this end, LPA was used to classify college students into different goal orientation profiles using 2-, 3-, and 4-factor conceptualizations of goal orientation. Our results indicated that five different goal orientation profiles were needed to classify college students in the 2- and

Conclusions

Researchers desiring to take a more person-centered approach to analyzing their data often turn towards median-split techniques or cluster analysis for the purposes of classifying students into homogeneous subgroups. In this article, we showcased a latent variable technique, LPA, which could be used for such a purpose and illustrated many of the advantages associated with its use. By using LPA, we were able to utilize more rigorous criteria when deciding upon our final cluster solutions,

References (73)

  • J.W. Atkinson

    An introduction to motivation

    (1964)
  • J.W. Atkinson

    The mainsprings of achievement-oriented activity

  • J.D. Banfield et al.

    Model-based Gaussian and non-Gaussian clustering

    Biometrics

    (1993)
  • K.E. Barron et al.

    Achievement goals and optimal motivation: testing multiple goal models

    Journal of Personality and Social Psychology

    (2001)
  • J. Brophy

    Goal theorists should move on from performance goals

    Educational Psychologist

    (2005)
  • R.B. Calinksi et al.

    A dendrite method for cluster analysis

    Communications in Statistics

    (1974)
  • J. Cohen

    Statistical power analysis for the behavioral sciences

    (1988)
  • D.E. Conroy et al.

    A 2 x 2 achievement goals questionnaire for sport: evidence for factorial invariance, temporal stability, and external validity

    Journal of Sport & Exercise Psychology

    (2003)
  • C.S. Dweck

    Motivational processes affecting learning

    American Psychologist

    (1986)
  • A.J. Elliot

    Approach and avoidance motivation and achievement goals

    Educational Psychologist

    (1999)
  • A.J. Elliot

    A conceptual history of the achievement goal construct

  • A.J. Elliot et al.

    A hierarchical model of approach and avoidance achievement motivation

    Journal of Personality and Social Psychology

    (1997)
  • A.J. Elliot et al.

    Approach and avoidance achievement goals and intrinsic motivation: a mediational analysis

    Journal of Personality and Social Psychology

    (1996)
  • A.J. Elliot et al.

    A 2 x 2 achievement goal framework

    Journal of Personality and Social Psychology

    (2001)
  • C.K. Enders

    Estimation by maximum likelihood

  • J.L. Etnier et al.

    An examination of goal orientation profiles and motivation in adult team sport

    International Journal of Sport Psychology

    (2004)
  • B.S. Everitt et al.

    Cluster analysis

    (2001)
  • Finney, S. J., & Davis S. L. (2003, April). Examining the invariance of the Achievement Goal Questionnaire across...
  • S.J. Finney et al.

    Examining the psychometric properties of the achievement goal questionnaire in a general academic context

    Educational and Psychological Measurement

    (2004)
  • P. Gagné

    Mean and covariance structure mixture models

  • H. Grant et al.

    Clarifying achievement goals and their impact

    Journal of Personality and Social Psychology

    (2003)
  • K.A. Hagtvet et al.

    The motive to avoid failure and test anxiety responses: empirical support for integration of two research traditions

    Anxiety, Stress, and Coping

    (1997)
  • J.F. Hair et al.

    Multivariate data analysis

    (1998)
  • J.M. Harackiewicz et al.

    Predictors and consequences of achievement goals in the college classroom: maintaining interest and making the grade

    Journal of Personality and Social Psychology

    (1997)
  • J.M. Harackiewicz et al.

    Rethinking achievement goals: when are they adaptive for college students and why?

    Educational Psychologist

    (1998)
  • J.M. Harackiewicz et al.

    Revision of achievement goal theory: necessary and illuminating

    Journal of Educational Psychology

    (2002)
  • Cited by (489)

    View all citing articles on Scopus
    View full text