Skip to main content
Top
Gepubliceerd in: Journal of Psychopathology and Behavioral Assessment 3/2006

01-09-2006 | Original Article

Careless Responding to Reverse-Worded Items: Implications for Confirmatory Factor Analysis

Gepubliceerd in: Journal of Psychopathology and Behavioral Assessment | Uitgave 3/2006

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Many self-report measures include some items worded in the direction opposite to that of other items. These so-called reverse-worded (RW) items can reduce the reliability and validity of a scale, and frequently form a separate method factor that does not appear to be substantively meaningful. One possible explanation for factors defined by RW items is respondent carelessness. The purpose of the present study was to evaluate whether relatively few careless responders to RW items can influence confirmatory-factor-analysis model fit enough that researchers would likely reject a one-factor model for a unidimensional scale. Results based on simulations indicated that if at least about 10% of participants respond to RW items carelessly, researchers are likely to reject a one-factor model for a unidimensional scale.
Literatuur
go back to reference Barnette, J. J. (2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60, 361–370.CrossRef Barnette, J. J. (2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60, 361–370.CrossRef
go back to reference Benson, J. (1987). Detecting item bias in affective scales. Educational and Psychological Measurement, 47, 55–67.CrossRef Benson, J. (1987). Detecting item bias in affective scales. Educational and Psychological Measurement, 47, 55–67.CrossRef
go back to reference Bentler, P. M. (1990). Comparative fit indices in structural models. Psychological Bulletin, 107, 238–246.CrossRef Bentler, P. M. (1990). Comparative fit indices in structural models. Psychological Bulletin, 107, 238–246.CrossRef
go back to reference Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software. Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software.
go back to reference Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.CrossRef Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.CrossRef
go back to reference Birnbaum, A. (1968). Some latent trait models. In F. M. Lord & M. R. Novick (eds), Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison & Wesley. Birnbaum, A. (1968). Some latent trait models. In F. M. Lord & M. R. Novick (eds), Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison & Wesley.
go back to reference Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods and Research, 21, 230–258.CrossRef Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods and Research, 21, 230–258.CrossRef
go back to reference Conrad, K. J., Wright, B. D., McKnight, P., McFall, A., Fontana, A., & Rosenheck, R. (2004). Comparing traditional and Rasch analyses of the Mississippi PTSD scale: Revealing limitations of reverse-scored items. Journal of Applied Measurement, 5, 15–30.PubMed Conrad, K. J., Wright, B. D., McKnight, P., McFall, A., Fontana, A., & Rosenheck, R. (2004). Comparing traditional and Rasch analyses of the Mississippi PTSD scale: Revealing limitations of reverse-scored items. Journal of Applied Measurement, 5, 15–30.PubMed
go back to reference Drasgow, F., Levine, M. V., & Williams, E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67–86. Drasgow, F., Levine, M. V., & Williams, E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67–86.
go back to reference Greenberger, E., Chen, C., Dmitrieva, J., & Farruggia, S. P. (2003). Item-wording and the dimensionality of the Rosenberg self-esteem scale: do they matter? Personality and Individual Differences, 35, 1241–1254.CrossRef Greenberger, E., Chen, C., Dmitrieva, J., & Farruggia, S. P. (2003). Item-wording and the dimensionality of the Rosenberg self-esteem scale: do they matter? Personality and Individual Differences, 35, 1241–1254.CrossRef
go back to reference Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
go back to reference Jöreskog, K. G., & Sörbom, D. (1981). LISREL V: Analysis of linear structural relationships by the method of maximum likelihood. Chicago: National Educational Resources. Jöreskog, K. G., & Sörbom, D. (1981). LISREL V: Analysis of linear structural relationships by the method of maximum likelihood. Chicago: National Educational Resources.
go back to reference Knight, R. G., Chisholm, B. J., Marsh, N. V., & Godfrey, H. P. (1988). Some normative, reliability, and factor analytic data for the revised UCLA Loneliness Scale. Journal of Clinical Psychology, 44, 203–206.PubMedCrossRef Knight, R. G., Chisholm, B. J., Marsh, N. V., & Godfrey, H. P. (1988). Some normative, reliability, and factor analytic data for the revised UCLA Loneliness Scale. Journal of Clinical Psychology, 44, 203–206.PubMedCrossRef
go back to reference Lai, J. C. L. (1994). Differential predictive power of the positively versus the negatively worded items of the life orientation test. Psychological Reports, 75, 1507–1515. Lai, J. C. L. (1994). Differential predictive power of the positively versus the negatively worded items of the life orientation test. Psychological Reports, 75, 1507–1515.
go back to reference Levine, M. V., & Drasgow, F. (1982). Appropriateness measurement: Review, critique, and validating studies. British Journal of Mathematical and Statistical Psychology, 35, 42–56. Levine, M. V., & Drasgow, F. (1982). Appropriateness measurement: Review, critique, and validating studies. British Journal of Mathematical and Statistical Psychology, 35, 42–56.
go back to reference Marsh, H. W. (1986). Negative item bias in ratings scales for preadolescent children: A cognitive-developmental phenomenon. Developmental Psychology, 22, 37–49.CrossRef Marsh, H. W. (1986). Negative item bias in ratings scales for preadolescent children: A cognitive-developmental phenomenon. Developmental Psychology, 22, 37–49.CrossRef
go back to reference Marsh, H. W. (1996). Positive and negative global self-esteem: A substantively meaningful distinction or artifactors? Journal of Personality and Social Psychology, 70, 810–819.PubMedCrossRef Marsh, H. W. (1996). Positive and negative global self-esteem: A substantively meaningful distinction or artifactors? Journal of Personality and Social Psychology, 70, 810–819.PubMedCrossRef
go back to reference Meijer, R. R., & Sijtsma, K. (1995). Detection of aberrant item score patterns: A review of recent developments. Applied Measurement in Education, 8, 261–272.CrossRef Meijer, R. R., & Sijtsma, K. (1995). Detection of aberrant item score patterns: A review of recent developments. Applied Measurement in Education, 8, 261–272.CrossRef
go back to reference Meijer, R. R., & Sijtsma, K. (2001). Methodological review: Evaluating person fit. Applied Psychological Measurement, 25, 107–135.CrossRef Meijer, R. R., & Sijtsma, K. (2001). Methodological review: Evaluating person fit. Applied Psychological Measurement, 25, 107–135.CrossRef
go back to reference Motl, R. W., Conroy, D. E., & Horan, P. M. (2000). The social physique anxiety scale: An example of the potential consequence of negatively worded items in factorial validity studies. Journal of Applied Measurement Special Issue: Constructing variables, 1, 327–345. Motl, R. W., Conroy, D. E., & Horan, P. M. (2000). The social physique anxiety scale: An example of the potential consequence of negatively worded items in factorial validity studies. Journal of Applied Measurement Special Issue: Constructing variables, 1, 327–345.
go back to reference Muthén, B. O. (1998–2004). Mplus Technical Appendices. Los Angeles, CA: Muthén & Muthén. Downloaded from www.statmodel.com on June, 2005. Muthén, B. O. (1998–2004). Mplus Technical Appendices. Los Angeles, CA: Muthén & Muthén. Downloaded from www.statmodel.com on June, 2005.
go back to reference Muthén, L. K., & Muthén, B. O. (1998–2004). Mplus user's guide, 3rd ed. Los Angeles, CA: Muthén & Muthén. Muthén, L. K., & Muthén, B. O. (1998–2004). Mplus user's guide, 3rd ed. Los Angeles, CA: Muthén & Muthén.
go back to reference Pilotte, W. J., & Gable, R. K. (1990). The impact of positive and negative item stems on the validity of a computer anxiety scale. Educational and Psychological Measurement, 50, 603–610.CrossRef Pilotte, W. J., & Gable, R. K. (1990). The impact of positive and negative item stems on the validity of a computer anxiety scale. Educational and Psychological Measurement, 50, 603–610.CrossRef
go back to reference Reise, S. P. (1995). Scoring method and detection of person misfit in a personality assessment context. Applied Psychological Measurement, 19, 213–229.CrossRef Reise, S. P. (1995). Scoring method and detection of person misfit in a personality assessment context. Applied Psychological Measurement, 19, 213–229.CrossRef
go back to reference Reise, S. P., & Flannery, P. W. (1996). Assessing person-fit on measures of typical performance. Applied Measurement in Education, 9, 9–26.CrossRef Reise, S. P., & Flannery, P. W. (1996). Assessing person-fit on measures of typical performance. Applied Measurement in Education, 9, 9–26.CrossRef
go back to reference Rodebaugh, T. L., Woods, C. M., Thissen, D. M., Heimberg, R. G., Chambless, D. L., & Rapee, R. M. (2004). More information from fewer questions: The factor structure and item properties of the original and brief Fear of Negative Evaluation Scale. Psychological Assessment, 16, 169–181. (Contributions of the first two authors are equal.)PubMedCrossRef Rodebaugh, T. L., Woods, C. M., Thissen, D. M., Heimberg, R. G., Chambless, D. L., & Rapee, R. M. (2004). More information from fewer questions: The factor structure and item properties of the original and brief Fear of Negative Evaluation Scale. Psychological Assessment, 16, 169–181. (Contributions of the first two authors are equal.)PubMedCrossRef
go back to reference Rodebaugh, T. L., Woods, C. M., Heimberg, R. G., Liebowitz, M. R., & Schneier, F. R. (in press). The Factor Structure, Item Properties, and Screening Utility of the Social Interaction Anxiety Scale. Psychological Assessment. Rodebaugh, T. L., Woods, C. M., Heimberg, R. G., Liebowitz, M. R., & Schneier, F. R. (in press). The Factor Structure, Item Properties, and Screening Utility of the Social Interaction Anxiety Scale. Psychological Assessment.
go back to reference Schmitt, N., & Stults, D. M. (1985). Factors defined by negatively keyed items: The result of careless respondents? Applied Psychological Measurement, 9, 367–373.CrossRef Schmitt, N., & Stults, D. M. (1985). Factors defined by negatively keyed items: The result of careless respondents? Applied Psychological Measurement, 9, 367–373.CrossRef
go back to reference Schriesheim, C. A., & Eisenbach, R. J. (1995). An exploratory and confirmatory factor- analytic investigation of item wording effects on the obtained factor structures of survey questionnaire measures. Journal of Management, 21, 1177–1193.CrossRef Schriesheim, C. A., & Eisenbach, R. J. (1995). An exploratory and confirmatory factor- analytic investigation of item wording effects on the obtained factor structures of survey questionnaire measures. Journal of Management, 21, 1177–1193.CrossRef
go back to reference Schriesheim, C. A., Eisenbach, R. J., & Hill, K. D. (1991). The effect of negation and polar opposite item reversals on questionnaire reliability and validity: An experimental investigation. Educational and Psychological Measurement, 51, 67–78.CrossRef Schriesheim, C. A., Eisenbach, R. J., & Hill, K. D. (1991). The effect of negation and polar opposite item reversals on questionnaire reliability and validity: An experimental investigation. Educational and Psychological Measurement, 51, 67–78.CrossRef
go back to reference Schriesheim, C. A., & Hill, K. D. (1981). Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psychological Measurement, 41, 1101–1114.CrossRef Schriesheim, C. A., & Hill, K. D. (1981). Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psychological Measurement, 41, 1101–1114.CrossRef
go back to reference Spector, P. E., Van Katwyk, P. T., Brannick, M. T., & Chen, P. Y. (1997). When two factors don't reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23, 659–677.CrossRef Spector, P. E., Van Katwyk, P. T., Brannick, M. T., & Chen, P. Y. (1997). When two factors don't reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23, 659–677.CrossRef
go back to reference Steiger, J. H., & Lind, J. M. (1980). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City, IA. Steiger, J. H., & Lind, J. M. (1980). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City, IA.
go back to reference Tomas, J. M., & Oliver, A. (1999). Rosenberg's self-esteem scale: Two factors or method effects. Structural Equation Modeling, 6, 84–98.CrossRef Tomas, J. M., & Oliver, A. (1999). Rosenberg's self-esteem scale: Two factors or method effects. Structural Equation Modeling, 6, 84–98.CrossRef
go back to reference Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10.CrossRef Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10.CrossRef
go back to reference Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 33, 448–457.PubMedCrossRef Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 33, 448–457.PubMedCrossRef
go back to reference Woods, C. M., & Rodebaugh, T. L. (2005). Factor Structures of the Original (FNE) and Brief (BFNE) Fear of Negative Evaluation Scales: Correction to an Erroneous Footnote. Psychological Assessment, 17, 385–386. Woods, C. M., & Rodebaugh, T. L. (2005). Factor Structures of the Original (FNE) and Brief (BFNE) Fear of Negative Evaluation Scales: Correction to an Erroneous Footnote. Psychological Assessment, 17, 385–386.
Metagegevens
Titel
Careless Responding to Reverse-Worded Items: Implications for Confirmatory Factor Analysis
Publicatiedatum
01-09-2006
Gepubliceerd in
Journal of Psychopathology and Behavioral Assessment / Uitgave 3/2006
Print ISSN: 0882-2689
Elektronisch ISSN: 1573-3505
DOI
https://doi.org/10.1007/s10862-005-9004-7

Andere artikelen Uitgave 3/2006

Journal of Psychopathology and Behavioral Assessment 3/2006 Naar de uitgave