Developing and Validating Clinical Questionnaires

Literature Cited

1. Am. Psychiatr. Assoc 2013. Diagnostic and Statistical Manual of Mental Disorders Arlington, VA: Am. Psychiatr. Publ. , 5th ed..
2. Anastasi A, Urbina S. 1997. Psychological Testing Upper Saddle River, NJ: Prentice Hall/Pearson Educ. , 7th ed..
3. Asparouhov T, Muthén B. 2010. Computing the strictly positive Satorra–Bentler chi-square test in Mplus Mplus Web Note 12. https://www.statmodel.com/examples/webnotes/webnote12.pdf
   [Google Scholar]
4. Ben-Porath YS, Tellegen A. 2008. Manual for Administration, Scoring, and Interpretation Minneapolis: Univ. Minn. Press
5. Bentler PM. 2009. Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika 74:137–43
   [Google Scholar]
6. Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. 2015. The Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. J. Trauma Stress 28:489–98
   [Google Scholar]
7. Bonifay W, Lane SP, Reise SP. 2016. Three concerns with applying a bifactor model as a structure of psychopathology. Clin. Psychol. Sci. 5:184–86
   [Google Scholar]
8. Bornovalova MA, Choate AM, Fatimah H, Petersen KJ, Wiernik BM. 2020. Appropriate use of bifactor analysis in psychopathology research: appreciating benefits and limitations. Biol. Psychiatry 88:18–27
   [Google Scholar]
9. Brown TA. 2003. Confirmatory factor analysis of the Penn State Worry Questionnaire: multiple factors or method effects?. Behav. Res. Ther. 41:1411–26
   [Google Scholar]
10. Brown TA. 2015. Confirmatory Factor Analysis for Applied Research New York: Guilford. , 2nd ed..
11. Brown TA, Barlow DH. 2009. A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: implications for assessment and treatment. Psychol. Assess. 21:256–71
    [Google Scholar]
12. Brown TA, Chorpita BF, Korotitsch W, Barlow DH. 1997. Psychometric properties of the Depression Anxiety Stress Scales (DASS) in clinical samples. Behav. Res. Ther. 35:79–89
    [Google Scholar]
13. Brown TA, White KS, Barlow DH. 2005. A psychometric reanalysis of the Albany Panic and Phobia Questionnaire. Behav. Res. Ther. 43:337–55
    [Google Scholar]
14. Campbell DT, Fiske DW. 1959. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol. Bull. 56:81–105
    [Google Scholar]
15. Campbell-Sills L, Liverant GI, Brown TA. 2004. Psychometric evaluation of the behavioral inhibition/behavioral activation scales in a large sample of outpatients with anxiety and mood disorders. Psychol. Assess. 16:244–54
    [Google Scholar]
16. Casler K, Bickel L, Hackett E. 2013. Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. Comput. Hum. Behav. 29:2156–60
    [Google Scholar]
17. Chmielewski M, Kucker SC. 2020. An MTurk crisis? Shifts in data quality and the impact on study results. Soc. Psychol. Personal. Sci. 11:464–73
    [Google Scholar]
18. Clark LA, Watson D. 1995. Constructing validity: basic issues in objective scale development. Psychol. Assess. 7:309–19
    [Google Scholar]
19. Clark LA, Watson D. 2019. Constructing validity: new developments in creating objective measuring instruments. Psychol. Assess. 31:1412–27
    [Google Scholar]
20. Cole DA, Ciesla JA, Steiger JH. 2007. The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychol. Methods 12:381–98
    [Google Scholar]
21. Comrey AL. 1988. Factor-analytic methods of scale development in personality and clinical psychology. J. Consult. Clin. Psychol. 56:754–61
    [Google Scholar]
22. Conti G, Frühwirth-Schnatter S, Heckman JJ, Piatek R. 2014. Bayesian exploratory factor analysis. J. Econom. 183:31–57
    [Google Scholar]
23. Cox A, Pant H, Gilson AN, Rodriguez JL, Young KR et al. 2012. Effects of augmenting response options on MMPI–2 RC scale psychometrics. J. Personal. Assess. 94:613–19
    [Google Scholar]
24. Cronbach LJ, Meehl PE. 1955. Construct validity in psychological tests. Psychol. Bull. 52:281–302
    [Google Scholar]
25. Curran PJ, West SG, Finch JF. 1996. The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol. Methods 1:16–29
    [Google Scholar]
26. Deacon B, Olatunji BO. 2007. Specificity of disgust sensitivity in the prediction of behavioral avoidance in contamination fear. Behav. Res. Ther. 45:2110–20
    [Google Scholar]
27. DeVellis RF. 2016. Scale Development: Theory and Applications Los Angeles: Sage
28. Embretson SE, Reise SP. 2013. Item Response Theory Mahwah, NJ: Psychol. Press
29. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. 1999. Evaluating the use of exploratory factor analysis in psychological research. Psychol. Methods 4:272–99
    [Google Scholar]
30. Finn JA, Ben-Porath YS, Tellegen A. 2015. Dichotomous versus polytomous response options in psychopathology assessment: method or meaningful variance?. Psychol. Assess. 27:184–93
    [Google Scholar]
31. Fossati A, Maffei C, Bagnato M, Donati D, Donini M et al. 1998. Brief communication: criterion validity of the Personality Diagnostic Questionnaire-4+ (PDQ-4+) in a mixed psychiatric sample. J. Personal. Disord. 12:172–78
    [Google Scholar]
32. Furnham A, Guenole N, Levine SZ, Chamorro-Premuzic T. 2013. The NEO Personality Inventory–Revised: factor structure and gender invariance from exploratory structural equation modeling analyses in a high-stakes setting. Assessment 20:14–23
    [Google Scholar]
33. Gallagher MW, Lopez SJ, Preacher KJ. 2009. The hierarchical structure of well-being. J. Personal. 77:1025–50
    [Google Scholar]
34. Gámez W, Chmielewski M, Kotov R, Ruggero C, Watson D. 2011. Development of a measure of experiential avoidance: the Multidimensional Experiential Avoidance Questionnaire. Psychol. Assess. 23:692–713
    [Google Scholar]
35. Garnier-Villarreal M, Jorgensen TD. 2020. Adapting fit indices for Bayesian structural equation modeling: comparison to maximum likelihood. Psychol. Methods 25:46–70
    [Google Scholar]
36. Gomez R. 2016. Factor structure of the Social Interaction Anxiety Scale and the Social Phobia Scale Short Forms. Personal. Individ. Differ. 96:83–87
    [Google Scholar]
37. Guadagnoli E, Velicer WF. 1988. Relation of sample size to the stability of component patterns. Psychol. Bull. 103:265–75
    [Google Scholar]
38. Guo J, Marsh HW, Parker PD, Dicke T, Lüdtke O, Diallo TMO. 2019. A systematic evaluation and comparison between exploratory structural equation modeling and Bayesian structural equation modeling. Struct. Equ. Model. 26:529–56
    [Google Scholar]
39. Haslam N, McGrath MJ, Viechtbauer W, Kuppens P. 2020. Dimensions over categories: a meta-analysis of taxometric research. Psychol. Med. 50:1418–32
    [Google Scholar]
40. Hatcher RL, Lindqvist K, Falkenström F. 2020. Psychometric evaluation of the Working Alliance Inventory—Therapist version: current and new short forms. Psychother. Res. 30:706–17
    [Google Scholar]
41. Hauser DJ, Schwarz N. 2016. Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behav. Res. Methods 48:400–7
    [Google Scholar]
42. Haynes SN, Richard D, Kubany ES. 1995. Content validity in psychological assessment: a functional approach to concepts and methods. Psychol. Assess. 7:238–47
    [Google Scholar]
43. Hayton JC, Allen DG, Scarpello V. 2004. Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organ. Res. Methods 7:191–205
    [Google Scholar]
44. Hoch A, Amsden GS. 1913. A guide to the descriptive study of personality. Rev. Neurol. Psychiatry 11:577–87
    [Google Scholar]
45. Hofmann SG, DiBartolo PM. 2000. An instrument to assess self-statements during public speaking: scale development and preliminary psychometric properties. Behav. Ther. 31:499–515
    [Google Scholar]
46. Hogan R. 1983. A socioanalytic theory of personality. Nebraska Symposium on Motivation, 1982: Personality—Current Theory and Research, M Page 55–89 Lincoln: Univ. Neb. Press
    [Google Scholar]
47. Hu LT, Bentler PM. 1999. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. 6:1–55
    [Google Scholar]
48. Hu LT, Bentler PM, Kano Y. 1992. Can test statistics in covariance structure analysis be trusted?. Psychol. Bull. 112:351–62
    [Google Scholar]
49. Hunsley J, Meyer GJ. 2003. The incremental validity of psychological testing and assessment: conceptual, methodological, and statistical issues. Psychol. Assess. 15:446–55
    [Google Scholar]
50. Hyler S. 1994. The Personality Diagnostic Questionnaire-4 (PDQ-4) New York: N.Y. State Psychiatr. Inst.
51. Insel TR. 2014. The NIMH Research Domain Criteria (RDoC) project: precision medicine for psychiatry. Am. J. Psychiatry 171:395–97
    [Google Scholar]
52. Ioannidis CA, Siegling AB. 2015. Criterion and incremental validity of the emotion regulation questionnaire. Front. Psychol. 6:247
    [Google Scholar]
53. Jöreskog KG. 2001. LISREL 8.50. Statistical Software https://ssicentral.com/index.php/products/lisrel/
    [Google Scholar]
54. Kamata A, Bauer DJ. 2008. A note on the relation between factor analytic and item response theory models. Struct. Equ. Model. 15:136–53
    [Google Scholar]
55. Kaplan D, Depaoli S 2012. Bayesian structural equation modeling. Handbook of Structural Equation Modeling R Hoyle 650–73 New York: Guilford
    [Google Scholar]
56. Khazanov GK, Ruscio AM, Forbes CN. 2020. The Positive Valence Systems Scale: development and validation. Assessment 27:1045–69
    [Google Scholar]
57. Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR et al. 2017. The Hierarchical Taxonomy of Psychopathology (HiTOP): a dimensional alternative to traditional nosologies. J. Abnorm. Psychol. 126:454–77
    [Google Scholar]
58. Kramer MD, Patrick CJ, Hettema JM, Moore AA, Sawyers CK, Yancey JR. 2020. Quantifying dispositional fear as threat sensitivity: development and initial validation of a model-based scale measure. Assessment 27:533–46
    [Google Scholar]
59. Kroenke K, Spitzer RL, Williams JB. 2001. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16:606–13
    [Google Scholar]
60. Krueger RF, Derringer J, Markon KE, Watson D, Skodol AE. 2012. Initial construction of a maladaptive personality trait model and inventory for DSM-5. Psychol. Med. 42:1879–90
    [Google Scholar]
61. Li CH. 2016. Confirmatory factor analysis with ordinal data: comparing robust maximum likelihood and diagonally weighted least squares. Behav. Res. Methods 48:936–49
    [Google Scholar]
62. Lim S, Jahng S. 2019. Determining the number of factors using parallel analysis and its recent variants. Psychol. Methods 24:452–67
    [Google Scholar]
63. Loevinger J. 1957. Objective tests as instruments of psychological theory. Psychol. Rep. 3:635–94
    [Google Scholar]
64. Lubbe D. 2019. Parallel analysis with categorical variables: impact of category probability proportions on dimensionality assessment accuracy. Psychol. Methods 24:339–51
    [Google Scholar]
65. MacCallum RC, Roznowski M, Necowitz LB. 1992. Model modifications in covariance structure analysis: the problem of capitalization on chance. Psychol. Bull. 111:490–504
    [Google Scholar]
66. MacCallum RC, Widaman KF, Zhang S, Hong S. 1999. Sample size in factor analysis. Psychol. Methods 4:84–99
    [Google Scholar]
67. MacInnis CC, Boss HC, Bourdage JS. 2020. More evidence of participant misrepresentation on MTurk and investigating who misrepresents. Personal. Individ. Differ. 152:109603
    [Google Scholar]
68. Markon KE. 2019. Bifactor and hierarchical models: specification, inference, and interpretation. Annu. Rev. Clin. Psychol. 15:51–69
    [Google Scholar]
69. Marsh HW, Lüdtke O, Muthén B, Asparouhov T, Morin AJ et al. 2010. A new look at the Big Five factor structure through exploratory structural equation modeling. Psychol. Assess. 22:471–91
    [Google Scholar]
70. Marsh HW, Morin AJ, Parker PD, Kaur G. 2014. Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annu. Rev. Clin. Psychol. 10:85–110
    [Google Scholar]
71. Marsh HW, Nagengast B, Morin AJS. 2013. Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and la dolce vita effects. Dev. Psychol. 49:1194–218
    [Google Scholar]
72. Mattick RP, Clarke JC. 1998. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav. Res. Ther. 36:455–70
    [Google Scholar]
73. McCrae RR, Zonderman AB, Costa PT Jr., Bond MH, Paunonen SV. 1996. Evaluating replicability of factors in the Revised NEO Personality Inventory: confirmatory factor analysis versus Procrustes rotation. J. Personal. Soc. Psychol. 70:552–66
    [Google Scholar]
74. Meade AW, Craig SB. 2012. Identifying careless responses in survey data. Psychol. Methods 17:437–55
    [Google Scholar]
75. Meng XL, Rosenthal R, Rubin D. 1992. Comparing correlated correlation coefficients. Psychol. Bull. 111:172–75
    [Google Scholar]
76. Mikolajczak M, Luminet O, Leroy C, Roy E. 2007. Psychometric properties of the Trait Emotional Intelligence Questionnaire: factor structure, reliability, construct, and incremental validity in a French-speaking population. J. Personal. Assess. 88:338–53
    [Google Scholar]
77. Millsap RE. 2012. Statistical Approaches to Measurement Invariance New York: Routledge
78. Morey LC. 2014. The Personality Assessment Inventory Lutz, FL: Psychol. Assess. Res.
79. Muthén B, Asparouhov T. 2012. Bayesian structural equation modeling: a more flexible representation of substantive theory. Psychol. Methods 17:313–35
    [Google Scholar]
80. Muthén LK, Muthén BO. 2002. How to use a Monte Carlo study to decide on sample size and determine power. Struct. Equ. Model. 9:599–620
    [Google Scholar]
81. Muthén LK, Muthén BO. 2019. Mplus 8.4. Statistical Software http://www.statmodel.com
    [Google Scholar]
82. Peters L, Sunderland M, Andrews G, Rapee RM, Mattick RP. 2012. Development of a short form Social Interaction Anxiety (SIAS) and Social Phobia Scale (SPS) using nonparametric item response theory: the SIAS-6 and the SPS-6. Psychol. Assess. 24:66–76
    [Google Scholar]
83. Preston CC, Colman AM. 2000. Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychol. 104:1–15
    [Google Scholar]
84. Reise SP. 2012. The rediscovery of bifactor measurement models. Multivar. Behav. Res. 47:667–96
    [Google Scholar]
85. Revelle W, Condon DM. 2019. Reliability from α to ω: a tutorial. Psychol. Assess. 31:1395–411
    [Google Scholar]
86. Rhemtulla M, Brosseau-Liard P, Savalei V. 2012. When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychol. Methods 17:354–73
    [Google Scholar]
87. Rodriguez A, Reise SP, Haviland MG. 2016. Applying bifactor statistical indices in the evaluation of psychological measures. J. Personal. Assess. 98:223–37
    [Google Scholar]
88. Rogers TA, Bardeen JR, Fergus TA, Benfer N. 2020. Factor structure and incremental utility of the Distress Tolerance Scale: a bifactor analysis. Assessment 27:297–308
    [Google Scholar]
89. Rosellini AJ, Brown TA. 2011. The NEO Five-Factor Inventory: latent structure and relationships with dimensions of anxiety and depressive disorders in a large clinical sample. Assessment 18:27–38
    [Google Scholar]
90. Rosellini AJ, Brown TA. 2019. The Multidimensional Emotional Disorder Inventory (MEDI): assessing transdiagnostic dimensions to validate a profile approach to emotional disorder classification. Psychol. Assess. 31:59–72
    [Google Scholar]
91. Rouquette A, Falissard B. 2011. Sample size requirements for the internal validation of psychiatric scales. Int. J. Methods Psychiatr. Res. 20:235–49
    [Google Scholar]
92. Saris WE, Satorra A, van der Veld WM. 2009. Testing structural equation models or detection of misspecifications?. Struct. Equ. Model. 16:561–82
    [Google Scholar]
93. Schmitt TA, Sass DA, Chappelle W, Thompson W. 2018. Selecting the “best” factor structure and moving measurement validation forward: an illustration. J. Personal. Assess. 100:345–62
    [Google Scholar]
94. Sellbom M. 2011. Elaborating on the construct validity of the Levenson Self-Report Psychopathy Scale in incarcerated and non-incarcerated samples. Law Hum. Behav. 35:440–51
    [Google Scholar]
95. Sellbom M. 2016. Elucidating the validity of the externalizing spectrum of psychopathology in correctional, forensic, and community samples. J. Abnorm. Psychol. 125:1027–38
    [Google Scholar]
96. Sellbom M, Tellegen A. 2019. Factor analysis in psychological assessment research: common pitfalls and recommendations. Psychol. Assess. 31:1428–41
    [Google Scholar]
97. Sijtsma K. 2009. On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika 74:107–20
    [Google Scholar]
98. Simms LJ, Zelazny K, Williams TF, Bernstein L. 2019. Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychol. Assess. 31:557–66
    [Google Scholar]
99. Snyder HR, Young JF, Hankin BL. 2017. Strong homotypic continuity in common psychopathology-, internalizing-, and externalizing-specific factors over time in adolescents. Clin. Psychol. Sci. 5:98–110
    [Google Scholar]
100. Tabachnick BG, Fidell LS, Ullman JB. 2013. Using Multivariate Statistics Boston: Pearson
101. Terluin B, van Marwijk HWJ, Adèr HJ, de Vet HCW, Penninx BWJH et al. 2006. The Four-Dimensional Symptom Questionnaire (4DSQ): a validation study of a multidimensional self-report questionnaire to assess distress, depression, anxiety and somatization. BMC Psychiatry 6:34
     [Google Scholar]
102. Thomas ML. 2019. Advances in applications of item response theory to clinical assessment. Psychol. Assess. 31:1442–55
     [Google Scholar]
103. Thurstone LL. 1947. Multiple Factor Analysis Chicago: Univ. Chicago Press
104. Tóth-Király I, Bőthe B, Orosz G. 2017. Exploratory structural equation modeling analysis of the Self-Compassion Scale. Mindfulness 8:881–92
     [Google Scholar]
105. van Erp S, Mulder J, Oberski DL. 2018. Prior sensitivity analysis in default Bayesian structural equation modeling. Psychol. Methods 23:363–88
     [Google Scholar]
106. Watson D, Friend R. 1969. Measurement of social-evaluative anxiety. J. Consult. Clin. Psychol. 33:448–57
     [Google Scholar]
107. Watson D, O'Hara MW, Naragon-Gainey K, Koffel E, Chmielewski M et al. 2012. Development and validation of new anxiety and bipolar symptom scales for an expanded version of the IDAS (the IDAS-II). Assessment 19:399–420
     [Google Scholar]
108. Watson D, O'Hara MW, Simms LJ, Kotov R, Chmielewski M et al. 2007. Development and validation of the Inventory of Depression and Anxiety Symptoms (IDAS). Psychol. Assess. 19:253–68
     [Google Scholar]
109. Waugh MH, Hopwood CJ, Krueger RF, Morey LC, Pincus AL, Wright AG. 2017. Psychological assessment with the DSM-5 Alternative Model for Personality Disorders: tradition and innovation. Prof. Psychol. Res. Pract. 48:79–89
     [Google Scholar]
110. Weijters B, Cabooter E, Schillewaert N. 2010. The effect of rating scale format on response styles: the number of response categories and response category labels. Int. J. Res. Mark. 27:236–47
     [Google Scholar]
111. Woodworth RS. 1920. Personal Data Sheet Chicago, IL: Stoelting
112. Wu H, Leung SO. 2017. Can Likert scales be treated as interval scales?—A simulation study. J. Soc. Serv. Res. 43:527–32
     [Google Scholar]
113. Wu Y, Zuo B, Wen F, Yan L. 2017. Rosenberg Self-Esteem Scale: method effects, factorial structure and scale invariance across migrant child and urban child populations in China. J. Personal. Assess. 99:83–93
     [Google Scholar]
114. Yang Y, Xia Y. 2015. On the number of factors to retain in exploratory factor analysis for ordered categorical data. Behav. Res. Methods 47:756–72
     [Google Scholar]
115. Zhang X, Noor R, Savalei V. 2016. Examining the effect of reverse worded items on the factor structure of the Need for Cognition scale. PLOS ONE 11:e0157795
     [Google Scholar]
116. Zinbarg RE, Barlow DH, Brown TA. 1997. Hierarchical structure and general factor saturation of the Anxiety Sensitivity Index: evidence and implications. Psychol. Assess. 9:277–84
     [Google Scholar]