Skip to main content

Advertisement

Log in

Piloting a Short Form of the Academic Competence Evaluation Scales

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

A growing body of research indicates that noncognitive factors are important predictors of students’ academic and life success (e.g., Garcia, The need to address noncognitive skills in the education policy agenda (Briefing Paper No. 386), http://files.eric.ed.gov/fulltext/ED558126.pdf, 2014). Despite this evidence base, there are few psychometrically sound measures of such factors appropriate for use in research and practice. One currently available measure is the Academic Competence Evaluation Scales (ACES; DiPerna and Elliott, Academic Competence Evaluation Scales, The Psychological Corporation, San Antonio, TX, 2000) which assesses the skills, attitudes, and behaviors of students that contribute to school success. The length of the ACES (73 items) may limit its use at the primary and secondary levels within a multi-tiered service delivery system or for large-scale educational research. To address this need, the current study piloted a short form of the ACES (ASF) with a sample of 301 elementary students. Results provided initial evidence for the reliability and validity of scores from the ASF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. We share the various concerns raised elsewhere (e.g., Farrington et al., 2012) regarding the appropriateness of using the term “noncognitive” when referring to these variables; however we decided to retain noncognitive here because it is the most commonly used term for these variables.

  2. After the deletion of one outlier identified through calculation of Mahalanobis distances with the total sample.

  3. After the deletion of three outliers detected by Mahalanobis distances calculated with this subsample.

References

  • Anthony, C. J., & DiPerna, J. C. (2017). Identifying sets of maximally efficient items from the Academic Competence Evaluation Scales—Teacher Form. School Psychology Quarterly, 32(4), 552.

    Article  PubMed  Google Scholar 

  • Borghans, L., Duckworth, A. L., Heckman, J. J., & Ter Weel, B. (2008). The economics and psychology of personality traits. Journal of Human Resources, 43(4), 972–1059.

    Article  Google Scholar 

  • Brady, C. E., Evans, S. W., Berlin, K., Bunford, N., & Kerns, L. (2012). Evaluating school impainnent with adolescents using the classroom performance survey. School Psychology Review, 41, 429–446.

    Google Scholar 

  • Cleary, T. J., Gubi, A., & Prescott, M. V. (2010). Motivation and self-regulation assessments: Professional practices and needs of school psychologists. Psychology in the Schools, 47, 985–1002. https://doi.org/10.1002/pits.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New Jersey: Lawrence Erlbaum.

    Google Scholar 

  • Credé, M., Harms, P., Niehorster, S., & Gaye-Valentine, A. (2012). An evaluation of the consequences of using short measures of the Big Five personality traits. Journal of Personality and Social Psychology, 102(4), 874–888. https://doi.org/10.1037/a0027403.

    Article  PubMed  Google Scholar 

  • Credé, M., Tynan, M. C., & Harms, P. D. (2017). Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology, 113(3), 492–511. https://doi.org/10.1037/pspp0000102.

    Article  PubMed  Google Scholar 

  • Cronbach, L. J., & Furby, L. (1970). How we should measure” change”: Or should we? Psychological Bulletin, 74(1), 68.

    Article  Google Scholar 

  • Demaray, M. K., & Jenkins, L. N. (2011). Relations among academic enablers and academic achievement in children with and without high levels of parent-rated symptoms of inattention, impulsivity, and hyperactivity. Psychology in the Schools, 48, 573–586. https://doi.org/10.1002/pits.20578.

    Article  Google Scholar 

  • DiPerna, J. C., & Elliott, S. N. (2000). Academic Competence Evaluation Scales. San Antonio, TX: The Psychological Corporation.

    Google Scholar 

  • Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251. https://doi.org/10.3102/0013189X15584327.

    Article  PubMed  Google Scholar 

  • Elliott, S. N., & Gresham, F. M. (2007). Social Skills Improvement System: Classwide Intervention Program guide. Bloomington, MN: Pearson Assessments.

    Google Scholar 

  • Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., et al. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research.

    Google Scholar 

  • Field, A. (2009). Discovering statistics using SPSS. Sage publications.

  • Garcia, E. (2014). The need to address noncognitive skills in the education policy agenda (Briefing Paper No. 386). Retrieved November 14, 2017 from Economic Policy Institute. http://files.eric.ed.gov/fulltext/ED558126.pdf.

  • Gresham, F. M., & Elliott, S. N. (1990). Social Skills Rating System (SSRS). Circle Pines, MN: American Guidance Service.

    Google Scholar 

  • Gresham, F. M., & Elliott, S. N. (2008). Social Skills Improvement System-Rating Scales. Minneapolis, MN: Pearson Assessments.

    Google Scholar 

  • Gresham, F. M., Elliott, S. N., Cook, C. R., Vance, M. J., & Kettler, R. (2010). Cross-Informant agreement for ratings for social skill and problem behavior ratings: An investigation of the Social Skills Improvement System-Rating Scales. Psychological Assessment, 22, 157–166. https://doi.org/10.1037/a0018124.

    Article  PubMed  Google Scholar 

  • Gresham, F. M., Elliott, S. N., Vance, M. J., & Cook, C. R. (2011). Comparability of Social Skills Rating System to the Social Skills Improvement System: Content and psychometric comparisons across elementary and secondary age levels. School Psychology Quarterly, 26, 27–44. https://doi.org/10.1037/a0022662.

    Article  Google Scholar 

  • Hambleton, R. K. (2010). Review of the Academic Competence Evaluation Scales. In R. A. Spies, J. F. Carlson, & K. F. Geisinger (Eds.), The eighteenth mental measurements yearbook (pp. 1–4). Lincoln, NE: Burns Institute of Mental Measurements.

    Google Scholar 

  • Heckman, J. J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411–482.

    Article  Google Scholar 

  • Hobart, J. C., Cano, S. J., Zajicek, J. P., & Thompson, A. J. (2007). Rating scales as outcome measures for clinical trials in neurology: Problems, solutions, and recommendations. The Lancet Neurology, 6(12), 1094–1105.

    Article  PubMed  Google Scholar 

  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118.

    Article  Google Scholar 

  • Kautz, T., Heckman, J. J., Diris, R., Ter Weel, B., & Borghans, L. (2014). Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success (No. w20749). National Bureau of Economic Research.

  • Kline, R. B. (2011). Principles and practices of structural equation modeling. New York, NY: Guilford.

    Google Scholar 

  • Malecki, C. K., & Elliot, S. N. (2002). Children’s social behaviors as predictors of academic achievement: A longitudinal analysis. School Psychology Quarterly, 17, 1–23.

    Article  Google Scholar 

  • McCormick, M. P., O’Connor, E. E., Cappella, E., & McClowry, S. G. (2013). Teacher–child relationships and academic achievement: A multilevel propensity score model approach. Journal of School Psychology, 51, 611–624. https://doi.org/10.1016/j.jsp.2013.05.001.

    Article  PubMed  Google Scholar 

  • McDermott, P. A., Green, L. F., Francis, J. M., & Stott, D. H. (1999). Learning Behaviors Scale. Philadelphia: Edumetric and Clinical Science. https://doi.org/10.1521/scpq.17.1.1.19902.

    Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide (7th Ed.). Los Angeles, CA: Muthén & Muthén.

  • Pearson. (2012). AIMSweb technical manual. Bloomington, MN: Pearson. Retrieved from www.aimsweb.com/wp-content/uploads/aimsweb-Technical-Manual.pdf.

  • Renaissance Learning. (2012). STAR Math technical manual. Wisconsin Rapids, WI: Renaissance Learning.

    Google Scholar 

  • Renaissance Learning. (2015). STAR Reading technical manual. Wisconsin Rapids, WI: Renaissance Learning.

    Google Scholar 

  • Reynolds, C. R., & Kamphaus, R. W. (2004). Behavior assessment system for children (2nd ed.). Circle Pines, MN: American Guidance Service.

    Google Scholar 

  • Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315.

    Article  PubMed  Google Scholar 

  • Rosen, J. A., Glennie, E. J., Dalton, B. W., Lennon, J. M., & Bozick, R. N. (2010). Noncognitive skills in the classroom: New perspectives on educational research. Research Triangle Park, NC: RTI International.

    Book  Google Scholar 

  • Sabers, D. L., & Bonner, S. (2010). Review of the Academic Competence Evaluation Scales. In R. A. Spies, J. F. Carlson, & K. F. Geisinger (Eds.), The eighteenth mental measurements yearbook (pp. 4–6). Lincoln, NE: Burns Institute of Mental Measurements.

    Google Scholar 

  • Salvia, J., Ysselydke, J. E., & Bolt, S. (2010). Assessment in special and inclusive education (11th ed.). Boston: Houghton Mifflin.

    Google Scholar 

  • Schrank, F. A., McGrew, K. S., & Mather, N. (2014). Woodcock–Johnson IV tests of achievement. Rolling Meadows, IL: Riverside.

    Google Scholar 

  • Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological Assessment, 12(1), 102–111. https://doi.org/10.1037/1040-3590.12.1.102.

    Article  PubMed  Google Scholar 

  • U.S. Department of Education Office for Civil Rights. (2016). Civil rights data collection: A first look. Retrieved December 12, 2017 from https://www2.ed.gov/about/offices/list/ocr/docs/2013-14-first-look.pdf.

  • Volpe, R. J., DuPaul, G. J., DiPerna, J. C., Jitendra, A. K., Lutz, G. L., Tresco, K., et al. (2006). Attention deficity hyperactivity disorder and scholastic achievement: A model of mediation via academic enablers. School Psychology Review, 35, 47–61.

    Google Scholar 

  • Wechsler, D. (2002). Wechsler individual achievement test (2nd ed.). San Antonio, TX: Psychological Corp.

    Google Scholar 

  • West, M. R., Kraft, M. A., Finn, A. S., Martin, R. E., Duckworth, A. L., Gabrieli, C. F., et al. (2016). Promise and Paradox-measuring students’ non-cognitive skills and the impact of schooling. Educational Evaluation and Policy Analysis, 38, 148–170. https://doi.org/10.3102/0162373715597298.

    Article  Google Scholar 

  • Zaslow, M., Halle, T., Martin, L., Cabrera, N., Calkins, J., Pitzer, L., et al. (2006). Child outcome measures in the study of child care quality. Evaluation Review, 30, 577–610. https://doi.org/10.1177/0193841X06291529.

    Article  PubMed  Google Scholar 

  • Zeehandelaar, D., & Winkler, A. M. (2013). What parents want: Education preferences and trade-offs. Washington, DC: Thomas B. Fordham Institute.

    Google Scholar 

Download references

Funding

This study was funded by the Institute of Education Sciences (Grant Numbers R305A090438 and R305B090007).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Christopher J. Anthony.

Ethics declarations

Conflict of interest

James DiPerna is the lead author of the Academic Competence Evaluation Scales.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent and assent was obtained for all participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anthony, C.J., DiPerna, J.C. Piloting a Short Form of the Academic Competence Evaluation Scales. School Mental Health 10, 314–321 (2018). https://doi.org/10.1007/s12310-018-9254-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-018-9254-7

Keywords

Navigation