Validating Test Score Interpretations by Cross-National Comparison
Comparing the Results of Students From Japan and Germany on an American Test of Economic Knowledge in Higher Education
Abstract
Cross-national assessment of students’ competences in higher education is becoming increasingly important in many disciplines including economics but there are few available instruments that meet psychological standards for assessing students’ economic competence in higher education (HE). One of them is the internationally valid Test of Understanding in College Economics (TUCE), which has been adapted and employed successfully in HE systems in various countries, but the test results have seldom been used for international comparisons of students’ Economic Content Knowledge (ECK). Here, we compare the German and the Japanese test adaptations of the TUCE with reference to the American original in order to determine their suitability for comparative analyses of ECK in HE among these countries. Having critically examined the two test adaptations, we present a comparative analysis of students’ test scores in Germany and Japan and evaluate potential differences with regard to students’ acquisition of ECK while investigating country-specific influence factors.
References
. (2004). Standards for educational and psychological testing (2nd ed.). Washington, DC: American Psychological Association.
(2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY: Longman.
(2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186–203.
(2000). Translating Questionnaires and other research instruments: Problems and solutions. Thousand Oaks, CA: Sage.
(1964). Comparative method in education. New York, NY: Holt, Rinehart, & Winston.
(2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für Psychologie, 223, doi: 10.1027/2151-2604/a000194.
(1956). Taxonomy of Educational Objectives, the classification of educational goals – Handbook I: Cognitive domain. New York, NY: McKay.
(2007). Comparative education research – approaches and methods. Hong Kong, China: Springer.
(2006). Confirmatory factor analysis for applied research. New York, NY: Guilford.
(1998). Alternative ways of assessing model fit. In , Testing structural equation models, vol. 154, (Sage focus editions). (pp. 136–162). Newbury Park, CA: Sage.
(
in press ). Effects of prior economic education, native language, and gender on economic knowledge of first-year students in higher education. A comparative study between Germany and the Unites States. In , Assessment of competence in higher education [Special issue]. Studies in Higher Education.(2010). Uncovering excellence in academic rankings: A closer look at the Shanghai ranking. Scientometrics, 83, 515–524.
(1977). Maximum-likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39, 1–38.
(2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT Likelihood Ratio. Applied Psychological Measurement, 29, 278–295. doi: 10.1177/0146621605275728
(2009). Booklet designs in large-scale assessments of student achievement: Theory and practice. Educational Measurement: Issues and Practice, 28, 39–53.
(2003). Methods for handling missing data. In , Handbook of psychology. Research methods in psychology, Vol. 2, (pp. 87–114). Hoboken, NJ: Wiley.
(2001). The next generation of the ITC test translation and adaption guidelines. European Journal of Psychological Assessment, 17, 164–172. doi: 10.1027/1015-5759.17.3.164
(2005). Adapting educational and psychological tests for cross-cultural assessment (pp. 3–38). Mahwah, NJ: Erlbaum.
(2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73.
(2008). Current issues in competence modeling and assessment. Zeitschrift für Psychologie, 216, 61–73. doi: 10.1027/0044-3409.216.2.61
(2013). An investigation of the nature of the influences of item stem and option representation on student responses to a mathematics test. European Journal of Psychological Education, 28, 1141–1161.
. (2012). Assessment of Higher Education Learning Outcomes. Feasibility Study Report. Volume 1 – Design and Implementation. Retrieved from www.oecd.org/edu/skills-beyond-school/AHELOFSReportVolume1.pdf
(2012). Student characteristics, behavior, and performance in economics classes. In , International handbook on teaching and learning economics (pp. 341–350). Northampton, MA: Edward Elgar.
(2009). An international comparison of university students’ knowledge of economics: Japan and the United States. Bulletin of Yamamura Gakuen College, 20, 13–43.
(2013). Analyzing observed composite differences across groups: Is partial measurement invariance enough? Methodology, 9, 1–12. doi: 10.1027/1614-2241/a000049
(2007). Test of understanding in college economics: Examiner's manual (4th ed.). New York, NY: National Council on Economic Education.
(2001). Competencies and key competencies: Educational perspective. In , International Encyclopedia of the social and behavioral sciences, Vol. 4, (pp. 2433–2436). Amsterdam, The Netherlands: Elsevier.
(2009). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44, 1–27.
(2010). Comparative studies on economic education in Asia-Pacific region. Tokio, Japan: Shumpusha.
(2014). Insights from a German assessment of business and economics competence. In , Higher education learning outcomes assessment – international perspectives (pp. 175–197). Frankfurt am Main, Germany: Lang.
(2013). Modeling and measurement of university students’ subject-specific competencies in the domain of business & economics – The ILLEV project. In , Modeling and measuring competencies in higher education (pp. 159–170). Rotterdam, The Netherlands: Sense.
