Abstract
In medical education, the effect of the educational environment on student achievement has primarily been investigated in comparisons between traditional and problem-based learning (PBL) curricula. As many of these studies have reached no clear conclusions on the superiority of the PBL approach, the effect of curricular reform on student performance remains an issue. We employed a theoretical framework that integrates antecedents of student achievement from various psychosocial domains to examine how students interact with their curricular environment. In a longitudinal study with N = 1,646 participants, we assessed students in a traditional and a PBL-centered curriculum. The measures administered included students’ perception of the learning environment, self-efficacy beliefs, positive study-related affect, social support, indicators of self-regulated learning, and academic achievement assessed through progress tests. We compared the relations between these characteristics in the two curricular environments. The results are two-fold. First, substantial relations of various psychosocial domains and their associations with achievement were identified. Second, our analyses indicated that there are no substantial differences between traditional and PBL-based curricula concerning the relational structure of psychosocial variables and achievement. Drawing definite conclusions on the role of curricular-level interventions in the development of student’s academic achievement is constrained by the quasi-experimental design as wells as the selection of variables included. However, in the specific context described here, our results may still support the view of student activity as the key ingredient in the acquisition of achievement and performance.
Similar content being viewed by others
References
Albanese, M. A., & Mitchell, S. (1993). Problem-based learning: A review of literature on its outcomes and implementation issues. Academic Medicine, 68(1), 52–81.
Artino, A. R., Dong, T., DeZee, K. J., Gilliland, W. R., Waechter, D. M., Cruess, D., & Durning, S. J. (2012a). Achievement goal structures and self-regulated learning: relationships and changes in medical school. Academic Medicine, 87(10), 1375–1381.
Artino, A. R., Holmboe, E. S., & Durning, S. J. (2012b). Control-value theory: Using achievement emotions to improve understanding of motivation, learning, and performance in medical education: AMEE Guide No. 64. Medical Teacher, 34(3), e148–e160.
Artino, A. R., La Rochelle, J. S., & Durning, S. J. (2010). Second-year medical students’ motivational beliefs, emotions, and achievement. Medical Education, 44(12), 1203–1212.
Baeten, M., Kyndt, E., Struyven, K., & Dochy, F. (2010). Using student-centred learning environments to stimulate deep approaches to learning: Factors encouraging or discouraging their effectiveness. Educational Research Review, 5(3), 243–260.
Barrows, H. S. (1996). Problem-based learning in medicine and beyond: A brief overview. New Directions for Teaching and Learning, 1996(68), 3–12.
Bate, E., Hommes, J., Duvivier, R., & Taylor, D. C. M. (2014). Problem-based learning (PBL): Getting the most out of your students—Their roles and responsibilities: AMEE Guide No. 84. Medical Teacher, 36(1), 1–12.
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and S4, Version 1.0-5. http://CRAN.R-project.org/package=lme4. Accessed 22 Jan 2015.
Bergman, E. M., Prince, K. J., Drukker, J., van der Vleuten, C. P. M., & Scherpbier, A. J. (2008). How much anatomy is enough? Anatomical Sciences Education, 1(4), 184–188.
Berkson, L. (1993). Problem-based learning: Have the expectations been met? Academic Medicine, 68(10), S79.
Borenstein, M. (2009). Introduction to meta-analysis. Chichester: Wiley.
Brose, A., Schmiedek, F., Lövdén, M., & Lindenberger, U. (2012). Daily variability in working memory is coupled with negative affect: The role of attention and motivation. Emotion, 12(3), 605.
Burt, D. B., Zembar, M. J., & Niederehe, G. (1995). Depression and memory impairment: a meta-analysis of the association, its pattern, and specificity. Psychological Bulletin, 117(2), 285.
Cohen, S., & Hoberman, H. M. (1983). Positive events and social supports as buffers of life change stress. Journal of Applied Social Psychology, 13(2), 99–125.
Cohen-Schotanus, J., Muijtjens, A. M. M., Schönrock-Adema, J., Geertsma, J., & van der Vleuten, C. P. M. (2008). Effects of conventional and problem-based learning on clinical and general competencies and career development. Medical Education, 42(3), 256–265.
Colliver, J. A. (2000). Effectiveness of problem-based learning curricula: research and theory. Academic Medicine, 75(3), 259–266.
Dettmer, S., Kuhlmey, A., & Schulz, S. (2010). Karriere- und Lebensplanung: Gehen oder bleiben? Dtsch Arztebl International, 107(1-2), A-30.
Diseth, Å. (2007). Approaches to learning, course experience and examination grade among undergraduate psychology students: testing of mediator effects and construct validity. Studies in Higher Education, 32(3), 373–388.
Diseth, Å. (2011). Self-efficacy, goal orientations and learning strategies as mediators between preceding and subsequent academic achievement. Learning and Individual Differences, 21(2), 191–195.
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-based learning: A meta-analysis. Learning and Instruction, 13(5), 533–568.
Dolmans, D. H. J. M., de Grave, W., Wolfhagen, I. H. A. P., & van der Vleuten, C. P. M. (2005). Problem-based learning: future challenges for educational practice and research. Medical Education, 39(7), 732–741.
Dolmans, D., & Gijbels, D. (2013). Research on problem-based learning: Future challenges. Medical Education, 47(2), 214–218.
Drukker, J., Hylkema, N., Prince, K. J. A. H., Scherpbier, A. J. J. A., Van der Vleuten, C. P., & van Mameren, H. (2003). Does problem-based learning lead to deficiencies in basic science knowledge? An empirical case on anatomy. Medical Education, 37(1), 15–21.
Duvivier, R. J., van Dalen, J., Muijtjens, A. M., Moulaert, V. R., van der Vleuten, C. P. M., & Scherpbier, A. J. (2011). The role of deliberate practice in the acquisition of clinical skills. BMC Medical Education, 11(1), 101.
Enders, C. K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8(1), 128–141.
Engel, G. (1977). The need for a new medical model: a challenge for biomedicine. Science, 196(4286), 129–136.
Farrow, R., & Norman, G. (2003). The effectiveness of PBL: the debate continues. Is meta-analysis helpful? Medical Education, 37(12), 1131–1132.
Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Analytical methods for social research. Cambridge, New York: Cambridge University Press.
Gijbels, D., Segers, M., & Struyf, E. (2008). Constructivist learning environments and the (im)possibility to change students’ perceptions of assessment demands and approaches to learning. Instructional Science, 36, 431–443.
Hartling, L., Spooner, C., Tjosvold, L., & Oswald, A. (2010). Problem-based learning in pre-clinical medical education: 22 years of outcome research. Medical Teacher, 32(1), 28–35.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266.
Hofmann, W., Schmeichel, B. J., & Baddeley, A. D. (2012). Executive functions and self-regulation. Trends in Cognitive Sciences, 16(3), 174–180.
Hommes, J., Bossche, P., Grave, W., Bos, G., Schuwirth, L., & Scherpbier, A. (2014). Understanding the effects of time on collaborative learning processes in problem based learning: a mixed methods study. Advances in Health Sciences Education, 19(4), 541–563.
Hommes, J., Rienties, B., Grave, W., Bos, G., Schuwirth, L., & Scherpbier, A. (2012). Visualising the invisible: a network approach to reveal the informal social side of student learning. Advances in Health Sciences Education, 17(5), 743–757.
Hox, J. J. (2002). Multilevel analysis: Techniques and applications. Quantitative methodology series. Mahwah, NJ: Lawrence Erlbaum Associates.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.
Kaufman, D. M., & Mann, K. V. (1996). Comparing students’ attitudes in problem-based and conventional curricula. Academic Medicine, 71(10), 1096–1099.
Kiessling, C., Schubert, B., Scheffner, D., & Burger, W. (2004). First year medical students’ perceptions of stress and support: a comparison between reformed and traditional track curricula. Medical Education, 38(5), 504–509.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press.
Koh, G. C.-H., Khoo, H. E., Wong, M. L., & Koh, D. (2008). The effects of problem-based learning during medical school on physician competency: a systematic review. Canadian Medical Association Journal, 178(1), 34–41.
Kulasegaram, K. M., Grierson, L. E. M., & Norman, G. R. (2013). The roles of deliberate practice and innate ability in developing expertise: evidence and implications. Medical Education, 47(10), 979–989.
Kusurkar, R. A., ten Cate, T. J., Vos, C. M. P., Westers, P., & Croiset, G. (2013). How motivation affects academic performance: a structural equation modelling analysis. Advances in Health Sciences Education, 18(1), 57–69.
Laireiter, A.-R. (1996). Skalen Soziale Unterstützung.: SSU. Mödling: Dr. Schuhfried.
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed). Wiley series in probability and statistics. Hoboken, N.J: Wiley.
Loyens, S. M. M., Magda, J., & Rikers, R. M. J. P. (2008). Self-directed learning in problem-based learning and its relationships with self-regulated learning. Educational Psychology Review, 20(4), 411–427.
Mattick, K., & Knight, L. (2007). High-quality learning: Harder to achieve than we think? Medical Education, 41(7), 638–644.
Maudsley, G. (1999). Do we all mean the same thing by “problem-based learning”? A review of the concepts and a formulation of the ground rules. Academic Medicine, 74(2), 178–185.
Moulaert, V., Verwijnen, M. G. M., Rikers, R., & Scherpbier, A. J. (2004). The effects of deliberate practice in undergraduate medical education. Medical Education, 38(10), 1044–1052.
Moust, J. H. C., van Berkel, H. J. M., & Schmidt, H. G. (2005). Signs of erosion: Reflections on three decades of problem-based learning at Maastricht University. Higher Education, 50(4), 665–683.
Muthén, L., & Muthén, B. (1998–2011). Mplus User’s Guide (6th ed.). Los Angeles, CA: Muthén & Muthén.
Neufeld, V. R., & Barrows, H. S. (1974). The “McMaster Philosophy”: an approach to medical education. Journal of Medical Education, 49(11), 1040–1050.
Neville, A. J., & Norman, G. R. (2007). PBL in the undergraduate MD Program at McMaster University: Three iterations in three decades. Academic Medicine, 82(4), 370–374.
Newman, M. (2003). A pilot systematic review and meta-analysis on the effectiveness of problem-based learning. On behalf of the Campbell Collaboration Systematic Review Group on the Effectiveness of Problem-based Learning. Newcastle upon Tyne, UK: Learning and Teaching Support Network-01, University of Newcastle.
Nijhuis, J. H., Segers, M. R., & Gijselaers, W. (2005). Influence of redesigning a learning environment on student perceptions and learning strategies. Learning Environments Research, 8(1), 67–93.
Nisbett, R. E., Aronson, J., Blair, C., Dickens, W., Flynn, J., Halpern, D. F., & Turkheimer, E. (2012). Intelligence: New findings and theoretical developments. American Psychologist, 67(2), 130–159.
Nolen-Hoeksema, S., Wisco, B. E., & Lyubomirsky, S. (2008). Rethinking rumination. Perspectives on Psychological Science, 3(5), 400–424.
Norman, G. (2004). Editorial: Beyond PBL. Advances in Health Sciences Education, 9(4), 257–260.
Norman, G. (2008). Problem-based learning makes a difference. But why? Canadian Medical Association Journal, 178(1), 61–62.
Norman, G. R., & Schmidt, H. G. (2000). Effectiveness of problem-based learning curricula: theory, practice and paper darts. Medical Education, 34(9), 721–728.
Nouns, Z. M., & Georg, W. (2010). Progress testing in German speaking countries. Medical Teacher, 32(6), 467–470.
Nouns, Z., Hanfler, S., Brauns, K., Foeller, T., Fuhrmann, S., Koelbel, S., & Mertens, A. (2004). Do progress tests predict the outcome of national exams? AMEE Conference 2004 Edinburgh. Short Communication 2F3.
Nouns, Z., Schauber, S., Witt, C., Kingreen, H., & Schüttpelz-Brauns, K. (2012). Development of knowledge in basic sciences: a comparison of two medical curricula. Medical Education, 46(12), 1206–1214.
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341.
Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology, 36(1), 36–48.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105.
Pintrich, P. R., Smith, D. A. F., García, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813.
Preacher, K. J. (2008). Latent growth curve modeling. Thousand Oaks: Sage.
R Core Team. (2013). R: A Language and Environment for Statistical Computing. Vienna, Austria. http://www.R-project.org/
Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353–387. doi:10.1037/a0026838.
Riediger, M., Wrzus, C., Schmiedek, F., Wagner, G. G., & Lindenberger, U. (2011). Is seeking bad mood cognitively demanding? Contra-hedonic orientation and working-memory capacity in everyday life. Emotion, 11(3), 656.
Rytkönen, H., Parpala, A., Lindblom-Ylänne, S., Virtanen, V., & Postareff, L. (2012). Factors affecting bioscience students’ academic achievement. Instructional Science, 40(2), 241–256.
Schauber, S. K., Hecht, M., Nouns, Z. M., & Dettmer, S. (2013). On the role of biomedical knowledge in the acquisition of clinical knowledge. Medical Education, 47(12), 1223–1235.
Schmidmaier, R., Holzer, M., Angstwurm, M., Nouns, Z., Reincke, M., & Fischer, M. R. (2010). Using the Progress Test Medizin (PTM) for evaluation of the Medical Curriculum Munich (MeCuM). GMS Zeitschrift für medizinische Ausbildung, 27(5), 1–14.
Schmidt, H. G. (1983). Problem-based learning: Rationale and description. Medical Education, 17(1), 11–16.
Schmidt, H. G., Rotgans, J. I., & Yew, E. H. J. (2011). The process of problem-based learning: what works and why. Medical Education, 45(8), 792–806.
Schmidt, H. G., van der Moldem, H. T., te Winkel, W. W., & Wijen, W. H. (2009). Constructivist, problem-based learning does work: A meta-analysis of curricular comparisons involving a single medical school. Educational Psychologist, 44(4), 227–249.
Schmitz, B., & Wiese, B. S. (2006). New perspectives for the evaluation of training sessions in self-regulated learning: Time-series analyses of diary data. Contemporary Educational Psychology, 31(1), 64–96.
Schwarzer, R., & Jerusalem, M. (1995). Generalized Self-Efficacy scale. In J. Weinman, S. Wright, & M. Johnston (Eds.), Measures in health psychology: A user’s portfolio. Causal and control beliefs (pp. 35–37). Windsor: NFER-NELSON.
Sliwinski, M. J., Smyth, J. M., Hofer, S. M., & Stawski, R. S. (2006). Intraindividual coupling of daily stress and cognition. Psychology and Aging, 21(3), 545.
Smits, P. B. A. (2002). Problem based learning in continuing medical education: a review of controlled evaluation studies. BMJ, 324(7330), 153–156.
Stegers-Jager, K. M., Cohen-Schotanus, J., & Themmen, A. P. N. (2012). Motivation, learning strategies, participation and medical school performance. Medical Education, 46(7), 678–688.
Strobel, J., & van Barneveld, A. (2009). When is PBL more effective? A Meta-synthesis of meta-analyses comparing PBL to conventional classrooms. Interdisciplinary Journal of Problem-based Learning, 3(1), 45–58.
Struyven, K., Dochy, F., Janssens, S., & Gielen, S. (2006). On the dynamics of students’ approaches to learning: The effects of the teaching/learning environment. Learning and Instruction, 16, 279–294.
ten Cate, O. (2001). What happens to the student? The neglected variable in educational outcome research. Advances in Health Sciences Education, 6(1), 81–88.
van der Veken, J., Valcke, M., de Maeseneer, J., & Derese, A. (2009a). Impact of the transition from a conventional to an integrated contextual medical curriculum on students’ learning patterns: A longitudinal study. Medical Teacher, 31(5), 433–441.
van der Veken, Jos, Valcke, M., de Maeseneer, J., Schuwirth, L., & Derese, A. (2009b). Impact on knowledge acquisition of the transition from a conventional to an integrated contextual medical curriculum. Medical Education, 43(7), 704–713.
Vanthournout, G., Donche, V., Gijbels, D., & Petegem, P. (2009). Alternative data-analysis techniques in research on student learning: illustrations of a person-oriented and developmental perspectives. Reflecting Education, 5(2), 35–51.
Verhoeven, B. H., Snellen-Balendong, H. A., Hay, I. T., Boon, J. M., Van Der Linde, M. J., Blitz-Lindeque, J. J., & Scherpbier, A. (2005). The versatility of progress testing assessed in an international context: a start for benchmarking global standardization? Medical Teacher, 27(6), 514–520.
Verhoeven, B. H., & Verwijnen, G. M. (1998). An analysis of progress test results of PBL and non-PBL students. Medical Teacher, 20(4), 310–316.
Verhoeven, B. H., Verwijnen, G. M., Scherpbier, A., & Van der Vleuten, C. P. (2002). Growth of medical knowledge. Medical Education, 36(8), 711–717.
Vernon, D. T., & Blake, R. L. (1993). Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine, 68(7), 550–563.
Walberg, H. J., & Tsai, S.-L. (1983). Matthew effects in education. American Educational Research Journal, 20(3), 359–373.
Wäschle, K., Allgaier, A., Lachner, A., Fink, S., & Nückles, M. (2014). Procrastination and self-efficacy: Tracing vicious and virtuous circles in self-regulated learning. Learning and Instruction, 29, 103–114.
Westermann, R., Heise, E., Spies, K., & Trautwein, U. (1996). Identifikation und Erfassung von Komponenten der Studienzufriedenheit. [Identifying and assessing components of student satisfaction.]. Psychologie in Erziehung und Unterricht, 43(1), 1–22.
Wild, K.-P., & Schiefele, U. (1994). Lernstrategien im Studium: Ergebnisse zur Faktorenstruktur und Reliabilität eines neuen Fragebogens. Zeitschrift für Differentielle und Diagnostische Psychologie, 15, 185–200.
Wolf, F. M. (1986). Meta-Analysis: Quantitative Methods for Research Synthesis. Meta-analysis: Quantitative Methods for Research Synthesis. Thousand Oaks: SAGE Publications.
Acknowledgments
The work of Stefan K. Schauber was funded by the German Federal Ministry of Education and Research (BMBF) within the project “Competence Acquisition and Learning Trajectories in Medical Training” (Grant 01JG1055).
Author information
Authors and Affiliations
Corresponding author
Appendix: Example Items
Appendix: Example Items
Example item from the clinical knowledge subset
Three hours after carrying a 130 kg washing machine, a 24-year old patient (180 cm, 60 kg) feels an intense pain in the right side of his chest. The initial pain abates after only a few minutes but a light feeling of pressure in the right half of the thorax develops, with pain associated with breathing that increases over time. Eventually an emergency doctor is notified. He finds slight shortness of breath, reduced breathing sounds and percussive response on the right side, a heart rate of 125/min, blood pressure of 120/80 mmHg and a blood oxygen saturation level of 95 % (under normal conditions).
What working diagnosis do you suggest?
-
a)
lung embolism
-
b)
heart attack
-
c)
pneumonia
-
d)
pleural effusion
-
e)
pneumothorax
Organ system: the respiratory tract
Medical discipline: anaesthesiology, emergency and intensive care medicine
Correct answer: e
Example item from the biomedical knowledge subset
Which of the following hormones is integral in compensating for a considerable drop in blood volume?
-
a)
adrenaline
-
b)
aldosterone
-
c)
thyroxine
-
d)
nitric oxide
Organ system: hormones, metabolism
Medical discipline: physiology, physics
Correct answer: b
Rights and permissions
About this article
Cite this article
Schauber, S.K., Hecht, M., Nouns, Z.M. et al. The role of environmental and individual characteristics in the development of student achievement: a comparison between a traditional and a problem-based-learning curriculum. Adv in Health Sci Educ 20, 1033–1052 (2015). https://doi.org/10.1007/s10459-015-9584-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-015-9584-2