Skip to main content

Advertisement

SpringerLink
Log in

Creating classroom-level measures of citizenship education climate

  • Published:
Learning Environments Research Aims and scope Submit manuscript

Abstract

Optimal classroom climates for civic education encourage the development of knowledge, skills and dispositions necessary for students to become involved citizens. One of the simplest and most common ways of measuring classroom climate in this field is to aggregate individual students’ perceptions of classroom climate to the group level; however, little has been done to assess the quality of such aggregate measures. We examined data from the United States sample of the 1999 IEA Civic Education Study to determine the reliability of aggregate measures of the openness of classroom discussion climate as reported by students. Student and class-level correlates of within-classroom variability in students’ reports were considered, and aggregate reports were correlated with student outcomes. Although the aggregated classroom climate measure had only modest reliability, we identified several within-classroom sources of variability in ratings. Classrooms were also more variable in their ratings if the average climate perceptions were lower on average, and if the classrooms served students with fewer educational resources at home. However, the aggregate measures did predict a variety of student outcomes even after controlling for within-group sources of variability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. One intact classroom per school was sampled by CIVED methodologists. For clarity in this paper, we use the term ‘classroom’ to refer to group-level measures of classroom climate; however, the term ‘school-level’ would be appropriate also.

  2. An additional research question was also considered that would have pertained to gaining evidence of convergent validity, correlating aggregate scores on the Openness of Classroom Climate measure to reports of classroom activities by others. However, the lack of appropriate criterion measures in the CIVED dataset made this difficult. Principals and teachers in a school were asked whether students “learned to understand people who have different ideas.” However, the teachers who were asked this question did not necessarily teach the tested class of students, and the principals were responding about civics classes in their school generally and not the tested class.

References

  • Amadeo, J. A., Torney-Purta, J., Lehmann, R., Husfeldt, V., & Nikolova, R. (2002). Civic knowledge and engagement: An IEA study of upper secondary students in sixteen countries. Amsterdam: IEA.

    Google Scholar 

  • Barber, C., & Torney-Purta, J. (2009). Gender differences in political attitudes and efficacy as influenced by national and school contexts: Analysis from the IEA Civic Education Study. In D. Baker & A. Wiseman (Eds.), Gender, equality, and education from international perspectives (International Perspectives on Education and Society, Vol. 10, pp. 357–394). Bingley, UK: Emerald Group Publishing.

  • Campbell, D. E. (2007). Sticking together: Classroom diversity and civic education. American Politics Research, 35, 57–78. doi:10.1177/1532673X06294503.

    Article  Google Scholar 

  • Campbell, D. E. (2008). Voice in the classroom: How an open classroom climate fosters political engagement among adolescents. Political Behavior, 30, 437–454. doi:10.1007/s11109-008-9063-z.

    Article  Google Scholar 

  • Caponera, E., & Losito, B. (2011, April). The roles of schools and communities in civic and citizenship education. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Retrieved from http://iccs.acer.edu.au/uploads/File/AERA2011/AERA_ICCS_SchoolsCommunity(NewOrleans2011).pdf.

  • Dassonneville, R., Quintelier, E., Hooghe, M., & Claes, E. (2012). The relation between civic education and political attitudes and behavior: A two-year panel study among Belgian late adolescents. Applied Developmental Science, 16, 140–150. doi:10.1080/10888691.2012.695265.

    Article  Google Scholar 

  • Ehman, L. H. (1980). Change in high school students’ political attitudes as a function of social studies classroom climate. American Educational Research Journal, 17, 253–265. doi:10.3102/00028312017002253.

    Article  Google Scholar 

  • Homana, G., & Barber, C. (2007). School climate for citizenship education: A comparative study of England and the United States. In Proceedings of the IRC-2006: Vol. 2. CivEd, PIRLS, and SITES (pp. 115–130). Amsterdam: IEA.

  • Homana, G., Barber, C., & Torney-Purta, J. (2006). Assessing school citizenship education climate: Implications for the social studies. CIRCLE working paper #48. College Park, MD: Center for Information and Research on Civic Learning and Engagement, University of Maryland.

  • Homana, G., Croninger, R., & Torney-Purta, J. (2010). Adolescent civic engagement in Australia and the United States: The role of communities of practice. Paper presented at the 4th IEA international research conference, Gothenburg, Sweden.

  • Husfeldt, V., Barber, C., & Torney-Purta, J. (2005). Students’ social attitudes and expected political participation: New scales in the enhanced database of the IEA Civic Education Study. CEDARS Report. College Park, MD: University of Maryland.

  • Ichilov, O. (2007). Civic knowledge of high school students in Israel: Personal and contextual determinants. Political Psychology, 28, 417–440. doi:10.1111/j.1467-9221.2007.00580.x.

    Article  Google Scholar 

  • Long, S., & Long, R. (1975). Controversy in the classroom: Student viewpoint and educational outcome. Teaching Political Science, 2(3), 275–299. doi:10.1080/00922013.1975.10740053.

  • Mahatmya, D., & Lohman, B. J. (2012). Predictors and pathways to civic involvement in emerging adulthood: Neighborhood, family, and school influences. Journal of Youth and Adolescence, 41, 1168–1183. doi:10.1007/s10964-012-9777-4.

    Article  Google Scholar 

  • Miller, A. D., & Murdock, T. B. (2007). Modeling latent true scores to determine the utility of aggregate student perceptions as indicators in HLM: The case of classroom goal structures. Contemporary Educational Psychology, 32, 83–104. doi:10.1016/j.cedpsych.2006.10.006.

    Article  Google Scholar 

  • Raudenbush, S., Bryk, A., Cheong, Y., & Congdon, R. (2004). HLM 6: Hierarchical linear and nonlinear models. Lincolnwood, IL: Scientific Software International.

    Google Scholar 

  • Sapiro, V. (2004). Not your parents’ political socialization: Introduction for a new generation. Annual Review of Political Science, 7, 1–23. doi:10.1146/annurev.polisci.7.012003.104840.

    Article  Google Scholar 

  • Schulz, W., Ainley, J., Fraillon, J., Kerr, D., & Losito, B. (2010a). ICCS 2009 international report: Civic knowledge, attitudes, and engagement among lower-secondary schools in 38 countries. Amsterdam: IEA.

    Google Scholar 

  • Schulz, W., Fraillon, J., Ainley, J., & van de gaer, E. (2010b, July). Explaining differences in civic knowledge across 38 countries. Paper prepared for the 4th IEA international research conference, Gothenburg. Retrieved from http://iccs.acer.edu.au/uploads/File/papers/IRCpaper_ICCS_CivicKnowledge(Gothenburg%201-3%20July).pdf.

  • Schulz, W., Fraillon, J., Ainley, J., & van de Gaer, E. (2011, April). Multi level analysis of factors explaining differences in civic knowledge. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

  • Schulz, W., & Sibberns, H. (2004). IEA Civic Education Study technical report. Amsterdam: IEA.

    Google Scholar 

  • Seidman, E., Tesng, V., & Weisner, T. S. (2006). Social setting theory and measurement. Retrieved from http://www.wtgrantfoundation.org/usr_doc/SeidmanTsengWeisner.pdf.

  • Steiner-Khamsi, G., Torney-Purta, J., & Schwille, J. (2002). New paradigms and recurring paradoxes in education for citizenship. New York: Elsevier.

    Google Scholar 

  • Torney-Purta, J., Barber, C., & Wilkenfeld, B. (2007). Latino adolescents’ civic development in the United States: Research results from the IEA Civic Education Study. Journal of Youth and Adolescence, 36(3), 111–126. doi:10.1007/s10964-006-9121-y.

    Article  Google Scholar 

  • Torney-Purta, J., Lehmann, R., Oswald, H., & Schulz, W. (2001). Citizenship and education in twenty-eight countries: Civic knowledge and engagement at age fourteen. Amsterdam: IEA.

    Google Scholar 

  • Torney-Purta, J., Schwille, J., & Amadeo, J. (1999). Civic education across countries: Twenty-four national case studies from the IEA Civic Education Project. Amsterdam: IEA.

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank Tamera Murdock for her assistance in developing the research questions addressed in this analysis, Judith Torney-Purta for her feedback on the findings, and Jessica Ross and Jennifer Schaafsma for their assistance in the preparation of this manuscript. This project was funded by a University of Missouri-Kansas City Faculty Research Grant.

Author information

Authors and Affiliations

  1. Division of Counseling and Educational Psychology, School of Education, University of Missouri, Kansas City, MO, 64110, USA

    Carolyn Barber, Sachiko Ogata Sweetwood & Makini King

Authors
  1. Carolyn Barber
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sachiko Ogata Sweetwood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Makini King
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carolyn Barber.

Appendix: Covariates and predictors used in analysis

Appendix: Covariates and predictors used in analysis

Research Question 2

Gender: Student self-report

Response Options: Male = 0; Female = 1.

  • Race: Re-coded from two variables: “What is your Race” and “What is your ethnicity [Latino/Non-Latino].”

  • Response Options: White non-Latino, African-American non-Latino, Latino, Asian, Native American, Multiracial, Other

  • Number of Books in the Home: “About how many books are there in your home?” Do not count newspapers, magazines or books for school.

  • Response Options: None, 1–10, 11–50, 51–100, 101–200, More than 200.

  • Learn to Understand People: “In school I have learned to understand people who have different ideas.”

  • Response Options: Strongly disagree, Disagree, Agree, Strongly agree, Don’t know.

  • Learn to Cooperate: “In school I have learned to co-operate in groups with other students.”

  • Response Options: Strongly disagree, Disagree, Agree, Strongly agree, Don’t know.

  • Learn to Contribute to Solving Problems: “In school I have learned to contribute to solving problems in the community [society].”

  • Response Options: Strongly disagree, Disagree, Agree, Strongly agree, Don’t know.

  • Total Civic Knowledge: IRT Scale (see Schulz and Sibberns 2004). M = 100, SD = 20. Includes both items on civic content knowledge and skills in interpreting political material.

  • Response Options (original items): Correct/Incorrect

  • Confidence in the Value of Participation: IRT Scale (see Schulz and Sibberns 2004). M = 10, SD = 2.

  • Sample Item: “Lots of positive changes happen in this school when students work together.”

  • Response Options (original items): Strongly disagree, Disagree, Agree, Strongly agree, Don’t know.

  • Read Newspaper: Composite of two items: Read National News in Newspaper and Read International News in Newspaper “How often do you read articles (stories) in the newspaper about what is happening in this country?” and “How often do you read articles (stories) in the newspaper about what is happening in other countries?”

  • Response Options: Never, Rarely, Sometimes, Often, Don’t know.

  • Discuss Politics with Teachers: “How often do you have discussions of what is happening in your national politics with teachers?” and “How often do you have discussions of what is happening in international politics with teachers?”

  • Response Options: Never, Rarely, Sometimes, Often, Don’t know.

  • Discuss Politics with Peers: “How often do you have discussions of what is happening in your national politics with people of your own age?” and “How often do you have discussions of what is happening in international politics with people of your own age?”

  • Response Options: Never, Rarely, Sometimes, Often, Don’t know.

  • Discuss Politics with Parents: “How often do you have discussions of what is happening in your national politics with parents or other adult family members?” and “How often do you have discussions of what is happening in international politics with parents or other adult family members?”

  • Response Options: Never, Rarely, Sometimes, Often, Don’t know.

  • Watch TV News: “How often do you listen to news broadcasts on television?”

  • Response Options: Never, Rarely, Sometimes, Often, Don’t know.

Research Question 3

Average Books in the Home: Aggregate of students’ reports of number of books in the home

Average Classroom Climate: Aggregate of students’ reports of the Openness of Classroom Climate for discussion (see text)

Research Question 4

Civic Content Knowledge: IRT Scale (see Schulz and Sibberns 2004). M = 20; SD = 100

Response Options (original items): Correct/Incorrect

  • Skills in Interpreting Political Material: IRT Scale (see Schulz and Sibberns 2004). M = 20; SD = 100.

  • Response Options (original items): Correct/Incorrect

  • Expectations of Informed Voting: IRT Scale (see Husfeldt et al. 2005) M = 20 SD; SD = 10.

  • Sample Item: “When you are an adult, what do you expect that you will do? Vote in national elections.”

  • Response Options (original items): I will certainly not do this, I will probably not do this, I will probably do this, I will certainly do this, Don’t know.

  • Expectations of Illegal Protest: IRT Scale (see Husfeldt et al. 2005) M = 20 SD; SD = 10.

  • Sample Item: “What do you expect that you will do? Block traffic as a form of protest”

  • Response Options (original items): I will certainly not do this, I will probably not do this, I will probably do this, I will certainly do this, Don’t know.

  • Support for Women’s Rights: IRT Scale (see Schulz and Sibberns 2004) M = 20; SD = 10.

  • Sample Item: “Women should run for public office [a seat in the legislature] and take part in the government just as men do.”

  • Response Options (original items): Strongly disagree, Disagree, Agree, Strongly agree, Don’t know.

  • Positive Attitudes toward Immigrants’ Rights: IRT Scale (see Schulz and Sibberns 2004) M = 20; SD = 10.

  • Sample Item: “Immigrants should have all the same rights that everyone else in a country has.”

  • Response Options (original items): Strongly disagree, Disagree, Agree, Strongly agree, Don’t know.

  • Trust in Government Institutions: IRT Scale (see Schulz and Sibberns 2004) M = 20; SD = 10.

  • Sample Item: “How much of the time can you trust each of the following institution: The national government”

  • Response Options (original items): Never, Only some of the time, Most of the time, Always, Don’t know.

  • Internal Political Efficacy: IRT Scale (see Husfeldt et al. 2005) M = 20 SD; SD = 10.

  • Sample Item: “I am able to understand most political issues easily.”

  • Response Options (original items): Strongly disagree, Disagree, Agree, Strongly agree, Don’t know.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barber, C., Sweetwood, S.O. & King, M. Creating classroom-level measures of citizenship education climate. Learning Environ Res 18, 197–216 (2015). https://doi.org/10.1007/s10984-015-9180-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10984-015-9180-7

Keywords

Search

Navigation