Context Matters: Social Psychological Factors That Underlie Academic Performance across Seven Institutions

INTRODUCTION

Broadening participation in science, technology, engineering, and mathematics (STEM) fields requires acknowledging the many different student experiences in higher education. Students who identify with groups historically excluded from STEM fields face barriers as they navigate these pathways into the STEM workforce (Seymour and Hewitt, 1997; Seymour and Hunter, 2019). Social challenges that disproportionately affect such students include social exclusion (Hurtado and Ruiz, 2012), a negative campus climate (Koo, 2021), and discrimination from peers and authority figures (Hurtado and Ruiz Alvarado, 2015; Park et al., 2020). Meanwhile, national efforts call for more STEM graduates to address the shortage of STEM workers in the labor market and to expand underrepresented minority (URM¹) participation (American Association for the Advancement of Science, 2011; National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011; President’s Council of Advisors on Science and Technology, 2012). Increased participation of students in STEM education will directly impact national science literacy, provide high-quality education to the STEM workforce, and contribute to critical scientific advances. One significant barrier to student participation and persistence in STEM is introductory STEM courses that students are required to complete before moving on to more advanced course work (Crisp et al., 2009; Mervis, 2011). In such classes, instructors deliver foundational content knowledge—often through undisrupted lecture (Stains et al., 2018)—to relatively large lecture halls of students. Failure and attrition rates are high (Seymour and Hunter, 2019), and students perform poorly in these courses relative to non-STEM courses (Mervis, 2011; Koester et al., 2016).  URM students face additional barriers in these introductory courses, in large part due to gaps in incoming preparation (Harris et al., 2020; Salehi et al., 2020) and limited opportunities in some institutions to interact with instructors (Hurtado et al., 2011). These barriers lead to underperformance and lower probability of persisting in STEM fields for URM students compared with their overrepresented majority (non-URM) peers (National Science Foundation, 2019; Salehi et al., 2020).

To address student performance barriers in introductory biology courses, recent research in biology education focuses on best practices for evidence-based teaching strategies that improve student academic performance (Offerdahl et al., 2011; Freeman et al., 2014; Theobald et al., 2020). Though important to our understanding of student learning, the majority of current research into student experiences in introductory biology takes place at large, predominantly White institutions, such as public, 4-year PhD- and master’s degree–granting research-intensive universities (hereafter, selective institutions; Schinske et al., 2017; Thompson et al., 2020). This overrepresents the experiences of these students in the contemporary literature. Less research has focused on other types of institutions, such as community colleges, minority-serving institutions, or more inclusive 4-year institutions (hereafter, inclusive institutions), even though they are essential educational institutions contributing to our future STEM workforce (Thompson et al., 2020). For example, while community colleges serve almost half of all undergraduate students in the United States, and an even larger percentage of URM and first-generation college students (American Association of Community Colleges [AACC], 2021), relatively little biology education research (BER) focuses on community colleges (Schinske et al., 2017). Further, the authors who conduct research on community colleges are often not affiliated with community colleges and less familiar with the specific institutional context (Schinske et al., 2017). The reason for this discrepancy is likely due to heavy teaching and service expectations for community college faculty, but still represents a gap in our understanding of the student experience.

By developing and implementing innovations and reforms informed primarily by research conducted at selective institutions, we overlook important contextual factors that likely differ between selective institutions and community colleges and inclusive institutions; factors such as instructor teaching loads and expectations, students’ levels of academic preparation, their educational experience based on salient social identities such as gender or race/ethnicity, and students’ socioeconomic status. Pursuing research questions across multiple institutional contexts enables research to address questions that are unattainable within a single institution. Multiple factors specific to an individual institution are likely to impact research outcomes. These could include (but are not limited to) teaching culture, student body, or class size. To demonstrate the importance of institutional context, we examined differences in demographic performance gaps, as well as two psychological factors, specifically test anxiety and ethnicity stigma consciousness (hereafter ESC; Sarason, 1961; Pintrich et al.,1993; Steele and Aronson, 1995; Picho and Brown, 2011). We tested their impacts on student performance across seven institutions from three different institution types: community colleges, 4-year inclusive institutions, and 4-year selective institutions. We selected these two social psychological factors because they are potentially impacted by institutional or classroom features (Matthews et al., 1999; Ergene, 2003; Massey and Fischer, 2005; Osborne and Walker, 2006; Kellow and Jones, 2008; Ballen et al., 2017b) and can have consequential impacts on learning and academic performance (Kellow and Jones, 2008; Appel and Kronberger, 2012; Salehi et al., 2019a).

We selected test anxiety to measure across contexts because high levels of test anxiety reduce student performance outcomes, and we predicted this may vary across multiple contexts based on factors such as class size, course competitiveness, or other traits that may relate to institution type (von der Embse et al., 2018). Test anxiety is characterized by feelings of tension, worried thoughts, and negative physiological reactions in an academic evaluative setting (American Psychiatric Association, 2013). Work across STEM fields shows that test anxiety negatively impacts assessment performance outcomes (Chapell et al., 2005; DordiNejad et al., 2011; Ali and Mohsin, 2013; Shapiro, 2014; Harris et al., 2019). Results from previous research showed that both in introductory biology (Ballen et al., 2017b; Cotner et al. 2020) and across STEM disciplines in higher education (Salehi et al., 2019b), for women only, test anxiety negatively influenced exam performance. Work conducted at a separate institution could not replicate these results and concluded that women underperform on exams for reasons other than gender-based differences in test anxiety (Harris et al., 2019). It is possible that students with higher content-related anxiety perceive test difficulty differently, which may in turn impact test anxiety (Hong and Karstensson, 2002). The impacts of test anxiety on performance of students with underrepresented identities in biology has not been studied as extensively, to our knowledge. One study conducted mediation analysis to examine the relationship between a number of social psychological factors, including test anxiety, as well as incoming preparation, minority and first-generation status, and academic performance outcomes within a college of biological sciences (Salehi et al., 2020). Results showed that URM and first-generation status was highly related to students’ incoming preparation. Lower measures of incoming preparation were associated with higher anxiety scores, and higher anxiety was associated with lower exam scores.

We also selected ESC to measure across contexts. ESC is a measure of stereotype threat, which we measured as the extent to which one feels self-conscious of the stigma associated with one’s race/ethnicity (Picho and Brown, 2011). Stereotype threat can invoke a disruptive state, undercutting performance and aspirations within a domain (Spencer et al., 2016). Ever since Steele and Aronson’s (1995) work on this phenomenon, research has documented its presence among negatively stereotyped individuals in hundreds of studies. Stereotype threat has been shown to reduce an individual’s ability to learn (Rydell et al., 2010; Taylor and Walton, 2011; Boucher et al., 2012). It can be reduced or invoked through relatively small changes in instructor behavior or the classroom environment, such as stating that tests are difficult and performance is linked to overall ability (Steele and Aronson, 1995). Little work on stereotype threat has been conducted in the context of postsecondary biology. Taasoobshirazi et al. (2019) administered study questions with different stereotype threat conditions in the instructions and examined biology performance based on gender, as well as self-efficacy, motivation, and domain identification. While they did not identify differences in gender outcomes by treatment, they observed women reported a greater domain identification with biology. To our knowledge, no previous work has focused on stereotype threat based on race/ethnicity in the context of postsecondary biology. Additionally, quantifying this social psychological factor as it varies across classrooms or institutions has not been explored, despite accumulating evidence that institutional context shapes student experiences (Chang et al., 2008; Hurtado et al., 2010; Wang, 2013; Winkle-Wagner and McCoy, 2018). For example, we may expect ESC to be higher in predominantly White spaces such as the selective institutions in our sample.

Using data generated collaboratively across multiple institutions, we tested the following research questions:

1. To what extent do we observe gaps in academic performance based on demographic status such as binary gender² and URM status in introductory biology across different institutional contexts?

2. To what extent do we observe differences in social psychological factors, that is, test anxiety and ESC, based on demographic status in introductory biology across different institutional contexts?

3. How do these social psychological factors mediate potential differences in performance outcomes based on demographic status across different institution contexts?

Traditional views of the performance gap attribute student success primarily to personal attributes, such as deficiencies in ability or motivation. Focusing on performance gaps in this way deflects attention away from the key responsibility educational institutions play in student success and applies underlying deficit perspectives to marginalized students in STEM (Smit, 2012; Zhao, 2016). Here we stress that the opportunity gap in primary and secondary education—for example, access to high-quality curricula, instruction, and technology—contributes to gaps in academic preparation before students even enter the undergraduate biology classroom. These gaps in academic preparation are then in turn reflected in student grades in introductory courses that fail to provide students with equal opportunities to perform (Salehi et al., 2019, 2020). Understanding the extent of performance disparities in courses and their prevalence across institutional contexts is an important first step to fulfill the responsibility of higher education to educate all students. Our focus on academic disparities aims to change structural barriers within our classrooms and institutions such that student performance patterns (e.g., exam grades, total grades) are not predicted primarily by characteristics such as race, class, ethnicity, gender, proficiency in a dominant language, or other marginalized student identities.

METHODS

Participants

We solicited participation through an existing professional network from instructors who teach biology at a range of institutions (Research Coordination Network, National Science Foundation RCN–UBE Incubator: Equity and Diversity in Undergraduate STEM; grant no. 1729935 awarded to S.C. and C.J.B.; RCN–UBE grant no. DBI-1919462 awarded to S.C., C.J.B., S.F., Harshman, and C.H.). Equity and Diversity in Undergraduate STEM is a network of educators and discipline-based education researchers who work together on research and teaching projects in the context of biology curricula and is supported by the National Science Foundation. The objectives of the network are to: “(1) reveal differences, if they exist, in the cultural climate for women and minorities in STEM disciplines (initially focusing on biology) as a function of geography, institution type, and cultural profile of the participating departments; (2) increase the number of faculty in the United States that are familiar with barriers to inclusion in STEM, and can apply evidence-based techniques for countering known barriers; (3) develop a community of faculty that can serve as leaders-at their home institutions and nationally-in inclusive teaching and assessment; and (4) identify cultural factors associated with a shift toward evidence-based teaching, especially pertaining to inclusive teaching” (Thompson et al., 2020).

From the RCN network, we collected data from 33 biology courses at seven institutions between 2016 and 2018. Participating institutions were a convenience sample chosen from a range of institutional types (public and private, large and small) and settings (college towns to large metropolitan areas). We classified these institutions based on undergraduate profile according to the Carnegie Classification of Institutions of Higher Education. This allowed us to group institutions by admissions selectivity criteria and by whether they were 2-year or 4-year institutions. The 2-year institutions in our sample were community colleges, and so use we this term throughout this research. Within 4-year institutions, we classified institutions as “selective” if they fell into the Carnegie classification of “more selective,” indicating that the 25th percentile of first-time first-year students received greater than 21 on their American College Testing (ACT) entrance exam score. We classified institutions as “inclusive” if they fell into the Carnegie category of “inclusive” or “selective,” with 25th percentile ACT scores less than 21. Additionally, we worked with representatives at each institution to generate Table 1, which summarizes some contextual characteristics of the participating institutions.

We worked with individual instructors who teach biology classes across these institutions to collect data. Our sample represented 12 courses at two community colleges (n = 454 students, average class size = 38 ± 6.68), six courses at three inclusive institutions that were also minority-serving institutions (n = 1045 students, average class size = 199 ± 49.85), and 15 courses at three selective institutions (n = 3594 students, average class size = 239 ± 40.26). During the 2-year study period, we obtained grade data disaggregated into exam scores, non-exam scores, and final grades and institutional data on gender and race/ethnicity from all courses in the study. Also, during the last few weeks of the semester for a subset of students, we administered a survey to measure test anxiety and ethnicity stigma consciousness (Table 2). Note we removed one institution, representing one class, from our analyses of demographic gaps due to the fact that 100% of student were from underrepresented racial minority groups. All aspects of research were reviewed and approved by each institution’s institutional review board.

Survey Measures

To measure test anxiety, we used a four-item test anxiety construct from the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993). The MSLQ test anxiety construct asked students about their experiences during testing, such as whether they felt distracted by their anxiety during exams. To measure ESC, we used the five-item ESC construct from the Social Identities and Attitudes Scale (SIAS; Picho and Brown, 2011). ESC measured the extent to which one is conscious about one’s ethnic identity and includes items such as “My ethnicity influences how teachers interact with me.” For both constructs, students rated their agreement with each item on a seven-point Likert scale ranging from “strongly disagree” to “strongly agree.” We have included the full list of survey items in Supplemental Table S1.

Analysis

We performed all statistical analyses in R (R Core Team, 2019). We reported exam scores as the average of all exam scores earned by each student, and we normalized this average for each course to control for variation in grading and exam structure across courses. Therefore, each student’s exam score can be interpreted as how many standard deviations that score is from the mean of a given course.

Validating Constructs: Confirmatory Factor Analysis.

We used confirmatory factor analysis (CFA) in the R package lavaan to verify our survey item structure for test anxiety and ethnicity stigma consciousness. We used CFA because these survey items were previously developed and established as theoretical models and validated in previous research (Picho and Brown, 2011; Pintrich et al., 1993; Rosseel, 2012). Thus, we were testing to confirm whether our data set supports the preexisting survey with established structure (Knekta et al., 2019). For advantages and more details of conducting CFA via lavaan in biology education, see Ballen and Salehi (2021). To evaluate how well the CFA model captured the variation in the survey data, we used structural equation modeling (SEM) fit indices: confirmatory fit index, root mean square error of approximation, SRMR. All the fit indices fell within the acceptable range, suggesting the survey items loaded on the defined factors and the CFA model properly captures the variation in the data.

Quantifying Performance Gaps: Mixed-Effect Analysis of Performance Gaps.

To test the extent to which gaps in academic performance are related to demographic status such as gender and URM status (together referred to as “demographic status”) in introductory biology, we ran mixed-effects multivariable regression analysis using the nlme package in R. We controlled for the random effect of courses nested within each institution.

Mediation Analysis.

We also tested: 1) whether performance outcomes based on gender and URM status were mediated by test anxiety or ESC; and 2) whether mediation varied across institution type (community college, inclusive institutions, or selective institutions). To this end, we used a SEM approach using lavaan package in R (Rosseel, 2012) on a subset of students for whom we had responses to all survey items and their exam scores (n = 337 students in 12 courses at two community colleges, n = 581 students in five courses at two inclusive institutions, and n = 756 students in five courses at three selective institutions). In lavaan, we used group analysis, with institution type as the grouping factor. Therefore, the proposed mediation model would be fit to the data of each institution type separately, and whether the model was a good fit for each institution type would be tested. We tested for mediation by using the Akaike information criterion (AIC) to compare two structural models: one including only indirect effects of demographic status mediated by social psychological factors and one that also included the direct effect of demographic status on exam scores. Within the SEM analysis, we included the latent variable structure supported by our CFA. For ease of interpretation, we normalized these construct responses for the entire sample across institution types, such that reported values reflect how many standard deviations a value is from the sample’s mean score.

RESULTS

Validating Constructs: CFA

We removed one of the survey items from the SIAS (“My ethnicity impacts how I interact with people of other ethnicities”), which had a poor loading in our sample (standardized factor loading <0.70). We evaluated model fit using several fit indices, and while RMSEA, SRMR, and CFI were consistent with cutoff values (Hu and Bentler, 1999; Knekta et al., 2019), our chi-square test of model fit was highly significant (CFI = 0.99, RMSEA = 0.03, SRMR = 0.016, χ² = 50.91, p < 0.001). However, this is likely an effect of our large sample size, because chi-square tests are highly sensitive to sample size. In large samples, even a very small difference is highly likely to be significant (Hu and Bentler, 1999). Therefore, with a large sample size, if the model estimated covariance matrix is slightly different from the observed covariance matrix, the chi-square statistics might be still significant. Given that all three other indices fell within the acceptable range, and our constructs had high reliability (α  = 0.78; test anxiety(α) = 0.89; ESC(α) = 0.89) we accepted our CFA in lavaan as a good fit.

Quantifying Performance Gaps: Mixed-Effect Analysis of Performance Gaps

For gender performance analysis, we found that the nested structure of the random effect explained 4% of the variance in the data. This suggests that the random effects due to courses nested within institutions were small. This is partly due to the fact that the course grade is normalized within each course. We found no significant effect of gender on average exam scores (mixed-effects model: β_CC = 0.09 ± 0.11, p = 0.96; β_inclusive = 0.07 ± 0.08, p = 0.93; β_selective = 0.08 ± 0.03, p = 0.13). While there was also variation in the extent of the gap between men and women, a greater percentage of courses showed no difference in exam scores by gender. For URM performance analysis, we similarly found that the nested structure of the random effects explained 4% of the variance in the data, which suggests that random effect is relatively small. We found that URM students underperformed relative to non-URM students on exams at both selective and inclusive 4-year institutions, but not at community colleges (mixed-effects model: β_CC = −0.23 ± 0.12, p = 0.30; β_inclusive = −0.27 ± 0.07, p = 0.003; and β_Selective = −0.44 ± 0.05, p < 0.001). As we only found significant URM performance gaps in our sample and no gender gaps, we proceeded to only examine how social psychological factors mediate URM performance gaps.

Mediation Analysis

Our structural equation model revealed variation in mediation effects across institution types (Figure 1). We found support for partial mediation effects, as the partial mediation model was a better fit for the data (p = 0.002) and as the ∆AIC between the partial mediation and full mediation model was 9, which is larger than our cutoff ∆AIC of 2. In this model, test anxiety was universally negatively associated with exam scores (β_CC = −0.27 ± 0.02, p < 0.001; β_inclusive = −0.13 ± 0.03, p = 0.007; β_selective = −0.33 ± 0.02, p < 0.001). However, only at inclusive institutions, URM students had higher test anxiety. For community colleges or selective institutions, there was no difference in test anxiety across URM status (β_CC = 0.16 ± 0.24, p = 0.21; β_inclusive = 0.30 ± 0.16, p = 0.001; β_selective = 0.18 ± 0.23, p = 0.21). These relations imply that test anxiety mediated the effect of URM status on exam scores only at the inclusive institutions, but not at community colleges or selective institutions. The size of the mediation path can be calculated by multiplying the coefficients included in each path (β_CC = −0.04 ± 0.04, p = 0.22; β_inclusive = −0.04 ± 0.02, p = 0.04; β_selective = −0.06 ± 0.05, p = 0.22). In other words, URM students at inclusive institutions had higher test anxiety than their non-URM peers, and this higher test anxiety led to lower exam performance. At community colleges and selective institutions, on the other hand, there was no difference across URM status in test anxiety.

ESC was not a significant mediator for the effect of URM status on exam scores for all the three institution types (β_CC = 0.01 ± 0.01, p = 0.41; β_inclusive = −0.01 ± 0.01, p = 0.26; β_selective = −0.008 ± 0.02, p = 0.72). However, the reason for the lack of mediation effect varied by institution type. We did not observe an effect of URM status on ESC at community colleges (β = 0.12 ± 0.15, p = 0.38), but ESC positively affected exam scores (β = 0.1 ± 0.04, p = 0.02). At inclusive institutions, there was no significant correlation of URM status with ESC (β = 0.15 ± 0.12, p = 0.12), and no correlation of ESC with exam scores (β = −0.07 ± 0.04, p = 0.11). However, we did observe a correlation of URM status with ESC at selective institutions (β = 0.57 ± 0.13, p < 0.001) similar to the previous work (Salehi et al., 2020), though there was still no effect of ESC on exam scores (β = −0.01 ± 0.04, p = 0.72). The observation that, for selective universities, none of social psychological factors are mediators for URM performance gaps aligns with previous findings (Salehi et al., 2020) that the incoming preparation was the major mediator for demographic performance gaps in an R1 institution, and after controlling for that effect, the social psychological factors did not have a mediating effect for demographic performance gaps.

DISCUSSION

We set out to test the importance of institutional context on student experiences in biology courses using data generated collaboratively through a research coordination network (NSF DBI-1919462) from seven institutions across three institution types: community colleges, inclusive institutions, and selective institutions.

We show some patterns emerge across all institutions, regardless of institution type, such as the absence of performance gaps based on gender. However, we found that gaps in performance, test anxiety, and ESC varied across different institutions. Specifically, while we did not observe performance gaps based on race/ethnicity at the two community colleges, we did observe them at the inclusive and selective institutions in our sample. Second, we found a significant indirect effect of URM status mediated by test anxiety on exam scores at the inclusive institutions, but not at the community colleges or selective institutions. Third, we found overall that URM students at the selective institutions were more likely to report higher ESC values (but this did not in turn impact performance), while at community colleges, ESC positively affected exam scores for all students. Based on these results, we put forth a number of general and specific research recommendations.

First, we stress the importance of multiple perspectives in BER through programs that encourage research and collaborations across institution types. Other researchers have echoed the need for data-driven approaches to STEM education that span institutional contexts beyond selective institutions to address equity in STEM (Estrada et al., 2016; Schinske et al., 2017; Thompson et al., 2020). For example, despite the important role of community colleges in educating students, only 3% of recent BER articles included either community college authors or a community college biology study context (Schinske et al., 2017). And despite the unique role of community colleges in educating biology students from diverse backgrounds, Schinske et al. (2017) showed only 7% of community college BER articles explicitly studied equity and diversity. Our results support the notion that institutional context impacts barriers to student performance and interventions to support student success will likely vary in their effectiveness based on the student population. Beyond the coarse categories of institution type, we found individual institutions and classrooms also differ from one another. Specifically, we found that URM students underperformed relative to non-URM students on exams at both the inclusive and selective institutions, but not at the community colleges; however, individual courses showed variation within institution types, with the majority of courses showing no significant gap between groups. This supports the idea that factors other than precollege background, academic preparation, and admission criteria explain a large share of performance outcomes (Massey et al. 2003; Espenshade and Radford 2009; Salehi et al. 2019). Future research on factors that positively influence performance will clarify similarities across these equitable classrooms.

Second, social psychological interventions have been shown to vary in effectiveness for different students, and our results highlight the need for research into targeted interventions to support inclusion in STEM (Schwartz et al., 2016). For example, we found that test anxiety was negatively associated with exam performance across institutions and mediated performance gaps for URMs at inclusive institutions. Future research would benefit from a focus on the impacts of test anxiety on performance of students with underrepresented identities in biology specifically. One study showed that lower measures of incoming preparation were associated with higher anxiety scores, which were associated with lower exam scores (Salehi et al., 2020). While it is unclear why we only observed this mediating effect of anxiety at inclusive institutions, our results suggest that strategies instructors can use to mitigate test anxiety are likely to benefit all students, and especially URM students at some institutions. These include lowering the stakes of exams, expressive writing, or two-stage exams (Ergene, 2003; Ballen et al., 2017a; Rempel et al., 2021). A meta- analysis of 56 interventions that target student behavior (N = 2482) revealed that approaches such as skill-focused strategies (e.g., test-taking skills training), behavioral strategies (e.g., relaxation training), or cognitive strategies (e.g., techniques to change negative thought patterns) are effective at reducing test anxiety. However, less work has brought these strategies into biology or STEM classrooms to test their impacts on performance outcomes. Additionally, the impacts of test anxiety on performance of students with underrepresented identities in biology has not been studied as extensively, to our knowledge.

Conversely, given our varying findings about the effects of ESC, researchers and faculty should be intentional in their implementation of stereotype threat interventions in their institutions. While past work has demonstrated that, for students who have high ESC, the threat or fear of conforming to a stereotype negatively impacts their exam performance (McFarland et al., 2003; Massey and Fischer, 2005), others have suggested that this phenomenon may not be as universal as previously thought (Cromley et al., 2013).

Our results showed that overall, self-reported ESC did not negatively impact academic performance for URM students, and its mediating effects varied across institutions. This suggests stereotype threat did not impact performance of URM students across contexts. In fact, at community colleges, while URM status did not predict student ESC value, the ESC value positively predicted exam score (though the effect size was small). In other words, for all students, regardless of their minority status, self-reported ESC was associated with higher exam scores. We may observe that one’s race/ethnicity does not undermine performance outcomes due to reduced negative stereotypes about race/ethnicity in more-inclusive contexts, such as community colleges. Future research would profit from an explicit focus on community colleges to understand this outcome and what other institutions can learn from equitable teaching practices that are effective at these institutions.

Fostering an inclusive environment serves to reduce the potential effects of stereotype threat (Steele, 2012) and is a leading explanation for positive performance outcomes among minoritized groups (Martin et al., 2017). Murphy and Zirkel (2015) found that students in higher education anticipated a higher sense of belonging in majors where they perceived their group to be represented and that self-reported feelings of belonging in the first weeks of college predicted second-semester grades among students of color, but not White college students. We found that URM status was correlated with higher ESC at the selective institutions in our sample, supporting previous work that stresses the importance of promoting sense of belonging and social fit on campuses or institutions (Massey et al., 2003; Martin et al., 2017). The selective institutions in our sample include the lowest percentage of URM students (Table 2). These low numbers may lead students to feel marginalized and isolated (Hurtado and Ruiz, 2012; Patton and Croom, 2017), experience hostile learning environments (Kelly et al., 2017), and possess few support networks (Palmer et al., 2011). In these predominantly White spaces, we must center interventions that call upon instructors and institutions, rather than the students, to act. Previous research describes many ways to promote inclusion in the classroom, such as employing equitable teaching strategies (Tanner, 2013) and active learning (Eddy and Hogan 2014; Ballen et al. 2017b; Casper et al. 2019; Theobald et al. 2019), learning names (Cooper et al., 2017), using humor (Cooper et al., 2018), including positive role models for all students as examples in course content (Wood et al., 2020; Yonas et al., 2020), conveying that diversity is valued (Purdie-Vaughns et al., 2008), and/or encouraging students to reflect on core personal values (Cohen et al., 2006; Martens et al., 2006; Miyake et al., 2010).

Future research would profit from an understanding of how stereotypes specific to different underrepresented identities emerge in undergraduate settings, how they impact performance outcomes, and how students navigate these situations. Neal-Jackson (2020) conducted a qualitative analysis of Black women at a prestigious predominantly White institution and found that they encountered gendered racial stereotyping that negatively impacted their relationship to the academic community and their prospects for success. This, along with other work (Cho et al., 2013; Morton and Parsons, 2018), also highlights the complexity associated with the multidimensionality of identity and the intersecting nature of those dimensions. We encourage researchers to continue exploring these ideas to determine what strategies increase academic performance and reduce threat for all students. Future work at community colleges and inclusive institutions will help us better understand how to alleviate ESC and inform interventions at selective institutions, where students from marginalized backgrounds are less likely to perceive social fit and more likely to experience stereotype threat.

Limitations

The work presented here is an initial step toward broadening the scope of discipline-based education research across different institutional contexts, though we faced a number of limitations that should be addressed in future work. We acknowledge that our approach here of grouping institutions by broad category of community college, inclusive, and selective suggests students share similar experiences with other students within the same broad institution type rather than across other factors, and this assumption is a limitation of our research. We do not expect that, for example, test anxiety fully mediates performance gaps across all inclusive 4 year-institutions in the United States. Our current sample includes 12 courses in two community colleges, six courses in three inclusive institutions, and 15 courses in three selective institutions. We administered surveys at a single time point, which might not be indicative of levels across the entire semester. Also, other social psychological measures (e.g., sense of belonging; Lewis et al., 2016; Rainey et al., 2018; Binning et al., 2019) and their contribution to student performance should be explored.

Additionally, we did not have access to proxies for incoming preparation (e.g., ACT/Scholastic Aptitude Test [SAT]) from most institutions and were therefore unable to control for this factor in our analyses. Previous research demonstrates that, from the outset, capable but less academically prepared students are not provided with an equal opportunity to excel due to incoming preparation gaps and assessment norms in introductory science classes, resulting in the appearance of “performance gaps” in introductory classes, as we show here (Salehi et al., 2019b). However, when incoming preparation is accounted for, these apparent gaps disappear. While we were not able to control for this, we note that student transcripts, applications to graduate and professional schools, and the internalization of poor performance do not take incoming preparation into account either (Harris et al., 2020). Future research may test how incoming preparation impacts demographic performance gaps across institution types and control for incoming preparation in introductory classes while examining the effect of social psychological factors. While the lack of these data is a limitation, we do not believe this impacts our main conclusion, which points to variation in outcomes across institutional contexts.

Furthermore, one of our analyses included social psychological measures that we obtained from survey data, which represented a subset of all students. This subset may not be representative of the overall sample, as suggested by the URM performance gap in the whole sample as opposed to the gap in the subsample (no URM performance gaps for community colleges in the whole sample, as opposed to the 0.2 SD URM performance gap for community colleges in the subsample; Figure 1). Thus, our analyses focusing on the impacts of these factors on performance may not be representative of the larger pool of students across these institutional contexts. Therefore, the observed differences here in student experiences and academic performance outcomes warrant further study to examine to what extent these differences are results of differences in institutional contexts.

CONCLUSIONS

We call for future research to 1) not exclusively focus on selective institutions and instead include data across different types of institutions; 2) systematically examine characteristics of institutional or classroom contexts that help or hinder equity in undergraduate education; and 3) design targeted psychological interventions and/or effective teaching strategies that fit the characteristics of institutional or classroom contexts. We found some similar trends across institution types, as well as substantial differences with respect to the size of demographic performance gaps and the relationship between social psychological factors and performance. Ongoing research programs and existing evaluative tools designed to assess learning gains and social psychological factors lay a solid foundational groundwork for testing hypotheses.

FOOTNOTES

ACKNOWLEDGMENTS

REFERENCES