Abstract
Individuals differ not only in background characteristics, often called “pretreatment heterogeneity,” but also in how they respond to a particular treatment, event, or intervention. A principal interaction of interest for questions of selection into treatment and causal inference in the social sciences is between the treatment and the propensity of treatment. Although the importance of “treatment-effect heterogeneity,” so defined, has been widely recognized in the causal inference literature, empirical quantitative social science research has not fully absorbed these lessons. In this chapter, we describe key estimation strategies for the study of heterogeneous treatment effects; we discuss recent research that attends to causal effect heterogeneity, with a focus on the study of effects of education, and what we gain from such attention; and we demonstrate the methods with an example of the effects of college on civic participation. The primary goal of this chapter is to encourage researchers to routinely examine treatment-effect heterogeneity with the same rigor they devote to pretreatment heterogeneity.
This research made use of facilities and resources at the California Center for Population Research, UCLA, which receives core support from the National Institute of Child Health and Human Development, Grant R24HD041022. Simon Thomas was supported by a UCLA Graduate Summer Research Mentorship program, the Institute for Research on Labor and Employment, and a pre-doctoral advanced quantitative methodology training grant (R305B080016) awarded to UCLA by the Institute of Education Sciences of the U.S. Department of Education. This research was conducted with restricted access to the Bureau of Labor Statistics (BLS) data. The views expressed here do not necessarily reflect the views of the BLS. We thank Yu Xie and Ben Jann for their contributions to related work, and Stephen Morgan and Judea Pearl for comments and suggestions. The ideas expressed herein are those of the authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Extending to multivalued treatments with j values, the observed outcome variable contains only 1/J of the information in the potential outcome random variable, rather than ½ in the binary treatment setup. In other words, the proportion of unobservable counterfactual states increases as the number of treatment values j increases, such that we have a matrix of potential outcomes with j 2 cells, only the diagonal of which are observed (see Morgan and Winship 2007 for a detailed discussion).
- 2.
Repeated observations of units of analysis can be used in fixed effects models to control for time-invariant unobserved properties of units, potentially increasing the plausibility of the assumption.
- 3.
- 4.
Bjorklund and Moffitt (1987) introduced the concept of the MTE and showed that the model was observationally equivalent to the switching regression model. See Carniero et al. (2011) for a description of related parameters, including the policy relevant treatment effect (PRTE) and the marginal policy relevant treatment effect (MPRTE). See Xie (2011) for a description of the incremental treatment effect (ITE), which is the average treatment effect for incremental units when a unit’s treatment status changes from d = 0 to d = 1 and when p increases from p 1 to p 2.
- 5.
- 6.
Low inducement is a major limitation if the instrument is so weak as to have very little impact on the treatment of interest. We revisit this issue in the empirical example section.
- 7.
We assume, for simplicity, that there are no “defiers,” that is, those individuals who would always do the opposite of treatment assignment.
- 8.
See Zhou and Xie (2011) for further discussion of the difference between propensity score-based and IV approaches.
- 9.
A recent study likewise suggests that economic returns to attending highly selective colleges are indistinguishable from zero among the full sample when adjustments for unobserved student characteristics are incorporated, while returns among Black and Hispanic students and students from disadvantaged families remain large (Dale and Krueger 2011). Hout (2012) reviews additional studies with corroborating results.
- 10.
Brand and Davis (2011) combine the stratification-multilevel approach with discrete-time event-history models. Xie et al. (2011) also use the matching-smoothing and smoothing-differencing methods for the effects of college on fertility and find comparable results to those using stratification-multilevel. This is due to a largely linear pattern in effects of college on fertility across the propensity for college.
- 11.
We impute missing values for our set of pretreatment covariates based on all other covariates. Most variables have 1–2% missing values. Only two variables are missing for more than 5% of the sample: parents’ income and high school college-preparatory program. We include an imputation indicator in our models.
- 12.
- 13.
- 14.
For the kth covariate in the jth stratum, we estimate the standardized mean covariate difference to quantify the balance between the treatment and the control groups for each covariate X (Morgan and Winship 2007):
$$ {B_{k,j }}=\frac{{|{{\bar{X}}_{k,j,D=1 }}-{{\bar{X}}_{k,j,D=0 }}|}}{{\sqrt{{\frac{{S_{k,j,D=1}^2+S_{k,j,D=0}^2}}{2}}}}} $$where \( \bar{X} \) is the sample mean and S 2 is the sample variance of the kth covariate in the jth stratum for the treated and control groups as indexed by D = (1,0). The standardized difference is larger in some strata than in others for selected covariates.
- 15.
We also fit alternative stratification-multilevel models in which we include quadratic terms in level-2. These terms were not statistically significant, and we do not present them here.
References
Abadie, A., & Imbens, G. W. (2006). Large sample properties of matching estimators for average treatment effects. Econometrica, 74(1), 235–267.
Angrist, J. D. (1998). Estimating the labor market impact on voluntary military service using social security date on military applicants. Econometrica, 66, 249–288.
Angrist, J. D. (2001). Estimation of limited dependent variable models with dummy endogenous regressors: Simple strategies for empirical practice. Journal of Business and Economic Statistics, 19, 2–16.
Angrist, J. D., & Krueger, A. (1999). Empirical strategies in labor economics. In O. Ashenfelter & D. Card (Eds.), Handbook of labor economics (Vol. 3A, pp. 1277–1366). Amsterdam: Elsevier.
Angrist, J. D., & Pischke, J.-S. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton: Princeton University Press.
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.
Becker, S., & Ichino, A. (2002). Estimation of average treatment effects based on propensity scores. The Stata Journal, 2(4), 358–377.
Bjorklund, A., & Moffitt, R. (1987). The estimation of wage gains and welfare gains in self-selection models. The Review of Economics and Statistics, 69(1), 42–49.
Bound, J., Jaeger, D. A., & Baker, R. M. (1995). Problems with instrumental variables estimation when the correlation between the instruments and the endogenous explanatory variable is weak. Journal of the American Statistical Association, 90(430), 443–450.
Brand, J. E. (2010). Civic returns to higher education: A note on heterogeneous effects. Social Forces, 89(2), 417–433.
Brand, J. E., & Burgard, S. A. (2008). Job displacement and social participation over the life course: Findings for a cohort of joiners. Social Forces, 87(1), 211–242.
Brand, J. E., & Davis, D. (2011). The impact of college education on fertility: Evidence for heterogeneous effects. Demography, 48(3), 863–887.
Brand, J. E., & Halaby, C. N. (2006). Regression and matching estimates of the effects of elite college attendance on educational and career achievement. Social Science Research, 35, 749–770.
Brand, J., & Simon Thomas, J. (2013). Job displacement among single mothers: Effects on children’s outcomes in young adulthood (Working Paper).
Brand, J. E., & Xie, Y. (2010). Who benefits most from college? Evidence for negative selection in heterogeneous economic returns to higher education. American Sociological Review, 75(2), 273–302.
Brand, J. E., Pfeffer, F., & Goldrick-Rab, S. (2012). Interpreting Community College effects in the presence of heterogeneity and complex counterfactuals (Working Paper PWP-2012-004). California Center for Population Research.
Card, D. (2001). Estimating the return to schooling: Progress on some persistent econometric problems. Econometrica, 69, 1127–1160.
Carniero, P., Heckman, J. J., & Vytlacil, E. (2011). Estimating marginal returns to education. American Economic Review, 101, 2754–2781.
Dale, S., & Krueger, A. B. (2011). Estimating the return to college selectivity over the career using administrative earning data (Working Paper). Princeton University.
DiPrete, T., & Gangl, M. (2004). Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociological Methodology, 34, 271–310.
Elwert, F., & Winship, C. (2010). Effect heterogeneity and bias in main-effects-only regression models. In R. Dechter, H. Geffner, & J. Y. Halpern (Eds.), Heuristics, probability, and causality: A tribute to Judea Pearl (pp. 327–336). London: College Publications.
Gangl, M. (2010). Causal inference in sociological research. Annual Review of Sociology, 36, 21–47.
Halaby, C. N. (2004). Panel models in sociological research: Theory into practice. Annual Review of Sociology, 30, 507–544.
Harding, D. J. (2003). Counterfactual models of neighborhood effects: The effect of neighborhood poverty on dropping out and teenage pregnancy. The American Journal of Sociology, 109(3), 676–719.
Heckman, J. J. (1978). Dummy endogenous variables in a simultaneous equation system. Econometrica, 46(4), 931–959.
Heckman, J. J. (2001). Micro data, heterogeneity, and the evaluation of public policy: Nobel lecture. Journal of Political Economy, 109, 673–748.
Heckman, J. J. (2005). The scientific model of causality. In R. M. Stolzenberg (Ed.), Sociological methodology (Vol. 35, pp. 1–98). Boston: Blackwell Publishing.
Heckman, J. J., & Robb, R., Jr. (1985). Alternative methods for evaluating the impact of interventions. Journal of Econometrics, 30, 239–267.
Heckman, J., Urzua, S., & Vytlacil, E. (2006). Understanding instrumental variables in models with essential heterogeneity. The Review of Economics and Statistics, 88, 389–432.
Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Associations, 81(396), 945–960.
Hout, M. (2012). Social and economic returns to college education in the United States. Annual Review of Sociology, 38, 379–400.
Imbens, G. W., & Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica, 62(2), 467–475.
Manski, C. F. (1995). Identification problems in the social sciences. Boston: Harvard University Press.
Moffitt, R. (2008). Estimating average and marginal treatment effects in heterogeneous populations (Working Paper).
Morgan, S. (2001). Counterfactuals, causal effect heterogeneity, and the Catholic school effect on learning. Sociology of Education, 74, 341–374.
Morgan, S., & Harding, D. (2006). Matching estimators of causal effects: Prospects and pitfalls in theory and practice. Sociological Methods and Research, 35(1), 3–60.
Morgan, S. L., & Todd, J. J. (2008). A diagnostic routine for the detection of consequential heterogeneity of causal effects with a demonstration from school effects research. Sociological Methodology, 38, 231–281.
Morgan, S., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. New York: Cambridge University Press.
Morgan, S., & Winship, C. (2012). Brining context and variability back into causal analysis. In H. Kincaid (Ed.), Oxford handbook of the philosophy of the social sciences (pp. 319–354). New York: Oxford University Press.
Musick, K., Brand, J. E., & Davis, D. (2012). Variation in the relationship between education and marriage: Marriage market mismatch? Journal of Marriage and Family, 74, 53–69.
Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd ed.). Cambridge: Cambridge University Press.
Putnam, R. (2000). Bowling Alone: The collapse and revival of American Community. New York: Simon & Schuster.
Quandt, R. E. (1972). A new approach to estimating switching regressions. Journal of the American Statistical Association, 67(338), 306–310.
Robins, J. M., Hernán, M. A., & Brumback, B. (2000). Marginal structural models and causal inference in epidemiology. Epidemiology, 11, 550–560.
Rosenbaum, P. (2002). Observational studies. New York: Springer.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.
Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516–524.
Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 685–701.
Tsai, S.-L., & Xie, Y. (2008). Changes in earnings returns to higher education in Taiwan since the 1990s. Population Review, 47(1), 1–20.
Willis, R. J., & Rosen, S. (1979). Education and self-selection. Journal of Political Economy, 87(5, part 2), S7–S36.
Winship, C., & Morgan, S. L. (1999). The estimation of causal effects from observational data. Annual Review of Sociology, 25, 659–706.
Xie, Y. (2007). Otis Dudley Duncan’s legacy: The demographic approach to quantitative reasoning in social science. Research in Social Stratification and Mobility, 25, 141–156.
Xie, Y. (2011). Population heterogeneity and causal inference (University of Michigan Population Studies Center Research Report 11–731).
Xie, Y., & Wu, X. (2005). Market premium, social process, and statisticism. American Sociological Review, 70, 865–870.
Xie, Y., Brand, J. E., & Jann, B. (2012). Estimating heterogeneous treatment effects with observational data. Sociological Methodology, 42(1), 314–347.
Zhou, X., & Xie, Y. (2011). Propensity-score-based methods versus MTE-based methods in causal inference (Population Studies Center, University of Michigan Working Paper Series 11–747).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendixes
Appendixes
Appendix A
Appendix B
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Brand, J.E., Thomas, J.S. (2013). Causal Effect Heterogeneity. In: Morgan, S. (eds) Handbook of Causal Analysis for Social Research. Handbooks of Sociology and Social Research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-6094-3_11
Download citation
DOI: https://doi.org/10.1007/978-94-007-6094-3_11
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-6093-6
Online ISBN: 978-94-007-6094-3
eBook Packages: Humanities, Social Sciences and LawSocial Sciences (R0)