IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/21459.html
   My bibliography  Save this paper

From Local to Global: External Validity in a Fertility Natural Experiment

Author

Listed:
  • Rajeev Dehejia
  • Cristian Pop-Eleches
  • Cyrus Samii

Abstract

We study issues related to external validity for treatment effects using over 100 replications of the Angrist and Evans (1998) natural experiment on the effects of sibling sex composition on fertility and labor supply. The replications are based on census data from around the world going back to 1960. We decompose sources of error in predicting treatment effects in external contexts in terms of macro and micro sources of variation. In our empirical setting, we find that macro covariates dominate over micro covariates for reducing errors in predicting treatments, an issue that past studies of external validity have been unable to evaluate. We develop methods for two applications to evidence-based decision-making, including determining where to locate an experiment and whether policy-makers should commission new experiments or rely on an existing evidence base for making a policy decision.

Suggested Citation

  • Rajeev Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2015. "From Local to Global: External Validity in a Fertility Natural Experiment," NBER Working Papers 21459, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:21459
    Note: CH DEV LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w21459.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Cruces, Guillermo & Galiani, Sebastian, 2007. "Fertility and female labor supply in Latin America: New causal evidence," Labour Economics, Elsevier, vol. 14(3), pages 565-573, June.
    2. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Econometric Estimators to Evaluate Social Programs, and to Forecast their Effects in New," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 71, Elsevier.
    3. Robert B. Olsen & Larry L. Orr & Stephen H. Bell & Elizabeth A. Stuart, 2013. "External Validity in Policy Evaluations That Choose Sites Purposively," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 32(1), pages 107-121, January.
    4. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2008. "Nonparametric Tests for Treatment Effect Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 389-405, August.
    5. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    6. Joshua D. Angrist, 2004. "Treatment effect heterogeneity in theory and practice," Economic Journal, Royal Economic Society, vol. 114(494), pages 52-83, March.
    7. Rajeev Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2021. "From Local to Global: External Validity in a Fertility Natural Experiment," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 217-243, January.
    8. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    9. Angrist, Joshua D & Evans, William N, 1998. "Children and Their Parents' Labor Supply: Evidence from Exogenous Variation in Family Size," American Economic Review, American Economic Association, vol. 88(3), pages 450-477, June.
    10. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    11. Daniel Aaronson & Rajeev Dehejia & Andrew Jordan & Cristian Pop-Eleches & Cyrus Samii & Karl Schulze, 2021. "The Effect of Fertility on Mothers’ Labor Supply over the Last Two Centuries [Semiparametric instrumental variables estimation of treatment response models]," The Economic Journal, Royal Economic Society, vol. 131(633), pages 1-32.
    12. Aronow, Peter M. & Carnegie, Allison, 2013. "Beyond LATE: Estimation of the Average Treatment Effect with an Instrumental Variable," Political Analysis, Cambridge University Press, vol. 21(4), pages 492-506.
    13. James Bisbee & Rajeev Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2017. "Local Instruments, Global Extrapolation: External Validity of the Labor Supply-Fertility Local Average Treatment Effect," Journal of Labor Economics, University of Chicago Press, vol. 35(S1), pages 99-147.
    14. Avraham Ebenstein, 2009. "When is the Local Average Treatment Close to the Average?: Evidence from Fertility and Labor Supply," Journal of Human Resources, University of Wisconsin Press, vol. 44(4).
    15. Donald B. Rubin, 1992. "Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation?," Journal of Educational and Behavioral Statistics, , vol. 17(4), pages 363-374, December.
    16. Francine D. Blau & Lawrence M. Kahn, 2003. "Understanding International Differences in the Gender Pay Gap," Journal of Labor Economics, University of Chicago Press, vol. 21(1), pages 106-144, January.
    17. Newey, Whitney K., 1994. "Series Estimation of Regression Functionals," Econometric Theory, Cambridge University Press, vol. 10(1), pages 1-28, March.
    18. Dehejia, Rajeev H, 2003. "Was There a Riverside Miracle? A Hierarchical Framework for Evaluating Programs with Grouped Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(1), pages 1-11, January.
    19. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    20. Erin Hartman & Richard Grieve & Roland Ramsahai & Jasjeet S. Sekhon, 2015. "From sample average treatment effect to population average treatment effect on the treated: combining experimental with observational studies to estimate population treatment effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(3), pages 757-778, June.
    21. Joseph Hotz, V. & Imbens, Guido W. & Mortimer, Julie H., 2005. "Predicting the efficacy of future training programs using past experiences at other locations," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 241-270.
    22. Government of India, 2017. "National Health Policy 2017," Working Papers id:11664, eSocialSciences.
    23. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    24. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    25. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
    26. Deon Filmer & Jed Friedman & Norbert Schady, 2009. "Development, Modernization, and Childbearing: The Role of Family Sex Composition," The World Bank Economic Review, World Bank, vol. 23(3), pages 371-398, October.
    27. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    28. Joshua Angrist & Ivan Fernandez-Val, 2010. "ExtrapoLATE-ing: External Validity and Overidentification in the LATE Framework," NBER Working Papers 16566, National Bureau of Economic Research, Inc.
    29. T. D. Stanley, 2001. "Wheat from Chaff: Meta-analysis as Quantitative Literature Review," Journal of Economic Perspectives, American Economic Association, vol. 15(3), pages 131-150, Summer.
    30. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    2. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    3. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    4. Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
    5. Iva Trako, 2018. "Fertility and Parental Labor-Force Participation: New Evidence from a Developing Country in the Balkans," PSE Working Papers halshs-01828471, HAL.
    6. Bo, Hao & Galiani, Sebastian, 2021. "Assessing external validity," Research in Economics, Elsevier, vol. 75(3), pages 274-285.
    7. Denis Fougère & Nicolas Jacquemet, 2019. "Causal Inference and Impact Evaluation," Economie et Statistique / Economics and Statistics, Institut National de la Statistique et des Etudes Economiques (INSEE), issue 510-511-5, pages 181-200.
    8. Iva Trako, 2018. "Fertility and Parental Labor-Force Participation: New Evidence from a Developing Country in the Balkans," Working Papers halshs-01828471, HAL.
    9. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    10. Chu, Yu-Wei Luke & Cuffe, Harold E & Doan, Nguyen, 2021. "Motherhood Employment Penalty and Gender Wage Gap Across Countries: 1990–2010," Working Paper Series 9446, Victoria University of Wellington, School of Economics and Finance.
    11. Anh P. Ngo, 2020. "Effects of Vietnam’s two-child policy on fertility, son preference, and female labor supply," Journal of Population Economics, Springer;European Society for Population Economics, vol. 33(3), pages 751-794, July.
    12. Isaiah Andrews & Emily Oster, 2017. "A Simple Approximation for Evaluating External Validity Bias," NBER Working Papers 23826, National Bureau of Economic Research, Inc.
    13. Chu, Yu-Wei Luke & Cuffe, Harold E. & Doan, Nguyen, 2020. "Motherhood Employment Penalty and Gender Wage Gap Across Countries: 1990–2010," MPRA Paper 99866, University Library of Munich, Germany.
    14. Andrews, Isaiah & Oster, Emily, 2019. "A simple approximation for evaluating external validity bias," Economics Letters, Elsevier, vol. 178(C), pages 58-62.
    15. Daido Kido, 2022. "Distributionally Robust Policy Learning with Wasserstein Distance," Papers 2205.04637, arXiv.org, revised Aug 2022.
    16. Nikolova, Elena & Polansky, Jakub, 2022. "Children and Female Employment in Mongolia," GLO Discussion Paper Series 1015, Global Labor Organization (GLO).
    17. Elena Nikolova & Jakub Polansky, 2022. "Children and Female Employment in Mongolia," Discussion Papers 61, Central European Labour Studies Institute (CELSI).
    18. Rahul Singh & Liyuan Xu & Arthur Gretton, 2020. "Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves," Papers 2010.04855, arXiv.org, revised Oct 2022.
    19. Nikolova, Elena & Polansky, Jakub, 2022. "Children and female employment in Mongolia," Economic Systems, Elsevier, vol. 46(3).
    20. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).

    More about this item

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • F63 - International Economics - - Economic Impacts of Globalization - - - Economic Development
    • J13 - Labor and Demographic Economics - - Demographic Economics - - - Fertility; Family Planning; Child Care; Children; Youth

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:21459. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.