IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_5493.html
   My bibliography  Save this paper

From Local to Global: External Validity in a Fertility Natural Experiment

Author

Listed:
  • Rajeev H. Dehejia
  • Cristian Pop-Eleches
  • Cyrus Samii

Abstract

Experimental evidence on a range of interventions in developing countries is accumulating rapidly. Is it possible to extrapolate from an experimental evidence base to other locations of policy interest (from “reference” to “target” sites)? And which factors determine the accuracy of such an extrapolation? We investigate applying the Angrist and Evans (1998) natural experiment (the effect of boy-boy or girl-girl as the first two children on incremental fertility and mothers’ labor force participation) to data from International IPUMS on 166 country-year censuses. We define the external validity function with extrapolation error depending on covariate differences between reference and target locations, and find that smaller differences in geography, education, calendar year, and mothers’ labor force participation lead to lower extrapolation error. As experimental evidence accumulates, out-of-sample extrapolation error does not systematically approach zero if the available evidence base is naïvely extrapolated, but does if the external validity function is used to select the most appropriate reference context for a given target (although absolute error remains meaningful relative to the magnitude of the treatment effect). We also investigate where to locate experiments and the decision problem associated with extrapolating from existing evidence rather than running a new experiment at a target site.

Suggested Citation

  • Rajeev H. Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2015. "From Local to Global: External Validity in a Fertility Natural Experiment," CESifo Working Paper Series 5493, CESifo Group Munich.
  • Handle: RePEc:ces:ceswps:_5493
    as

    Download full text from publisher

    File URL: https://www.cesifo-group.de/DocDL/cesifo1_wp5493.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Cruces, Guillermo & Galiani, Sebastian, 2007. "Fertility and female labor supply in Latin America: New causal evidence," Labour Economics, Elsevier, vol. 14(3), pages 565-573, June.
    2. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    3. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    4. Francine D. Blau & Lawrence M. Kahn, 2003. "Understanding International Differences in the Gender Pay Gap," Journal of Labor Economics, University of Chicago Press, vol. 21(1), pages 106-144, January.
    5. Joshua D. Angrist, 2004. "Treatment effect heterogeneity in theory and practice," Economic Journal, Royal Economic Society, vol. 114(494), pages 52-83, March.
    6. Newey, Whitney K., 1994. "Series Estimation of Regression Functionals," Econometric Theory, Cambridge University Press, vol. 10(1), pages 1-28, March.
    7. Dehejia, Rajeev H, 2003. "Was There a Riverside Miracle? A Hierarchical Framework for Evaluating Programs with Grouped Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(1), pages 1-11, January.
    8. Dehejia, Rajeev & Pop-Eleches, Cristian & Samii, Cyrus, 2015. "From Local to Global: External Validity in a Fertility Natural Experiment," IZA Discussion Papers 9300, Institute of Labor Economics (IZA).
    9. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    10. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    11. Erin Hartman & Richard Grieve & Roland Ramsahai & Jasjeet S. Sekhon, 2015. "From sample average treatment effect to population average treatment effect on the treated: combining experimental with observational studies to estimate population treatment effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(3), pages 757-778, June.
    12. Joseph Hotz, V. & Imbens, Guido W. & Mortimer, Julie H., 2005. "Predicting the efficacy of future training programs using past experiences at other locations," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 241-270.
    13. repec:ucp:jlabec:doi:10.1086/691280 is not listed on IDEAS
    14. Angrist, Joshua D & Evans, William N, 1998. "Children and Their Parents' Labor Supply: Evidence from Exogenous Variation in Family Size," American Economic Review, American Economic Association, vol. 88(3), pages 450-477, June.
    15. Aaronson, Daniel & Dehejia, Rajeev & Jordon, Andrew & Pop-Eleches, Cristian & Samii, Cyrus & Schultze, Karl, 2017. "The Effect of Fertility on Mothers’ Labor Supply over the Last Two Centuries," MPRA Paper 76768, University Library of Munich, Germany.
    16. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    17. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    18. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
    19. Aronow, Peter M. & Carnegie, Allison, 2013. "Beyond LATE: Estimation of the Average Treatment Effect with an Instrumental Variable," Political Analysis, Cambridge University Press, vol. 21(4), pages 492-506.
    20. Deon Filmer & Jed Friedman & Norbert Schady, 2009. "Development, Modernization, and Childbearing: The Role of Family Sex Composition," World Bank Economic Review, World Bank Group, vol. 23(3), pages 371-398, October.
    21. James Bisbee & Rajeev Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2017. "Local Instruments, Global Extrapolation: External Validity of the Labor Supply-Fertility Local Average Treatment Effect," Journal of Labor Economics, University of Chicago Press, vol. 35(S1), pages 99-147.
    22. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    23. Joshua Angrist & Ivan Fernandez-Val, 2010. "ExtrapoLATE-ing: External Validity and Overidentification in the LATE Framework," NBER Working Papers 16566, National Bureau of Economic Research, Inc.
    24. Avraham Ebenstein, 2009. "When is the Local Average Treatment Close to the Average?: Evidence from Fertility and Labor Supply," Journal of Human Resources, University of Wisconsin Press, vol. 44(4).
    25. T. D. Stanley, 2001. "Wheat from Chaff: Meta-analysis as Quantitative Literature Review," Journal of Economic Perspectives, American Economic Association, vol. 15(3), pages 131-150, Summer.
    26. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    27. Donald B. Rubin, 1992. "Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation?," Journal of Educational and Behavioral Statistics, , vol. 17(4), pages 363-374, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lucia Corno & Nicole Hildebrandt & Alessandra Voena, 2017. "Age of Marriage, Weather Shocks, and the Direction of Marriage Payments," Working Papers 2017-055, Human Capital and Economic Opportunity Working Group.
    2. Rajeev Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2015. "From Local to Global: External Validity in a Fertility Natural Experiment," NBER Working Papers 21459, National Bureau of Economic Research, Inc.
    3. Cristina Corduneanu-Huci & Michael T. Dorsch & Paul Maarek, 2017. "Learning to constrain: Political competition and randomized controlled trials in development," THEMA Working Papers 2017-24, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    4. James Bisbee & Rajeev Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2017. "Local Instruments, Global Extrapolation: External Validity of the Labor Supply-Fertility Local Average Treatment Effect," Journal of Labor Economics, University of Chicago Press, vol. 35(S1), pages 99-147.
    5. Andor, Mark A. & Fels, Katja M. & Renz, Jan & Rzepka, Sylvi, 2018. "Do planning prompts increase educational success? Evidence from randomized controlled trials in MOOCs," Ruhr Economic Papers 790, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Andor, Mark Andreas & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2017. "Social norms and energy conservation beyond the US," Ruhr Economic Papers 714, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Meager, Rachael, 2019. "Understanding the average impact of microcredit expansions: a Bayesian hierarchical analysis of seven randomized experiments," LSE Research Online Documents on Economics 88190, London School of Economics and Political Science, LSE Library.
    8. Rosenzweig, Mark & Udry, Christopher, 2016. "External Validity in a Stochastic World," Center Discussion Papers 242440, Yale University, Economic Growth Center.
    9. Iva Trako, 2018. "Fertility and Parental Labor-Force Participation: New Evidence from a Developing Country in the Balkans," PSE Working Papers halshs-01828471, HAL.
    10. Isaiah Andrews & Emily Oster, 2017. "Weighting for External Validity," NBER Working Papers 23826, National Bureau of Economic Research, Inc.
    11. Guber, Raphael, 2018. "Instrument Validity Tests with Causal Trees: With an Application to the Same-sex Instrument," MEA discussion paper series 201805, Munich Center for the Economics of Aging (MEA) at the Max Planck Institute for Social Law and Social Policy.
    12. Amy Ellen Schwartz & Douglas Almond & Ajin Lee, 2016. "Retention Heterogeneity in New York City Schools," Center for Policy Research Working Papers 198, Center for Policy Research, Maxwell School, Syracuse University.
    13. repec:bla:jecsur:v:32:y:2018:i:2:p:518-540 is not listed on IDEAS

    More about this item

    Keywords

    external validity; fertility; labor supply; experiments;

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D04 - Microeconomics - - General - - - Microeconomic Policy: Formulation; Implementation; Evaluation
    • D10 - Microeconomics - - Household Behavior - - - General
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_5493. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Klaus Wohlrabe). General contact details of provider: http://edirc.repec.org/data/cesifde.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.