IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_5493.html
   My bibliography  Save this paper

From Local to Global: External Validity in a Fertility Natural Experiment

Author

Listed:
  • Rajeev H. Dehejia
  • Cristian Pop-Eleches
  • Cyrus Samii

Abstract

Experimental evidence on a range of interventions in developing countries is accumulating rapidly. Is it possible to extrapolate from an experimental evidence base to other locations of policy interest (from “reference” to “target” sites)? And which factors determine the accuracy of such an extrapolation? We investigate applying the Angrist and Evans (1998) natural experiment (the effect of boy-boy or girl-girl as the first two children on incremental fertility and mothers’ labor force participation) to data from International IPUMS on 166 country-year censuses. We define the external validity function with extrapolation error depending on covariate differences between reference and target locations, and find that smaller differences in geography, education, calendar year, and mothers’ labor force participation lead to lower extrapolation error. As experimental evidence accumulates, out-of-sample extrapolation error does not systematically approach zero if the available evidence base is naïvely extrapolated, but does if the external validity function is used to select the most appropriate reference context for a given target (although absolute error remains meaningful relative to the magnitude of the treatment effect). We also investigate where to locate experiments and the decision problem associated with extrapolating from existing evidence rather than running a new experiment at a target site.

Suggested Citation

  • Rajeev H. Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2015. "From Local to Global: External Validity in a Fertility Natural Experiment," CESifo Working Paper Series 5493, CESifo Group Munich.
  • Handle: RePEc:ces:ceswps:_5493
    as

    Download full text from publisher

    File URL: http://www.cesifo-group.de/DocDL/cesifo1_wp5493.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    2. Francine D. Blau & Lawrence M. Kahn, 2003. "Understanding International Differences in the Gender Pay Gap," Journal of Labor Economics, University of Chicago Press, vol. 21(1), pages 106-144, January.
    3. Joshua D. Angrist, 2004. "Treatment effect heterogeneity in theory and practice," Economic Journal, Royal Economic Society, vol. 114(494), pages 52-83, March.
    4. Dehejia, Rajeev H, 2003. "Was There a Riverside Miracle? A Hierarchical Framework for Evaluating Programs with Grouped Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(1), pages 1-11, January.
    5. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    6. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    7. Joseph Hotz, V. & Imbens, Guido W. & Mortimer, Julie H., 2005. "Predicting the efficacy of future training programs using past experiences at other locations," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 241-270.
    8. Angrist, Joshua D & Evans, William N, 1998. "Children and Their Parents' Labor Supply: Evidence from Exogenous Variation in Family Size," American Economic Review, American Economic Association, vol. 88(3), pages 450-477, June.
    9. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    10. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    11. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
    12. Aronow, Peter M. & Carnegie, Allison, 2013. "Beyond LATE: Estimation of the Average Treatment Effect with an Instrumental Variable," Political Analysis, Cambridge University Press, vol. 21(04), pages 492-506, September.
    13. Joshua Angrist & Ivan Fernandez-Val, 2010. "ExtrapoLATE-ing: External Validity and Overidentification in the LATE Framework," NBER Working Papers 16566, National Bureau of Economic Research, Inc.
    14. Avraham Ebenstein, 2009. "When is the Local Average Treatment Close to the Average?: Evidence from Fertility and Labor Supply," Journal of Human Resources, University of Wisconsin Press, vol. 44(4).
    15. T. D. Stanley, 2001. "Wheat from Chaff: Meta-analysis as Quantitative Literature Review," Journal of Economic Perspectives, American Economic Association, vol. 15(3), pages 131-150, Summer.
    16. Donald B. Rubin, 1992. "Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation?," Journal of Educational and Behavioral Statistics, , vol. 17(4), pages 363-374, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lucia Corno & Nicole Hildebrandt & Alessandra Voena, 2017. "Age of Marriage, Weather Shocks, and the Direction of Marriage Payments," Working Papers 2017-055, Human Capital and Economic Opportunity Working Group.
    2. Cristina Corduneanu-Huci & Michael T. Dorsch & Paul Maarek, 2017. "Learning to constrain: Political competition and randomized controlled trials in development," THEMA Working Papers 2017-24, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    3. James Bisbee & Rajeev Dehejia & Cristian Pop-Eleches & Cyrus Samii, 2017. "Local Instruments, Global Extrapolation: External Validity of the Labor Supply-Fertility Local Average Treatment Effect," Journal of Labor Economics, University of Chicago Press, vol. 35(S1), pages 99-147.
    4. Andor, Mark Andreas & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2017. "Social norms and energy conservation beyond the US," Ruhr Economic Papers 714, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    5. Mark Rosenzweig & Christopher Udry, 2016. "External Validity in a Stochastic World," Working Papers id:11134, eSocialSciences.
    6. Isaiah Andrews & Emily Oster, 2017. "Weighting for External Validity," NBER Working Papers 23826, National Bureau of Economic Research, Inc.
    7. Amy Ellen Schwartz & Douglas Almond & Ajin Lee, 2016. "Retention Heterogeneity in New York City Schools," Center for Policy Research Working Papers 198, Center for Policy Research, Maxwell School, Syracuse University.

    More about this item

    Keywords

    external validity; fertility; labor supply; experiments;

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D04 - Microeconomics - - General - - - Microeconomic Policy: Formulation; Implementation; Evaluation
    • D10 - Microeconomics - - Household Behavior - - - General
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_5493. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Klaus Wohlrabe). General contact details of provider: http://edirc.repec.org/data/cesifde.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.