IDEAS home Printed from https://ideas.repec.org/p/iab/iabdpa/202029.html
   My bibliography  Save this paper

TippingSens: An R Shiny Application to Facilitate Sensitivity Analysis for Causal Inference Under Confounding

Author

Listed:
  • Haensch, Anna-Carolina

    (Institute for Employment Research (IAB), Nuremberg, Germany ; GESIS Insitute for the Social Sciences)

  • Drechsler, Jörg

    (Institute for Employment Research (IAB), Nuremberg, Germany)

  • Bernhard, Sarah

    (Institute for Employment Research (IAB), Nuremberg, Germany)

Abstract

"Most strategies for causal inference based on quasi-experimental or observational data critically rely on the assumption of unconfoundedness. If this assumption is suspect, sensitivity analysis can be a viable tool to evaluate the impact of confounding on the analysis of interest. One of the earliest proposals for such a sensitivity analysis was suggested by Rosenbaum/ Rubin (1983). However, while it is straightforward to obtain estimates for the causal effect under specific assumptions regarding an unobserved confounder, conducting a full sensitivity analysis based on a range of parameter settings is unwieldy based on the simple forking tables which Rosenbaum and Rubin used. To tackle the multiple parameter problem of the Rosenbaum-Rubin approach, we present an interactive R Shiny application called TippingSens, which visualizes the impact of various parameter settings on the estimated causal effect. Borrowing from the literature on tipping point analysis, the flexible app facilitates manipulating all parameters simultaneously. We demonstrate the usefulness of our app by conducting a sensitivity analysis for a quasi-experiment measuring the effect of vocational training programs on unemployed men. The online supplement accompanying this paper provides a step-by-step introduction to the app using the original illustrative example from Rosenbaum/Rubin (1983)." (Author's abstract, IAB-Doku) ((en))

Suggested Citation

  • Haensch, Anna-Carolina & Drechsler, Jörg & Bernhard, Sarah, 2020. "TippingSens: An R Shiny Application to Facilitate Sensitivity Analysis for Causal Inference Under Confounding," IAB-Discussion Paper 202029, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
  • Handle: RePEc:iab:iabdpa:202029
    as

    Download full text from publisher

    File URL: https://doku.iab.de/discussionpapers/2020/dp2920.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    2. Manski, Charles F, 1990. "Nonparametric Bounds on Treatment Effects," American Economic Review, American Economic Association, vol. 80(2), pages 319-323, May.
    3. Marion Devaux & Franco Sassi, 2015. "The Labour Market Impacts of Obesity, Smoking, Alcohol Use and Related Chronic Diseases," OECD Health Working Papers 86, OECD Publishing.
    4. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    5. Mullahy, John & Sindelar, Jody, 1996. "Employment, unemployment, and problem drinking," Journal of Health Economics, Elsevier, vol. 15(4), pages 409-434, August.
    6. Dorner, Matthias & Heining, Jörg & Jacobebbinghaus, Peter & Seth, Stefan, 2010. "Sample of Integrated Labour Market Biographies (SIAB) 1975-2008," FDZ Methodenreport 201009_en, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    7. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    8. Patrick Bayer & Stephen L. Ross & Giorgio Topa, 2008. "Place of Work and Place of Residence: Informal Hiring Networks and Labor Market Outcomes," Journal of Political Economy, University of Chicago Press, vol. 116(6), pages 1150-1196, December.
    9. Richard Blundell & Amanda Gosling & Hidehiko Ichimura & Costas Meghir, 2007. "Changes in the Distribution of Male and Female Wages Accounting for Employment Composition Using Bounds," Econometrica, Econometric Society, vol. 75(2), pages 323-363, March.
    10. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    11. Changhui Kang, 2011. "Family Size and Educational Investments in Children: Evidence from Private Tutoring Expenditures in South Korea," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 73(1), pages 59-78, February.
    12. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    13. Paul R. Rosenbaum, 2007. "Sensitivity Analysis for m-Estimates, Tests, and Confidence Intervals in Matched Observational Studies," Biometrics, The International Biometric Society, vol. 63(2), pages 456-464, June.
    14. Sascha O. Becker & Marco Caliendo, 2007. "Sensitivity analysis for average treatment effects," Stata Journal, StataCorp LP, vol. 7(1), pages 71-83, February.
    15. Stefanie Hof, 2014. "Does Private Tutoring Work? The Effectiveness of Private Tutoring: A Nonparametric Bounds Analysis," Economics of Education Working Paper Series 0096, University of Zurich, Department of Business Administration (IBW).
    16. Lechner, Michael, 1999. "Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification," Journal of Business & Economic Statistics, American Statistical Association, vol. 17(1), pages 74-90, January.
    17. Petra E. Todd & Jeffrey A. Smith, 2001. "Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods," American Economic Review, American Economic Association, vol. 91(2), pages 112-118, May.
    18. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    19. Charles F. Manski & John V. Pepper, 2000. "Monotone Instrumental Variables, with an Application to the Returns to Schooling," Econometrica, Econometric Society, vol. 68(4), pages 997-1012, July.
    20. Fredriksson, Peter & Johansson, Per, 2008. "Dynamic Treatment Assignment," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 435-445.
    21. Paul R. Rosenbaum, 1999. "Reduced Sensitivity to Hidden Bias at Upper Quantiles in Observational Studies with Dilated Treatment Effects," Biometrics, The International Biometric Society, vol. 55(2), pages 560-564, June.
    22. James J. Heckman & Jora Stixrud & Sergio Urzua, 2006. "The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 411-482, July.
    23. Rosenbaum, Paul R., 2010. "Design Sensitivity and Efficiency in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 692-702.
    24. Matthias Dorner & Jörg Heining & Peter Jacobebbinghaus & Stefan Seth, 2010. "The Sample of Integrated Labour Market Biographies," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 130(4), pages 599-608.
    25. Ziggy MacDonald & Michael A. Shields, 2004. "Does problem drinking affect employment? Evidence from England," Health Economics, John Wiley & Sons, Ltd., vol. 13(2), pages 139-155, February.
    26. Dorner, Matthias & Heining, Jörg & Jacobebbinghaus, Peter & Seth, Stefan, 2010. "Sample of Integrated Labour Market Biographies (SIAB) 1975-2008," FDZ Datenreport. Documentation on Labour Market Data 201001_en, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    27. Stefanie Hof, 2014. "Does private tutoring work? The effectiveness of private tutoring: a nonparametric bounds analysis," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 347-366, August.
    28. Gerrit Mueller & Erik Plug, 2006. "Estimating the Effect of Personality on Male and Female Earnings," ILR Review, Cornell University, ILR School, vol. 60(1), pages 3-22, October.
    29. Pannenberg, Markus, 2010. "Risk attitudes and reservation wages of unemployed workers: Evidence from panel data," Economics Letters, Elsevier, vol. 106(3), pages 223-226, March.
    30. Charles F. Manski & John V. Pepper, 2009. "More on monotone instrumental variables," Econometrics Journal, Royal Economic Society, vol. 12(s1), pages 200-216, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    3. Tobias Brändle & Lukas Fervers, 2021. "Give it Another Try: What are the Effects of a Job Creation Scheme Especially Designed for Hard-to-Place Workers?," Journal of Labor Research, Springer, vol. 42(3), pages 382-417, December.
    4. Jorge Rodríguez & Fernando Saltiel & Sergio Urzúa, 2022. "Dynamic treatment effects of job training," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 242-269, March.
    5. Hagen, Tobias, 2016. "Econometric evaluation of a placement coaching program for recipients of disability insurance benefits in Switzerland," Working Paper Series 10, Frankfurt University of Applied Sciences, Faculty of Business and Law.
    6. Thomas Kruppe & Julia Lang, 2018. "Labour market effects of retraining for the unemployed: the role of occupations," Applied Economics, Taylor & Francis Journals, vol. 50(14), pages 1578-1600, March.
    7. Hagen, Tobias, 2016. "Econometric Evaluation of a Placement Coaching Program for Recipients of Disability Insurance Benefits in Switzerland," VfS Annual Conference 2016 (Augsburg): Demographic Change 145736, Verein für Socialpolitik / German Economic Association.
    8. Sarah Bernhard & Thomas Kruppe, 2012. "Effectiveness of Further Vocational Training in Germany – Empirical Findings for Persons Receiving Means-tested Unemployment Benefits," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 132(4), pages 501-526.
    9. Sianesi, Barbara, 2017. "Evidence of randomisation bias in a large-scale social experiment: The case of ERA," Journal of Econometrics, Elsevier, vol. 198(1), pages 41-64.
    10. Markus Frölich, 2004. "Programme Evaluation with Multiple Treatments," Journal of Economic Surveys, Wiley Blackwell, vol. 18(2), pages 181-224, April.
    11. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    12. Christine Dauth & Julia Lang, 2019. "Can the unemployed be trained to care for the elderly? The effects of subsidized training in elderly care," Health Economics, John Wiley & Sons, Ltd., vol. 28(4), pages 543-555, April.
    13. Dauth, Christine & Lang, Julia, 2017. "Should the unemployed care for the elderly? : The effect of subsidized occupational and further training in elderly care," IAB-Discussion Paper 201713, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    14. Lang, Julia & Dauth, Christine, 2017. "Should the unemployed care for the elderly? The effect of subsidized occupational and further training in geriatric care," VfS Annual Conference 2017 (Vienna): Alternative Structures for Money and Banking 168130, Verein für Socialpolitik / German Economic Association.
    15. Joachim Wolff & Gesine Stephan, 2013. "Subsidized work before and after the German Hartz reforms: design of major schemes, evaluation results and lessons learnt," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 2(1), pages 1-24, December.
    16. Dauth, Christine, 2017. "Regional discontinuities and the effectiveness of further training subsidies for low-skilled employees," IAB-Discussion Paper 201707, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    17. Stefanie Hof, 2014. "Does Private Tutoring Work? The Effectiveness of Private Tutoring: A Nonparametric Bounds Analysis," Economics of Education Working Paper Series 0096, University of Zurich, Department of Business Administration (IBW).
    18. Matthew A Masten & Alexandre Poirier, 2023. "Choosing exogeneity assumptions in potential outcome models," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 327-349.
    19. Alberto Abadie & Guido W. Imbens, 2002. "Simple and Bias-Corrected Matching Estimators for Average Treatment Effects," NBER Technical Working Papers 0283, National Bureau of Economic Research, Inc.
    20. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.

    More about this item

    Keywords

    Determinanten ; Erhebungsmethode ; Experiment ; Intervention ; Kausalanalyse ; Kennzahlen ; Methode ; Änderung ; Arbeitslose ; Validität ; Weiterbildung ; Wirkungsforschung;
    All these keywords.

    JEL classification:

    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • C87 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Econometric Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iab:iabdpa:202029. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: IAB, Geschäftsbereich Wissenschaftliche Fachinformation und Bibliothek (email available below). General contact details of provider: https://edirc.repec.org/data/iabbbde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.