IDEAS home Printed from https://ideas.repec.org/p/zbw/rwirep/330177.html

The robustness dashboard

Author

Listed:
  • Bensch, Gunther
  • Rose, Julian
  • Brodeur, Abel
  • Ankel-Peters, Jörg

Abstract

Transparent communication of robustness is essential in empirical research, yet existing tools can be difficult to interpret. This paper introduces the robustness dashboard, a graphical tool that visualizes the results of robustness reproductions into a single, intuitive graph. The robustness dashboard draws on a multiverse of analytical paths to illustrate their effect on statistical significance. It complements existing visualisations, such as specification curves, by offering a more compact and accessible summary of robustness patterns. The dashboard can be easily implemented using our Stata command "repframe" and scaled according to the number of outcomes, ranging from stand-alone reproductions to meta-reproductions.

Suggested Citation

  • Bensch, Gunther & Rose, Julian & Brodeur, Abel & Ankel-Peters, Jörg, 2025. "The robustness dashboard," Ruhr Economic Papers 1167, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    • Handle: RePEc:zbw:rwirep:330177
    DOI: 10.4419/96973352
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/330177/1/1939359449.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.4419/96973352?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ankel-Peters, Jörg & Brodeur, Abel & Dreber, Anna & Johannesson, Magnus & Neubauer, Florian & Rose, Julian, 2024. "A Protocol for Structured Robustness Reproductions and Replicability Assessments," I4R Discussion Paper Series 143, The Institute for Replication (I4R).
    2. Lukas Fink & Jan Marcus, 2025. "Replication code availability over time and across fields: Evidence from the German Socio‐Economic Panel," Economic Inquiry, Western Economic Association International, vol. 63(2), pages 357-386, April.
    3. Vilhuber, Lars, 2023. "Reproducibility and transparency versus privacy and confidentiality: Reflections from a data editor," Journal of Econometrics, Elsevier, vol. 235(2), pages 2285-2294.
    4. Young, Alwyn, 2022. "Consistency without inference: instrumental variables in practical application," LSE Research Online Documents on Economics 115011, London School of Economics and Political Science, LSE Library.
    5. Nick Huntington-Klein & Claus Pörtner & Yubraj Acharya & Matus Adamkovic & Joop Adema & Lameck Ondieki Agasa & Imtiaz Ahmad & Mevlude Akbulut-Yuksel & Martin Eckhoff Andresen & David Angenendt & José-, 2025. "The Sources of Researcher Variation in Economics," Working Papers hal-05187084, HAL.
    6. Conley, Timothy G. & Kelly, Morgan, 2025. "The standard errors of persistence," Journal of International Economics, Elsevier, vol. 153(C).
    7. Campbell, Douglas & Brodeur, Abel & Dreber, Anna & Johannesson, Magnus & Kopecky, Joseph & Lusher, Lester & Tsoy, Nikita, 2024. "The Robustness Reproducibility of the American Economic Review," I4R Discussion Paper Series 124, The Institute for Replication (I4R).
    8. Anna Dreber & Magnus Johannesson, 2025. "A framework for evaluating reproducibility and replicability in economics," Economic Inquiry, Western Economic Association International, vol. 63(2), pages 338-356, April.
    9. Joel Ferguson & Rebecca Littman & Garret Christensen & Elizabeth Levy Paluck & Nicholas Swanson & Zenan Wang & Edward Miguel & David Birke & John-Henry Pezzuto, 2023. "Survey of open science practices and attitudes in the social sciences," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    10. Young, Alwyn, 2019. "Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results," LSE Research Online Documents on Economics 101401, London School of Economics and Political Science, LSE Library.
    11. Alwyn Young, 2019. "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 557-598.
    12. Lars Vilhuber & Jack Cavanagh, 2025. "Report of the AEA Data Editor," AEA Papers and Proceedings, American Economic Association, vol. 115, pages 944-957, May.
    13. Young, Alwyn, 2022. "Consistency without Inference: Instrumental Variables in Practical Application," European Economic Review, Elsevier, vol. 147(C).
    14. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    15. Sridevi K. Prasad & Fiona Kastel & Suvarna Pande & Chenxiao Zhang & Douglas M. Glandon, 2024. "A checklist to guide sensitivity analyses and replications of impact evaluations," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 16(3), pages 332-348, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brodeur, Abel & Mikola, Derek & Cook, Nikolai, 2024. "Mass Reproducibility and Replicability: A New Hope," IZA Discussion Papers 16912, IZA Network @ LISER.
    2. Ellis, Jimmy R. & Gershenson, Seth, 2020. "Gender, peer advising, and college success," Labour Economics, Elsevier, vol. 62(C).
    3. Marcus, Jan, 2025. "Replication code as a cornerstone of the credibility revolution 2.0," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 117(C).
    4. Slichter, David & Tran, Nhan, 2023. "Do better journals publish better estimates?," MPRA Paper 118433, University Library of Munich, Germany.
    5. Balafoutas, Loukas & Celse, Jeremy & Karakostas, Alexandros & Umashev, Nicholas, 2025. "Incentives and the replication crisis in social sciences: A critical review of open science practices," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 114(C).
    6. Sebastian Kranz, 2024. "From Replications to Revelations: Heteroskedasticity-Robust Inference," Papers 2411.14763, arXiv.org, revised Dec 2024.
    7. Aksoy, Cevat Giray & Poutvaara, Panu & Schikora, Felicitas, 2023. "First time around: Local conditions and multi-dimensional integration of refugees," Journal of Urban Economics, Elsevier, vol. 137(C).
    8. Diegmann, André & Pohlan, Laura & Weber, Andrea, 2024. "Do Politicians Affect Firm Outcomes? Evidence from Connections to the German Federal Parliament," IZA Discussion Papers 17031, IZA Network @ LISER.
    9. Allakulov, Umrbek & Cocciolo, Serena & Das, Binayak & Habib, Md. Ahasan & Rambjer, Lovisa & Tompsett, Anna, 2023. "Transparency, governance, and water and sanitation: Experimental evidence from schools in rural Bangladesh," Journal of Development Economics, Elsevier, vol. 163(C).
    10. Arne Henningsen & Guy Low & David Wuepper & Tobias Dalhaus & Hugo Storm & Dagim Belay & Stefan Hirsch, 2026. "Estimating Causal Effects With Observational Data: Guidelines for Agricultural and Applied Economists," Journal of Agricultural Economics, Wiley Blackwell, vol. 77(2), pages 356-382, June.
    11. Nguyen, Cuong Viet & Tran, Tuyen Quang & Van Vu, Huong, 2024. "The long-term effects of war on foreign direct investment and economic development: evidence from Vietnam," Journal of Urban Economics, Elsevier, vol. 143(C).
    12. Andreoni, James & Serra-Garcia, Marta, 2021. "Time inconsistent charitable giving," Journal of Public Economics, Elsevier, vol. 198(C).
    13. Johnsen, Åshild A. & Kvaløy, Ola, 2021. "Conspiracy against the public - An experiment on collusion11“People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices.” (Adam," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 94(C).
    14. repec:osf:osfxxx:sw6kd_v1 is not listed on IDEAS
    15. Eszter Czibor & Danny Hsu & David Jimenez-Gomez & Susanne Neckermann & Burcu Subasi, 2022. "Loss-Framed Incentives and Employee (Mis-)Behavior," Management Science, INFORMS, vol. 68(10), pages 7518-7537, October.
    16. Abhijit Banerjee & Rema Hanna & Benjamin A Olken & Elan Satriawan & Sudarno Sumarto, "undated". "Food vs. Food Stamps: Evidence from an At-Scale Experiment in Indonesia," Working Papers 2373, Communications Section.
    17. Anand Acharya & Lynda Khalaf & Marcel Voia & Myra Yazbeck & David Wensley, 2021. "Severity of Illness and the Duration of Intensive Care," Working Papers 2021-003, Human Capital and Economic Opportunity Working Group.
    18. LaFave, Daniel & Beyene, Abebe Damte & Bluffstone, Randall & Dissanayake, Sahan T.M. & Gebreegziabher, Zenebe & Mekonnen, Alemu & Toman, Michael, 2021. "Impacts of improved biomass cookstoves on child and adult health: Experimental evidence from rural Ethiopia," World Development, Elsevier, vol. 140(C).
    19. Blesse, Sebastian & Diegmann, André, 2022. "The place-based effects of police stations on crime: Evidence from station closures," Journal of Public Economics, Elsevier, vol. 207(C).
    20. Hermes, Henning & Krauß, Marina & Lergetporer, Philipp & Peter, Frauke & Wiederhold, Simon, 2022. "Early child care and labor supply of lower-SES mothers: A randomized controlled trial," DICE Discussion Papers 394, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    21. Tlou D. Raphela & Koketso M. Mpshane, 2026. "“Drowning in trauma”: mental health impacts of floods in overpopulated informal settlements in South Africa," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 122(8), pages 1-12, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General
    • C87 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Econometric Software

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:rwirep:330177. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/rwiesde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.