IDEAS home Printed from https://ideas.repec.org/a/aea/apandp/v110y2020p75-79.html

Forecasting the Results of Experiments: Piloting an Elicitation Strategy

Author

Listed:
  • Stefano DellaVigna
  • Nicholas Otis
  • Eva Vivalt

Abstract

Forecasts of experimental results can clarify the interpretation of research results, mitigate publication bias, and improve experimental designs. We collect forecasts of the results of three Registered Reports preliminarily accepted to the Journal of Development Economics, randomly varying four features: (1) small versus large reference values, (2) whether predictions are in raw units or standard deviations, (3) text-entry versus slider responses, and (4) small versus large slider bounds. Forecasts are generally robust to elicitation features, though wider slider bounds are associated with higher forecasts throughout the forecast distribution. We make preliminary recommendations on how many forecasts should be gathered.

Suggested Citation

  • Stefano DellaVigna & Nicholas Otis & Eva Vivalt, 2020. "Forecasting the Results of Experiments: Piloting an Elicitation Strategy," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 75-79, May.
    • Handle: RePEc:aea:apandp:v:110:y:2020:p:75-79
    DOI: 10.1257/pandp.20201080
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/doi/10.1257/pandp.20201080
    Download Restriction: no
    File URL: http://doi.org/10.3886/E117601V1
    Download Restriction: no
    File URL: https://www.aeaweb.org/doi/10.1257/pandp.20201080.appx
    Download Restriction: no
    File URL: https://www.aeaweb.org/doi/10.1257/pandp.20201080.ds
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    File URL: https://libkey.io/10.1257/pandp.20201080?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mackenzie Alston & Tatyana Deryugina & Olga Shurchkov, 2025. "Leaving Money on the Table," NBER Working Papers 33657, National Bureau of Economic Research, Inc.
    2. Frederico Finan & Demian Pouzo, 2021. "Reinforcing RCTs with Multiple Priors while Learning about External Validity," Papers 2112.09170, arXiv.org, revised Sep 2024.
    3. Henning Schaak & Jens Rommel & Julian Sagebiel & Jesus Barreiro-Hurlé & Douadia Bougherara & Luigi Cemablo & Marija Cerjak & Tajana Čop & Mikołaj Czajkowski & María Espinosa-Goded & Julia Höhler & Car, 2022. "How Well Can Experts Predict Farmers' Risk Preferences ?," Post-Print hal-04971746, HAL.
    4. Tamim,Abdulrazzak & Smith,Emma & Palmer,I. Bailey & Miguel,Edward & Leone,Samuel & Rozo, Sandra & Stillman,Sarah, 2025. "Housing Subsidies for Refugees : Experimental Evidence on Life Outcomes and Social Integration in Jordan," Policy Research Working Paper Series 11042, The World Bank.
    5. Olckers, Matthew, 2021. "On track for retirement?," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 76-88.
    6. Grewenig, Elisabeth & Gründler, Klaus & Lergetporer, Philipp & Potrafke, Niklas & Werner, Katharina & Zeidler, Helen, 2026. "Expertise and Prediction Accuracy," IZA Discussion Papers 18453, IZA Network @ LISER.
    7. Chadimová, Kateřina & Cahlíková, Jana & Cingl, Lubomír, 2022. "Foretelling what makes people pay: Predicting the results of field experiments on TV fee enforcement," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 100(C).
    8. Matias Giaccobasso & Brad Nathan & Ricardo Perez-Truglia & Alejandro Zentner, 2025. "Where Do My Tax Dollars Go? Tax Morale Effects of Perceived Government Spending," American Economic Journal: Applied Economics, American Economic Association, vol. 17(4), pages 223-259, October.
    9. Leonardo Iacovone & David McKenzie & Rachael Meager, 2025. "Bayesian Impact Evaluation With Informative Priors: An Application to a Colombian Management and Export Improvement Program," Econometrica, Econometric Society, vol. 93(5), pages 1915-1935, September.
    10. Yang, Dean & Allen, James & Mahumane, Arlete & Riddell, James & Yu, Hang, 2023. "Knowledge, stigma, and HIV testing: An analysis of a widespread HIV/AIDS program," Journal of Development Economics, Elsevier, vol. 160(C).
    11. Vu, Patrick, 2024. "Why are replication rates so low?," Journal of Econometrics, Elsevier, vol. 245(1).
    12. Stefano DellaVigna & Nicholas Otis & Eva Vivalt, 2020. "Forecasting the Results of Experiments: Piloting an Elicitation Strategy," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 75-79, May.
    13. Bah, Tijan L. & Batista, Catia & Gubert, Flore & McKenzie, David, 2023. "Can information and alternatives to irregular migration reduce “backway” migration from The Gambia?," Journal of Development Economics, Elsevier, vol. 165(C).
    14. Del Carmen,Giselle & Espinal Hernandez,Edgardo Enrique & De Gouvea Scot De Arruda,Thiago, 2022. "Targeting in Tax Compliance Interventions : Experimental Evidence from Honduras," Policy Research Working Paper Series 9967, The World Bank.
    15. Abdulrazzak Tamim & Emma C. Smith & Bailey Palmer & Edward Miguel & Samuel Leone & Sandra V. Rozo & Sarah Stillman, 2025. "Housing Subsidies for Refugees: Experimental Evidence on Life Outcomes and Social Integration in Jordan," NBER Working Papers 33408, National Bureau of Economic Research, Inc.
    16. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • O10 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:apandp:v:110:y:2020:p:75-79. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.