IDEAS home Printed from https://ideas.repec.org/a/aea/apandp/v110y2020p75-79.html

Forecasting the Results of Experiments: Piloting an Elicitation Strategy

Author

Listed:
  • Stefano DellaVigna
  • Nicholas Otis
  • Eva Vivalt

Abstract

Forecasts of experimental results can clarify the interpretation of research results, mitigate publication bias, and improve experimental designs. We collect forecasts of the results of three Registered Reports preliminarily accepted to the Journal of Development Economics, randomly varying four features: (1) small versus large reference values, (2) whether predictions are in raw units or standard deviations, (3) text-entry versus slider responses, and (4) small versus large slider bounds. Forecasts are generally robust to elicitation features, though wider slider bounds are associated with higher forecasts throughout the forecast distribution. We make preliminary recommendations on how many forecasts should be gathered.

Suggested Citation

  • Stefano DellaVigna & Nicholas Otis & Eva Vivalt, 2020. "Forecasting the Results of Experiments: Piloting an Elicitation Strategy," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 75-79, May.
    • Handle: RePEc:aea:apandp:v:110:y:2020:p:75-79
    DOI: 10.1257/pandp.20201080
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/doi/10.1257/pandp.20201080
    Download Restriction: no
    File URL: http://doi.org/10.3886/E117601V1
    Download Restriction: no
    File URL: https://www.aeaweb.org/doi/10.1257/pandp.20201080.appx
    Download Restriction: no
    File URL: https://www.aeaweb.org/doi/10.1257/pandp.20201080.ds
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    File URL: https://libkey.io/10.1257/pandp.20201080?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Olckers, Matthew, 2021. "On track for retirement?," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 76-88.
    2. Leonardo Iacovone & David McKenzie & Rachael Meager, 2025. "Bayesian Impact Evaluation With Informative Priors: An Application to a Colombian Management and Export Improvement Program," Econometrica, Econometric Society, vol. 93(5), pages 1915-1935, September.
    3. Vu, Patrick, 2024. "Why are replication rates so low?," Journal of Econometrics, Elsevier, vol. 245(1).
    4. Bah, Tijan L. & Batista, Catia & Gubert, Flore & McKenzie, David, 2023. "Can information and alternatives to irregular migration reduce “backway” migration from The Gambia?," Journal of Development Economics, Elsevier, vol. 165(C).
    5. Stefano DellaVigna & Nicholas Otis & Eva Vivalt, 2020. "Forecasting the Results of Experiments: Piloting an Elicitation Strategy," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 75-79, May.
    6. Abdulrazzak Tamim & Emma C. Smith & Bailey Palmer & Edward Miguel & Samuel Leone & Sandra V. Rozo & Sarah Stillman, 2025. "Housing Subsidies for Refugees: Experimental Evidence on Life Outcomes and Social Integration in Jordan," NBER Working Papers 33408, National Bureau of Economic Research, Inc.
    7. Frederico Finan & Demian Pouzo, 2021. "Reinforcing RCTs with Multiple Priors while Learning about External Validity," Papers 2112.09170, arXiv.org, revised Sep 2024.
    8. Elisabeth Grewenig & Klaus Gründler & Philipp Lergetporer & Niklas Potrafke & Katharina Werner & Helen Zeidler, 2026. "Expertise and Prediction Accuracy," CESifo Working Paper Series 12522, CESifo.
    9. Chadimová, Kateřina & Cahlíková, Jana & Cingl, Lubomír, 2022. "Foretelling what makes people pay: Predicting the results of field experiments on TV fee enforcement," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 100(C).
    10. Matias Giaccobasso & Brad Nathan & Ricardo Perez-Truglia & Alejandro Zentner, 2025. "Where Do My Tax Dollars Go? Tax Morale Effects of Perceived Government Spending," American Economic Journal: Applied Economics, American Economic Association, vol. 17(4), pages 223-259, October.
    11. Del Carmen,Giselle & Espinal Hernandez,Edgardo Enrique & De Gouvea Scot De Arruda,Thiago, 2022. "Targeting in Tax Compliance Interventions : Experimental Evidence from Honduras," Policy Research Working Paper Series 9967, The World Bank.
    12. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    13. Mackenzie Alston & Tatyana Deryugina & Olga Shurchkov, 2025. "Leaving Money on the Table," NBER Working Papers 33657, National Bureau of Economic Research, Inc.
    14. Henning Schaak & Jens Rommel & Julian Sagebiel & Jesus Barreiro-Hurlé & Douadia Bougherara & Luigi Cemablo & Marija Cerjak & Tajana Čop & Mikołaj Czajkowski & María Espinosa-Goded & Julia Höhler & Car, 2022. "How Well Can Experts Predict Farmers' Risk Preferences ?," Post-Print hal-04971746, HAL.
    15. Tamim,Abdulrazzak & Smith,Emma & Palmer,I. Bailey & Miguel,Edward & Leone,Samuel & Rozo, Sandra & Stillman,Sarah, 2025. "Housing Subsidies for Refugees : Experimental Evidence on Life Outcomes and Social Integration in Jordan," Policy Research Working Paper Series 11042, The World Bank.
    16. Yang, Dean & Allen, James & Mahumane, Arlete & Riddell, James & Yu, Hang, 2023. "Knowledge, stigma, and HIV testing: An analysis of a widespread HIV/AIDS program," Journal of Development Economics, Elsevier, vol. 160(C).

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • O10 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:apandp:v:110:y:2020:p:75-79. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.