IDEAS home Printed from https://ideas.repec.org/a/aea/aecrev/v108y2018i10p3057-3103.html
   My bibliography  Save this article

Evaluating Strategic Forecasters

Author

Listed:
  • Rahul Deb
  • Mallesh M. Pai
  • Maher Said

Abstract

Motivated by the question of how one should evaluate professional election forecasters, we study a novel dynamic mechanism design problem without transfers. A principal who wishes to hire only high-quality forecasters is faced with an agent of unknown quality. The agent privately observes signals about a publicly observable future event, and may strategically misrepresent information to inflate the principal's perception of his quality. We show that the optimal deterministic mechanism is simple and easy to implement in practice: it evaluates a single, optimally timed prediction. We study the generality of this result and its robustness to randomization and noncommitment.

Suggested Citation

  • Rahul Deb & Mallesh M. Pai & Maher Said, 2018. "Evaluating Strategic Forecasters," American Economic Review, American Economic Association, vol. 108(10), pages 3057-3103, October.
    • Handle: RePEc:aea:aecrev:v:108:y:2018:i:10:p:3057-3103
    Note: DOI: 10.1257/aer.20170299
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/doi/10.1257/aer.20170299
    Download Restriction: no
    File URL: https://www.aeaweb.org/articles/attachments?retrieve=ix3Dzube-pLxysuPmCgMqM-QuBne6tF7
    Download Restriction: no
    File URL: https://www.aeaweb.org/articles/attachments?retrieve=wHC0TzmPHMo5qKkpqUAJkPL69a_BcMRN
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Philippe Aghion & Matthew O. Jackson, 2016. "Inducing Leaders to Take Risky Decisions: Dismissal, Tenure, and Term Limits," American Economic Journal: Microeconomics, American Economic Association, vol. 8(3), pages 1-38, August.
    2. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    3. Trueman, Brett, 1994. "Analyst Forecasts and Herding Behavior," The Review of Financial Studies, Society for Financial Studies, vol. 7(1), pages 97-124.
    4. Guo, Yingni & Hörner, Johannes, 2015. "Dynamic Mechanisms without Money," Economics Series 310, Institute for Advanced Studies.
    5. Marco Battaglini, 2005. "Long-Term Contracting with Markovian Consumers," American Economic Review, American Economic Association, vol. 95(3), pages 637-658, June.
    6. Wojciech Olszewski & Marcin Pęski, 2011. "The Principal-Agent Approach to Testing Experts," American Economic Journal: Microeconomics, American Economic Association, vol. 3(2), pages 89-113, May.
    7. Lahiri, Kajal & Sheng, Xuguang, 2008. "Evolution of forecast disagreement in a Bayesian learning model," Journal of Econometrics, Elsevier, vol. 144(2), pages 325-340, June.
    8. Eső, Péter & Szentes, Balázs, 2017. "Dynamic contracting: an irrelevance theorem," LSE Research Online Documents on Economics 69403, London School of Economics and Political Science, LSE Library.
    9. Daniel Krähmer & Roland Strausz, 2011. "Optimal Procurement Contracts with Pre-Project Planning," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 78(3), pages 1015-1041.
    10. Strausz, Roland, 2003. "Deterministic mechanisms and the revelation principle," Economics Letters, Elsevier, vol. 79(3), pages 333-337, June.
    11. Bengt Holmström, 1999. "Managerial Incentive Problems: A Dynamic Perspective," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(1), pages 169-182.
    12. G. Elliott & C. Granger & A. Timmermann (ed.), 2013. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 2, number 2.
    13. Olszewski, Wojciech, 2015. "Calibration and Expert Testing," Handbook of Game Theory with Economic Applications,, Elsevier.
    14. Roger B. Myerson, 1981. "Optimal Auction Design," Mathematics of Operations Research, INFORMS, vol. 6(1), pages 58-73, February.
    15. Alessandro Pavan & Ilya Segal & Juuso Toikka, 2014. "Dynamic Mechanism Design: A Myersonian Approach," Econometrica, Econometric Society, vol. 82(2), pages 601-653, March.
    16. Crawford, Vincent P & Sobel, Joel, 1982. "Strategic Information Transmission," Econometrica, Econometric Society, vol. 50(6), pages 1431-1451, November.
    17. Raphael Boleslavsky & Maher Said, 2013. "Progressive Screening: Long-Term Contracting with a Privately Known Stochastic Process," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(1), pages 1-34.
    18. Cremer, Jacques & McLean, Richard P, 1988. "Full Extraction of the Surplus in Bayesian and Dominant Strategy Auctions," Econometrica, Econometric Society, vol. 56(6), pages 1247-1257, November.
    19. Krishna, Vijay & Morgan, John, 2004. "The art of conversation: eliciting information from experts through multi-stage communication," Journal of Economic Theory, Elsevier, vol. 117(2), pages 147-179, August.
    20. Ying Chen & Navin Kartik & Joel Sobel, 2008. "Selecting Cheap-Talk Equilibria," Econometrica, Econometric Society, vol. 76(1), pages 117-136, January.
    21. Wei Li, 2007. "Changing One's Mind when the Facts Change: Incentives of Experts and the Design of Reporting Protocols," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1175-1194.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cipullo, Davide & Reslow, André, 2019. "Biased Forecasts to Affect Voting Decisions? The Brexit Case," Working Paper Series 364, Sveriges Riksbank (Central Bank of Sweden).
    2. Aleksei Smirnov & Egor Starkov, 2019. "Timing of predictions in dynamic cheap talk: experts vs. quacks," ECON - Working Papers 334, Department of Economics - University of Zurich.
    3. Dirk Bergemann & Tan Gan & Yingkai Li, 2023. "Managing Persuasion Robustly: The Optimality of Quota Rules," Papers 2310.10024, arXiv.org.
    4. repec:cup:judgdm:v:15:y:2020:i:5:p:863-880 is not listed on IDEAS
    5. Yingkai Li & Jonathan Libgober, 2023. "Implementing Evidence Acquisition: Time Dependence in Contracts for Advice," Papers 2310.19147, arXiv.org, revised Sep 2024.
    6. Andrew Gelman & Jessica Hullman & Christopher Wlezien & George Elliott Morris, 2020. "Information, incentives, and goals in election forecasts," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(5), pages 863-880, September.
    7. Deb, Rahul & Mitchell, Matthew & Pai, Mallesh M., 2022. "(Bad) reputation in relational contracting," Theoretical Economics, Econometric Society, vol. 17(2), May.
    8. Lukyanov, Georgy, 2023. "Reputation for competence in a cheap-talk setting," Research in Economics, Elsevier, vol. 77(3), pages 285-294.
    9. Rajiv Vohra & Francisco Espinosa & Debraj Ray, 2021. "A Principal-Agent Relationship with No Advantage to Commitment," Working Papers 2021-003, Brown University, Department of Economics.
    10. Rahul Deb & Matthew Mitchell & Mallesh Pai, 2019. "Our distrust is very expensive," Working Papers tecipa-632, University of Toronto, Department of Economics.
    11. Fan Wu, 2024. "Incentivizing Information Acquisition," Papers 2410.13978, arXiv.org, revised Nov 2024.
    12. Dell’Era, Michele, 2020. "Talking to influence," Economics Letters, Elsevier, vol. 192(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arve, Malin & Zwart, Gijsbert, 2023. "Optimal procurement and investment in new technologies under uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 147(C).
    2. Dirk Bergemann & Alessandro Pavan, 2015. "Introduction to JET Symposium Issue on "Dynamic Contracts and Mechanism Design"," Cowles Foundation Discussion Papers 2016, Cowles Foundation for Research in Economics, Yale University.
    3. Pavesi, Filippo & Scotti, Massimo, 2022. "Good lies," European Economic Review, Elsevier, vol. 141(C).
      • Filippo Pavesi & Massimo Scotti, 2019. "Good Lies," Working Paper Series 39, Economics Discipline Group, UTS Business School, University of Technology, Sydney.
    4. Irene Valsecchi, 2013. "The expert problem: a survey," Economics of Governance, Springer, vol. 14(4), pages 303-331, November.
    5. Bergemann, Dirk & Pavan, Alessandro, 2015. "Introduction to Symposium on Dynamic Contracts and Mechanism Design," Journal of Economic Theory, Elsevier, vol. 159(PB), pages 679-701.
    6. Deb, Rahul & Said, Maher, 2015. "Dynamic screening with limited commitment," Journal of Economic Theory, Elsevier, vol. 159(PB), pages 891-928.
    7. Boleslavsky, Raphael & Lewis, Tracy R., 2016. "Evolving influence: Mitigating extreme conflicts of interest in advisory relationships," Games and Economic Behavior, Elsevier, vol. 98(C), pages 110-134.
    8. Boaz Zik, 2023. "Efficient sequential screening with informational externalities," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 75(2), pages 567-590, February.
    9. Aleksei Smirnov & Egor Starkov, 2019. "Timing of predictions in dynamic cheap talk: experts vs. quacks," ECON - Working Papers 334, Department of Economics - University of Zurich.
    10. Jean Tirole, 2016. "From Bottom of the Barrel to Cream of the Crop: Sequential Screening With Positive Selection," Econometrica, Econometric Society, vol. 84(4), pages 1291-1343, July.
    11. Terstiege, Stefan, 2016. "Gathering imperfect information before signing a contract," Games and Economic Behavior, Elsevier, vol. 97(C), pages 70-87.
    12. Dirk Bergemann & Juuso Välimäki, 2019. "Dynamic Mechanism Design: An Introduction," Journal of Economic Literature, American Economic Association, vol. 57(2), pages 235-274, June.
    13. Hao Li & Xianwen Shi, 2017. "Discriminatory Information Disclosure," American Economic Review, American Economic Association, vol. 107(11), pages 3363-3385, November.
    14. Irene Valsecchi, 2013. "Non-uniqueness of equilibrium action profiles with equal size in one-shot cheap-talk games," Theory and Decision, Springer, vol. 74(1), pages 31-53, January.
    15. Krähmer, Daniel & Strausz, Roland, 2015. "Ex post information rents in sequential screening," Games and Economic Behavior, Elsevier, vol. 90(C), pages 257-273.
    16. Lu, Jingfeng & Wang, Zijia, 2021. "Optimal selling mechanisms with buyer price search," Journal of Economic Theory, Elsevier, vol. 196(C).
    17. Lim, Wooyoung, 2014. "Communication in bargaining over decision rights," Games and Economic Behavior, Elsevier, vol. 85(C), pages 159-179.
    18. Meng, Dawen & Sun, Lei & Tian, Guoqiang, 2022. "Dynamic mechanism design on social networks," Games and Economic Behavior, Elsevier, vol. 131(C), pages 84-120.
    19. Krasikov, Ilia & Lamba, Rohit, 2021. "A theory of dynamic contracting with financial constraints," Journal of Economic Theory, Elsevier, vol. 193(C).
    20. von Wangenheim, Jonas, 2017. "Consumer-Optimal Information Design," Rationality and Competition Discussion Paper Series 53, CRC TRR 190 Rationality and Competition.

    More about this item

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • D72 - Microeconomics - - Analysis of Collective Decision-Making - - - Political Processes: Rent-seeking, Lobbying, Elections, Legislatures, and Voting Behavior
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aecrev:v:108:y:2018:i:10:p:3057-3103. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.