IDEAS home Printed from https://ideas.repec.org/p/ste/nystbu/17-02.html
   My bibliography  Save this paper

Evaluating Strategic Forecasters

Author

Listed:
  • Rahul Deb
  • Mallesh M. Pai
  • Maher Said

Abstract

Motivated by the question of how one should evaluate professional election forecasters, we study a novel dynamic mechanism design problem without transfers. A principal who wishes to hire only high-quality forecasters is faced with an agent of unknown quality. The agent privately observes signals about a publicly observable future event, and may strategically misrepresent information to inflate the principal's perception of his quality. We show that the optimal deterministic mechanism is simple and easy to implement in practice: it evaluates a single, optimally timed prediction. We study the generality of this result and its robustness to randomization and noncommitment.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Rahul Deb & Mallesh M. Pai & Maher Said, 2017. "Evaluating Strategic Forecasters," Working Papers 17-02, New York University, Leonard N. Stern School of Business, Department of Economics.
  • Handle: RePEc:ste:nystbu:17-02
    as

    Download full text from publisher

    File URL: http://w4.stern.nyu.edu/economics/docs/workingpapers/2017/Said_EvaluatingForecasters.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Eső, Péter & Szentes, Balázs, 2017. "Dynamic contracting: an irrelevance theorem," LSE Research Online Documents on Economics 69403, London School of Economics and Political Science, LSE Library.
    2. Olszewski, Wojciech, 2015. "Calibration and Expert Testing," Handbook of Game Theory with Economic Applications,, Elsevier.
    3. Ying Chen & Navin Kartik & Joel Sobel, 2008. "Selecting Cheap-Talk Equilibria," Econometrica, Econometric Society, vol. 76(1), pages 117-136, January.
    4. Yingni Guo & Johannes Horner, 2015. "Dynamic Mechanisms without Money," Cowles Foundation Discussion Papers 1985, Cowles Foundation for Research in Economics, Yale University.
    5. Daniel Krähmer & Roland Strausz, 2011. "Optimal Procurement Contracts with Pre-Project Planning," Review of Economic Studies, Oxford University Press, vol. 78(3), pages 1015-1041.
    6. Roger B. Myerson, 1981. "Optimal Auction Design," Mathematics of Operations Research, INFORMS, vol. 6(1), pages 58-73, February.
    7. Krishna, Vijay & Morgan, John, 2004. "The art of conversation: eliciting information from experts through multi-stage communication," Journal of Economic Theory, Elsevier, vol. 117(2), pages 147-179, August.
    8. Bengt Holmstrom, 1999. "Managerial Incentive Problems: A Dynamic Perspective," NBER Working Papers 6875, National Bureau of Economic Research, Inc.
    9. Wei Li, 2007. "Changing One's Mind when the Facts Change: Incentives of Experts and the Design of Reporting Protocols," Review of Economic Studies, Oxford University Press, vol. 74(4), pages 1175-1194.
    10. Marco Battaglini, 2005. "Long-Term Contracting with Markovian Consumers," American Economic Review, American Economic Association, vol. 95(3), pages 637-658, June.
    11. Philippe Aghion & Matthew O. Jackson, 2016. "Inducing Leaders to Take Risky Decisions: Dismissal, Tenure, and Term Limits," American Economic Journal: Microeconomics, American Economic Association, vol. 8(3), pages 1-38, August.
    12. G. Elliott & C. Granger & A. Timmermann (ed.), 2013. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 2, number 2.
    13. Crawford, Vincent P & Sobel, Joel, 1982. "Strategic Information Transmission," Econometrica, Econometric Society, vol. 50(6), pages 1431-1451, November.
    14. Cremer, Jacques & McLean, Richard P, 1988. "Full Extraction of the Surplus in Bayesian and Dominant Strategy Auctions," Econometrica, Econometric Society, vol. 56(6), pages 1247-1257, November.
    15. Strausz, Roland, 2003. "Deterministic mechanisms and the revelation principle," Economics Letters, Elsevier, vol. 79(3), pages 333-337, June.
    16. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    17. Trueman, Brett, 1994. "Analyst Forecasts and Herding Behavior," Review of Financial Studies, Society for Financial Studies, vol. 7(1), pages 97-124.
    18. Wojciech Olszewski & Marcin Pęski, 2011. "The Principal-Agent Approach to Testing Experts," American Economic Journal: Microeconomics, American Economic Association, vol. 3(2), pages 89-113, May.
    19. Alessandro Pavan & Ilya Segal & Juuso Toikka, 2014. "Dynamic Mechanism Design: A Myersonian Approach," Econometrica, Econometric Society, vol. 82(2), pages 601-653, March.
    20. Lahiri, Kajal & Sheng, Xuguang, 2008. "Evolution of forecast disagreement in a Bayesian learning model," Journal of Econometrics, Elsevier, vol. 144(2), pages 325-340, June.
    21. Bengt Holmström, 1999. "Managerial Incentive Problems: A Dynamic Perspective," Review of Economic Studies, Oxford University Press, vol. 66(1), pages 169-182.
    22. Raphael Boleslavsky & Maher Said, 2013. "Progressive Screening: Long-Term Contracting with a Privately Known Stochastic Process," Review of Economic Studies, Oxford University Press, vol. 80(1), pages 1-34.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cipullo, Davide & Reslow, André, 2019. "Biased Forecasts to Affect Voting Decisions? The Brexit Case," Working Paper Series 364, Sveriges Riksbank (Central Bank of Sweden).
    2. Aleksei Smirnov & Egor Starkov, 2019. "Timing of predictions in dynamic cheap talk: experts vs. quacks," ECON - Working Papers 334, Department of Economics - University of Zurich.
    3. Dirk Bergemann & Tan Gan & Yingkai Li, 2023. "Managing Persuasion Robustly: The Optimality of Quota Rules," Papers 2310.10024, arXiv.org.
    4. repec:cup:judgdm:v:15:y:2020:i:5:p:863-880 is not listed on IDEAS
    5. Yingkai Li & Jonathan Libgober, 2023. "Optimal Scoring for Dynamic Information Acquisition," Papers 2310.19147, arXiv.org.
    6. Andrew Gelman & Jessica Hullman & Christopher Wlezien & George Elliott Morris, 2020. "Information, incentives, and goals in election forecasts," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(5), pages 863-880, September.
    7. Deb, Rahul & Mitchell, Matthew & Pai, Mallesh M., 2022. "(Bad) reputation in relational contracting," Theoretical Economics, Econometric Society, vol. 17(2), May.
    8. Lukyanov, Georgy, 2023. "Reputation for competence in a cheap-talk setting," Research in Economics, Elsevier, vol. 77(3), pages 285-294.
    9. Rajiv Vohra & Francisco Espinosa & Debraj Ray, 2021. "A Principal-Agent Relationship with No Advantage to Commitment," Working Papers 2021-003, Brown University, Department of Economics.
    10. Rahul Deb & Matthew Mitchell & Mallesh Pai, 2019. "Our distrust is very expensive," Working Papers tecipa-632, University of Toronto, Department of Economics.
    11. Dell’Era, Michele, 2020. "Talking to influence," Economics Letters, Elsevier, vol. 192(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pavesi, Filippo & Scotti, Massimo, 2022. "Good lies," European Economic Review, Elsevier, vol. 141(C).
      • Filippo Pavesi & Massimo Scotti, 2019. "Good Lies," Working Paper Series 39, Economics Discipline Group, UTS Business School, University of Technology, Sydney.
    2. Arve, Malin & Zwart, Gijsbert, 2023. "Optimal procurement and investment in new technologies under uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 147(C).
    3. Dirk Bergemann & Alessandro Pavan, 2015. "Introduction to JET Symposium Issue on "Dynamic Contracts and Mechanism Design"," Cowles Foundation Discussion Papers 2016, Cowles Foundation for Research in Economics, Yale University.
    4. Deb, Rahul & Said, Maher, 2015. "Dynamic screening with limited commitment," Journal of Economic Theory, Elsevier, vol. 159(PB), pages 891-928.
    5. Boleslavsky, Raphael & Lewis, Tracy R., 2016. "Evolving influence: Mitigating extreme conflicts of interest in advisory relationships," Games and Economic Behavior, Elsevier, vol. 98(C), pages 110-134.
    6. Irene Valsecchi, 2013. "The expert problem: a survey," Economics of Governance, Springer, vol. 14(4), pages 303-331, November.
    7. Bergemann, Dirk & Pavan, Alessandro, 2015. "Introduction to Symposium on Dynamic Contracts and Mechanism Design," Journal of Economic Theory, Elsevier, vol. 159(PB), pages 679-701.
    8. Boaz Zik, 2023. "Efficient sequential screening with informational externalities," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 75(2), pages 567-590, February.
    9. Dirk Bergemann & Juuso Välimäki, 2019. "Dynamic Mechanism Design: An Introduction," Journal of Economic Literature, American Economic Association, vol. 57(2), pages 235-274, June.
    10. Hao Li & Xianwen Shi, 2017. "Discriminatory Information Disclosure," American Economic Review, American Economic Association, vol. 107(11), pages 3363-3385, November.
    11. Liu, Yaozhou Franklin & Sanyal, Amal, 2012. "When second opinions hurt: A model of expert advice under career concerns," Journal of Economic Behavior & Organization, Elsevier, vol. 84(1), pages 1-16.
    12. Jean Tirole, 2016. "From Bottom of the Barrel to Cream of the Crop: Sequential Screening With Positive Selection," Econometrica, Econometric Society, vol. 84, pages 1291-1343, July.
    13. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    14. Peter Sorensen & Marco Ottaviani, 2000. "Herd Behavior and Investment: Comment," American Economic Review, American Economic Association, vol. 90(3), pages 695-704, June.
    15. Irene Valsecchi, 2013. "Non-uniqueness of equilibrium action profiles with equal size in one-shot cheap-talk games," Theory and Decision, Springer, vol. 74(1), pages 31-53, January.
    16. Krähmer, Daniel & Strausz, Roland, 2015. "Ex post information rents in sequential screening," Games and Economic Behavior, Elsevier, vol. 90(C), pages 257-273.
    17. Lu, Jingfeng & Wang, Zijia, 2021. "Optimal selling mechanisms with buyer price search," Journal of Economic Theory, Elsevier, vol. 196(C).
    18. Hahn, Volker, 2011. "Sequential aggregation of verifiable information," Journal of Public Economics, Elsevier, vol. 95(11), pages 1447-1454.
    19. Marco Ottaviani & Peter Norman Sorensen, 2002. "Professional Advice: The Theory of Reputational Cheap Talk," Discussion Papers 02-05, University of Copenhagen. Department of Economics.
    20. Rohit Lamba & Ilia Krasikov, 2017. "A Theory of Dynamic Contracting with Financial Constraints," 2017 Meeting Papers 1544, Society for Economic Dynamics.

    More about this item

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • D72 - Microeconomics - - Analysis of Collective Decision-Making - - - Political Processes: Rent-seeking, Lobbying, Elections, Legislatures, and Voting Behavior
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ste:nystbu:17-02. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Amanda Murphy (email available below). General contact details of provider: https://edirc.repec.org/data/ednyuus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.