IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2605.07671.html

The Endogeneity of Miscalibration: Impossibility and Escape in Scored Reporting

Author

Listed:
  • Lauri Lov'en
  • Sasu Tarkoma

Abstract

Eliciting truthful reports from autonomous agents is a core problem in scalable AI oversight: a principal scores the agent's report using a strictly proper scoring rule, but the agent also benefits from the report through a non-accuracy channel (approval for autonomous action, allocation share, downstream control). The same structure appears in classical mechanism-design settings such as marketplace operation. Our main result is an endogeneity: the principal's optimal oversight necessarily uses a non-affine approval function to screen types, yet any non-affine approval makes truthful reporting suboptimal under the combined objective whenever deviation is undetectable. The principal cannot avoid the perturbation that undermines calibration. This impossibility holds for all strictly proper scoring rules, with a closed-form perturbation formula. A constructive escape exists: a step-function approval threshold achieves first-best screening for every strictly proper scoring rule, because the agent's binary inflate-or-not choice creates a type-space threshold regardless of the generator's curvature. Under the Brier score specifically, the type-independent inflation cost yields a welfare equivalence between second-best and first-best; we prove this equivalence is unique to Brier (the welfare gap under smooth $C^1$ oversight is bounded below by $\Omega(\text{Var}(1/G'') (\gamma/\beta)^2)$ for every non-Brier rule). Two instances develop the framework: AI agent oversight (the lead motivating setting) and marketplace operation (a parallel mechanism-design domain). The message for AI alignment is direct: smooth scoring-based oversight cannot elicit truthful reports from a strategic agent; sharp thresholds are the calibration-preserving design.

Suggested Citation

  • Lauri Lov'en & Sasu Tarkoma, 2026. "The Endogeneity of Miscalibration: Impossibility and Escape in Scored Reporting," Papers 2605.07671, arXiv.org.
  • Handle: RePEc:arx:papers:2605.07671
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2605.07671
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Paul Milgrom & Ilya Segal, 2002. "Envelope Theorems for Arbitrary Choice Sets," Econometrica, Econometric Society, vol. 70(2), pages 583-601, March.
    2. Milgrom,Paul, 2004. "Putting Auction Theory to Work," Cambridge Books, Cambridge University Press, number 9780521536721, August.
    3. Dye, Ra, 1985. "Disclosure Of Nonproprietary Information," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 23(1), pages 123-145.
    4. Jean-Jacques Laffont & Jean Tirole, 1993. "A Theory of Incentives in Procurement and Regulation," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262121743, December.
    5. Laffont, Jean-Jacques & Tirole, Jean, 1986. "Using Cost Observation to Regulate Firms," Journal of Political Economy, University of Chicago Press, vol. 94(3), pages 614-641, June.
    6. Matthew Gentzkow & Emir Kamenica, 2017. "Competition in Persuasion," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 84(1), pages 300-322.
    7. George J. Mailath & Larry Samuelson, 2001. "Who Wants a Good Reputation?," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 68(2), pages 415-441.
    8. Drew Fudenberg & David K. Levine, 2008. "Reputation And Equilibrium Selection In Games With A Patient Player," World Scientific Book Chapters, in: Drew Fudenberg & David K Levine (ed.), A Long-Run Collaboration On Long-Run Games, chapter 7, pages 123-142, World Scientific Publishing Co. Pte. Ltd..
    9. Bergemann, Dirk & Morris, Stephen, 2016. "Bayes correlated equilibrium and the comparison of information structures in games," Theoretical Economics, Econometric Society, vol. 11(2), May.
    10. Alessandro Lizzeri, 1999. "Information Revelation and Certification Intermediaries," RAND Journal of Economics, The RAND Corporation, vol. 30(2), pages 214-231, Summer.
    11. Grossman, Sanford J, 1981. "The Informational Role of Warranties and Private Disclosure about Product Quality," Journal of Law and Economics, University of Chicago Press, vol. 24(3), pages 461-483, December.
    12. Roger B. Myerson, 1981. "Optimal Auction Design," Mathematics of Operations Research, INFORMS, vol. 6(1), pages 58-73, February.
    13. Dirk Bergemann & Tibor Heumann & Stephen Morris, 2026. "Information Design and Mechanism Design: An Integrated Framework," Cowles Foundation Discussion Papers 2494, Cowles Foundation for Research in Economics, Yale University.
    14. Skreta, Vasiliki & Veldkamp, Laura, 2009. "Ratings shopping and asset complexity: A theory of ratings inflation," Journal of Monetary Economics, Elsevier, vol. 56(5), pages 678-695, July.
    15. Green, Jerry & Laffont, Jean-Jacques, 1977. "Characterization of Satisfactory Mechanisms for the Revelation of Preferences for Public Goods," Econometrica, Econometric Society, vol. 45(2), pages 427-438, March.
    16. Mohammad Akbarpour & Shengwu Li, 2020. "Credible Auctions: A Trilemma," Econometrica, Econometric Society, vol. 88(2), pages 425-467, March.
    17. Gibbard, Allan, 1973. "Manipulation of Voting Schemes: A General Result," Econometrica, Econometric Society, vol. 41(4), pages 587-601, July.
    18. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    19. Myerson, Roger B, 1979. "Incentive Compatibility and the Bargaining Problem," Econometrica, Econometric Society, vol. 47(1), pages 61-73, January.
    20. Piotr Dworczak, 2020. "Mechanism Design With Aftermarkets: Cutoff Mechanisms," Econometrica, Econometric Society, vol. 88(6), pages 2629-2661, November.
    21. H. Moulin, 1980. "On strategy-proofness and single peakedness," Public Choice, Springer, vol. 35(4), pages 437-455, January.
    22. Becker, Bo & Milbourn, Todd, 2011. "How did increased competition affect credit ratings?," Journal of Financial Economics, Elsevier, vol. 101(3), pages 493-514, September.
    23. Dirk Bergemann & Stephen Morris, 2019. "Information Design: A Unified Perspective," Journal of Economic Literature, American Economic Association, vol. 57(1), pages 44-95, March.
    24. Paul R. Milgrom, 1981. "Good News and Bad News: Representation Theorems and Applications," Bell Journal of Economics, The RAND Corporation, vol. 12(2), pages 380-391, Autumn.
    25. Crawford, Vincent P & Sobel, Joel, 1982. "Strategic Information Transmission," Econometrica, Econometric Society, vol. 50(6), pages 1431-1451, November.
    26. Rochet, Jean-Charles, 1987. "A necessary and sufficient condition for rationalizability in a quasi-linear context," Journal of Mathematical Economics, Elsevier, vol. 16(2), pages 191-200, April.
    27. Shengwu Li, 2017. "Obviously Strategy-Proof Mechanisms," American Economic Review, American Economic Association, vol. 107(11), pages 3257-3287, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Doherty, Neil A. & Kartasheva, Anastasia V. & Phillips, Richard D., 2012. "Information effect of entry into credit ratings market: The case of insurers' ratings," Journal of Financial Economics, Elsevier, vol. 106(2), pages 308-330.
    2. Francesco Sangiorgi & Chester Spatt, 2017. "Opacity, Credit Rating Shopping, and Bias," Management Science, INFORMS, vol. 63(12), pages 4016-4036, December.
    3. Josheski Dushko & Karamazova Elena, 2021. "Auction theory and a note on game mechanisms," Croatian Review of Economic, Business and Social Statistics, Sciendo, vol. 7(1), pages 43-59, May.
    4. Strausz, Roland, 2006. "Deterministic versus stochastic mechanisms in principal-agent models," Journal of Economic Theory, Elsevier, vol. 128(1), pages 306-314, May.
    5. Laffont, Jean-Jacques & Pouyet, Jerome, 2004. "The subsidiarity bias in regulation," Journal of Public Economics, Elsevier, vol. 88(1-2), pages 255-283, January.
    6. Andreas A. Haupt & Nicole Immorlica & Brendan Lucier, 2023. "Certification Design for a Competitive Market," Papers 2301.13449, arXiv.org.
    7. X. Ruiz del Portal, 2012. "Conditions for incentive compatibility in models with multidimensional allocation functions and one-dimensional types," Review of Economic Design, Springer;Society for Economic Design, vol. 16(4), pages 311-321, December.
    8. Eduardo Perez & Delphine Prady, 2012. "Complicating to Persuade?," Working Papers hal-03583827, HAL.
    9. Hagenbach, Jeanne & Perez-Richet, Eduardo, 2018. "Communication with evidence in the lab," Games and Economic Behavior, Elsevier, vol. 112(C), pages 139-165.
    10. Winand Emons & Claude Fluet, 2019. "Strategic communication with reporting costs," Theory and Decision, Springer, vol. 87(3), pages 341-363, October.
    11. Ian Ball & Deniz Kattwinkel, 2019. "Probabilistic Verification in Mechanism Design," Papers 1908.05556, arXiv.org, revised Jan 2025.
    12. Mohamed Mostagir & James Siderius, 2023. "Strategic Reviews," Management Science, INFORMS, vol. 69(2), pages 904-921, February.
    13. , & Frechette, Guilaume & Perego, Jacopo, 2019. "Rules and Commitment in Communication," CEPR Discussion Papers 14085, Centre for Economic Policy Research.
    14. Eduardo M Azevedo & Eric Budish, 2019. "Strategy-proofness in the Large," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 86(1), pages 81-116.
    15. Ian Ball & Deniz Kattwinkel, 2019. "Probabilistic Verification in Mechanism Design," CRC TR 224 Discussion Paper Series crctr224_2019_124, University of Bonn and University of Mannheim, Germany.
    16. Persson, Petra, 2018. "Attention manipulation and information overload," Behavioural Public Policy, Cambridge University Press, vol. 2(1), pages 78-106, May.
    17. Eduardo Perez & Delphine Prady, 2012. "Complicating to Persuade?," Working Papers hal-03583827, HAL.
    18. Shuo Liu & Dimitri Migrow, 2019. "Designing organizations in volatile markets," ECON - Working Papers 319, Department of Economics - University of Zurich.
    19. Xiao-Jun Zhang, 2012. "Information relevance, reliability and disclosure," Review of Accounting Studies, Springer, vol. 17(1), pages 189-226, March.
    20. Bester, Helmut & Strausz, Roland, 2007. "Contracting with imperfect commitment and noisy communication," Journal of Economic Theory, Elsevier, vol. 136(1), pages 236-259, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2605.07671. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.