IDEAS home Printed from https://ideas.repec.org/p/nwu/cmsems/1425.html
   My bibliography  Save this paper

Strategic Manipulation of Empirical Tests

Author

Listed:
  • Wojciech Olszewski
  • Alvaro Sandroni

Abstract

Theories can be produced by individuals seeking a good reputation of knowledge. Hence, a significant question is how to test theories anticipating that they might have been produced by (potentially uninformed) experts who prefer their theories not to be rejected. If a theory that predicts exactly like the data generating process is not rejected with high probability then the test is said to not reject the truth. On the other hand, if a false expert, with no knowledge over the data generating process, can strategically select theories that will not be rejected then the test can be ignorantly passed. These tests have limited use because they cannot feasibly dismiss completely uninformed experts. Many tests proposed in the literature (e.g., calibration tests) can be ignorantly passed. Dekel and Feinberg (2006) introduced a class of tests that seemingly have some power of dismissing uninformed experts. We show that some tests from their class can also be ignorantly passed. One of those tests, however, does not reject the truth and cannot be ignorantly passed. Thus, this empirical test can dismiss false experts.We also show that a false reputation of knowledge can be strategically sustained for an arbitrary, but given, number of periods, no matted which test is used (provided that it does not reject the truth). However, false experts can be discredited, even with bounded data sets, if the domain of permissible theories is mildly restricted.

Suggested Citation

  • Wojciech Olszewski & Alvaro Sandroni, 2006. "Strategic Manipulation of Empirical Tests," Discussion Papers 1425, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
  • Handle: RePEc:nwu:cmsems:1425
    as

    Download full text from publisher

    File URL: http://faculty.wcas.northwestern.edu/~wol737/Test.pdf
    File Function: main text
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Nabil I. Al-Najjar & Jonathan Weinstein, 2008. "Comparative Testing of Experts," Econometrica, Econometric Society, vol. 76(3), pages 541-559, May.
    2. Wojciech Olszewski & Alvaro Sandroni, 2008. "Manipulability of Future-Independent Tests," Econometrica, Econometric Society, vol. 76(6), pages 1437-1466, November.
    3. Fudenberg, Drew & Levine, David K., 1999. "An Easier Way to Calibrate," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 131-137, October.
    4. Fudenberg, Drew & Levine, David K., 1999. "Conditional Universal Consistency," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 104-130, October.
    5. Fudenberg, Drew & Levine, David K., 1995. "Consistency and cautious fictitious play," Journal of Economic Dynamics and Control, Elsevier, vol. 19(5-7), pages 1065-1089.
    6. Alvaro Sandroni, 2003. "The reproducible properties of correct forecasts," International Journal of Game Theory, Springer;Game Theory Society, vol. 32(1), pages 151-159, December.
    7. Sergiu Hart & Andreu Mas-Colell, 2000. "A Simple Adaptive Procedure Leading to Correlated Equilibrium," Econometrica, Econometric Society, vol. 68(5), pages 1127-1150, September.
    8. Hart, Sergiu & Mas-Colell, Andreu, 2001. "A General Class of Adaptive Strategies," Journal of Economic Theory, Elsevier, vol. 98(1), pages 26-54, May.
    9. Vladimir Vovk & Glenn Shafer, 2005. "Good randomized sequential probability forecasting is always possible," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 747-763.
    10. Alvaro Sandroni & Wojciech Olszewski, 2008. "Falsifiability," PIER Working Paper Archive 08-016, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    11. Anderson Robert M. & Zame William R., 2001. "Genericity with Infinitely Many Parameters," The B.E. Journal of Theoretical Economics, De Gruyter, vol. 1(1), pages 1-64, February.
    12. Kalai, Ehud & Lehrer, Ehud & Smorodinsky, Rann, 1999. "Calibrated Forecasting and Merging," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 151-169, October.
    13. Rustichini, Aldo, 1999. "Optimal Properties of Stimulus--Response Learning Models," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 244-273, October.
    14. Ehud Lehrer & Eilon Solan, 2003. "No-Regret with Bounded Computational Capacity," Discussion Papers 1373, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    15. Eddie Dekel & Yossi Feinberg, 2006. "Non-Bayesian Testing of a Stochastic Prediction," Review of Economic Studies, Oxford University Press, vol. 73(4), pages 893-906.
    16. Shmaya, Eran, 2008. "Many inspections are manipulable," Theoretical Economics, Econometric Society, vol. 3(3), September.
    17. Yossi Feinberg & Colin Stewart, 2008. "Testing Multiple Forecasters," Econometrica, Econometric Society, vol. 76(3), pages 561-582, May.
    18. Lehrer, Ehud, 2001. "Any Inspection Is Manipulable," Econometrica, Econometric Society, vol. 69(5), pages 1333-1347, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wojciech Olszewski & Alvaro Sandroni, 2008. "Manipulability of Future-Independent Tests," Econometrica, Econometric Society, vol. 76(6), pages 1437-1466, November.
    2. Dean Foster & Rakesh Vohra, 2011. "Calibration: Respice, Adspice, Prospice," Discussion Papers 1537, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    3. Colin, Stewart, 2011. "Nonmanipulable Bayesian testing," Journal of Economic Theory, Elsevier, vol. 146(5), pages 2029-2041, September.
    4. Francisco Barreras & Álvaro J. Riascos, 2016. "Screening multiple potentially false experts," MONOGRAFÍAS 015075, QUANTIL.
    5. Yossi Feinberg & Colin Stewart, 2008. "Testing Multiple Forecasters," Econometrica, Econometric Society, vol. 76(3), pages 561-582, May.
    6. Feinberg, Yossi & Lambert, Nicolas S., 2011. "Mostly Calibrated," Research Papers 2090, Stanford University, Graduate School of Business.
    7. Hu, Tai Wei & Shmaya, Eran, 2013. "Expressible inspections," Theoretical Economics, Econometric Society, vol. 8(2), May.
    8. Al-Najjar, Nabil & Sandroni, Alvaro, 2013. "A difficulty in the testing of strategic experts," Mathematical Social Sciences, Elsevier, vol. 65(1), pages 5-9.
    9. Francisco Barreras, 2017. "Screening Multiple Uninformed Experts," DOCUMENTOS DE TRABAJO QUANTIL 015282, QUANTIL.
    10. Al-Najjar, Nabil I. & Sandroni, Alvaro & Smorodinsky, Rann & Weinstein, Jonathan, 2010. "Testing theories with learnable and predictive representations," Journal of Economic Theory, Elsevier, vol. 145(6), pages 2203-2217, November.

    More about this item

    JEL classification:

    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nwu:cmsems:1425. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Fran Walker). General contact details of provider: http://edirc.repec.org/data/cmnwuus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.