Strategic Manipulation of Empirical Tests
Theories can be produced by individuals seeking a good reputation of knowledge. Hence, a significant question is how to test theories anticipating that they might have been produced by (potentially uninformed) experts who prefer their theories not to be rejected. If a theory that predicts exactly like the data generating process is not rejected with high probability then the test is said to not reject the truth. On the other hand, if a false expert, with no knowledge over the data generating process, can strategically select theories that will not be rejected then the test can be ignorantly passed. These tests have limited use because they cannot feasibly dismiss completely uninformed experts. Many tests proposed in the literature (e.g., calibration tests) can be ignorantly passed. Dekel and Feinberg (2006) introduced a class of tests that seemingly have some power of dismissing uninformed experts. We show that some tests from their class can also be ignorantly passed. One of those tests, however, does not reject the truth and cannot be ignorantly passed. Thus, this empirical test can dismiss false experts.We also show that a false reputation of knowledge can be strategically sustained for an arbitrary, but given, number of periods, no matted which test is used (provided that it does not reject the truth). However, false experts can be discredited, even with bounded data sets, if the domain of permissible theories is mildly restricted.
|Date of creation:||Mar 2006|
|Date of revision:|
|Contact details of provider:|| Postal: Center for Mathematical Studies in Economics and Management Science, Northwestern University, 580 Jacobs Center, 2001 Sheridan Road, Evanston, IL 60208-2014|
Web page: http://www.kellogg.northwestern.edu/research/math/
More information through EDIRC
|Order Information:|| Email: |
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Wojciech Olszewski & Alvaro Sandroni, 2008.
"Manipulability of Future-Independent Tests,"
Econometric Society, vol. 76(6), pages 1437-1466, November.
- Drew Fudenberg & David K. Levine, 1997.
"Conditional Universal Consistency,"
Levine's Working Paper Archive
471, David K. Levine.
- Fudenberg, Drew & Levine, David K., 1999.
"An Easier Way to Calibrate,"
Games and Economic Behavior,
Elsevier, vol. 29(1-2), pages 131-137, October.
- Nabil I. Al-Najjar & Jonathan Weinstein, 2008.
"Comparative Testing of Experts,"
Econometric Society, vol. 76(3), pages 541-559, 05.
- Alvaro Sandroni & Wojciech Olszewski, 2008. "Falsifiability," PIER Working Paper Archive 08-016, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
- Alvaro Sandroni, 2003. "The reproducible properties of correct forecasts," International Journal of Game Theory, Springer;Game Theory Society, vol. 32(1), pages 151-159, December.
- Vladimir Vovk & Glenn Shafer, 2005. "Good randomized sequential probability forecasting is always possible," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 747-763.
- Fudenberg, Drew & Levine, David K., 1995.
"Consistency and cautious fictitious play,"
Journal of Economic Dynamics and Control,
Elsevier, vol. 19(5-7), pages 1065-1089.
- Sergiu Hart & Andreu Mas-Colell, 1997.
"A Simple Adaptive Procedure Leading to Correlated Equilibrium,"
Game Theory and Information
9703006, EconWPA, revised 24 Mar 1997.
- Sergiu Hart & Andreu Mas-Colell, 2000. "A Simple Adaptive Procedure Leading to Correlated Equilibrium," Econometrica, Econometric Society, vol. 68(5), pages 1127-1150, September.
- S. Hart & A. Mas-Collel, 2010. "A Simple Adaptive Procedure Leading to Correlated Equilibrium," Levine's Working Paper Archive 572, David K. Levine.
- Sergiu Hart & Andreu Mas-Colell, 1996. "A simple adaptive procedure leading to correlated equilibrium," Economics Working Papers 200, Department of Economics and Business, Universitat Pompeu Fabra, revised Dec 1996.
- Ehud Lehrer & Eilon Solan, 2003. "No-Regret with Bounded Computational Capacity," Discussion Papers 1373, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
- Sergiu Hart & Andreu Mas-Colell, 1999.
"A General Class of Adaptive Strategies,"
Game Theory and Information
9904001, EconWPA, revised 23 Mar 2000.
- Ehud Kalai, 1995.
"Calibrated Forecasting and Merging,"
1144, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
- Ehud Kalai & Ehud Lehrer & Rann Smorodinsky, 2010. "Calibrated Forecasting and Merging," Levine's Working Paper Archive 584, David K. Levine.
- Ehud Kalai, 1995. "Calibrated Forecasting and Merging," Discussion Papers 1144R, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
- Yossi Feinberg & Colin Stewart, 2008.
"Testing Multiple Forecasters,"
Econometric Society, vol. 76(3), pages 561-582, 05.
- Eddie Dekel & Yossi Feinberg, 2006.
"Non-Bayesian Testing of a Stochastic Prediction,"
1418, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
- Lehrer, Ehud, 2001. "Any Inspection Is Manipulable," Econometrica, Econometric Society, vol. 69(5), pages 1333-47, September.
- Shmaya, Eran, 2008. "Many inspections are manipulable," Theoretical Economics, Econometric Society, vol. 3(3), September.
- Rustichini, Aldo, 1999. "Optimal Properties of Stimulus--Response Learning Models," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 244-273, October.
- Anderson Robert M. & Zame William R., 2001. "Genericity with Infinitely Many Parameters," The B.E. Journal of Theoretical Economics, De Gruyter, vol. 1(1), pages 1-64, February.
When requesting a correction, please mention this item's handle: RePEc:nwu:cmsems:1425. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Fran Walker)
If references are entirely missing, you can add them using this form.