IDEAS home Printed from https://ideas.repec.org/p/vie/viennp/1512.html
   My bibliography  Save this paper

Who gives Direction to Statistical Testing? Best Practice meets Mathematically Correct Tests

Author

Abstract

We are interested in statistical tests that are able to uncover that one method is better than another one. The Wilcoxon-Mann-Whitney rank-sum and the Wilcoxon sign-rank test are the most popular tests for showing that two methods are di¤erent. Yet all of the 32 papers in Economics we surveyed misused them to claim evidence that one method is better, without making any additional assumptions. We present eight nonparametric tests that can correctly identify which method is better in terms of a stochastic inequality, median di¤erence and di¤erence in medians or means, without adding any as- sumptions. We show that they perform very well in the data sets from the surveyed papers. The two tests for comparing medians are novel, constructed in the spirit of Mood?s test.

Suggested Citation

  • Karl H.Schlag, 2015. "Who gives Direction to Statistical Testing? Best Practice meets Mathematically Correct Tests," Vienna Economics Papers 1512, University of Vienna, Department of Economics.
  • Handle: RePEc:vie:viennp:1512
    as

    Download full text from publisher

    File URL: http://homepage.univie.ac.at/Papers.Econ/RePEc/vie/viennp/vie1512.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Jessica Cohen & Pascaline Dupas, 2008. "Free Distribution or Cost-Sharing? Evidence from a Malaria Prevention Experiment," NBER Working Papers 14406, National Bureau of Economic Research, Inc.
    2. Friedman, Daniel & Huck, Steffen & Oprea, Ryan & Weidenholzer, Simon, 2015. "From imitation to collusion: Long-run learning in a low-information environment," Journal of Economic Theory, Elsevier, vol. 155(C), pages 185-205.
    3. Masella, Paolo & Meier, Stephan & Zahn, Philipp, 2014. "Incentives and group identity," Games and Economic Behavior, Elsevier, vol. 86(C), pages 12-25.
    4. Charness, Gary & Cobo-Reyes, Ramón & Jiménez, Natalia, 2014. "Identities, selection, and contributions in a public-goods game," Games and Economic Behavior, Elsevier, vol. 87(C), pages 322-338.
    5. Khalmetski, Kiryl & Ockenfels, Axel & Werner, Peter, 2015. "Surprising gifts: Theory and laboratory evidence," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 163-208.
    6. Dufour, J-M. & Hallin, M., 1990. "Improved Eaton Bounds for Linear Combinations of Bounded Random Variables , with Statistical Applications," Papers 9104, Universite Libre de Bruxelles - C.E.M.E..
    7. Joseph P. Romano, 2004. "On Non-parametric Testing, the Uniform Behaviour of the "t"-test, and Related Problems," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 31(4), pages 567-584.
    8. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 1-45.
    9. Hugh-Jones, David & Kurino, Morimitsu & Vanberg, Christoph, 2014. "An experimental study on the incentives of the probabilistic serial mechanism," Games and Economic Behavior, Elsevier, vol. 87(C), pages 367-380.
    10. Elena Asparouhova & Peter Bossaerts & Jon Eguia & William Zame, 2015. "Asset Pricing and Asymmetric Reasoning," Journal of Political Economy, University of Chicago Press, vol. 123(1), pages 66-122.
    11. Dittmann, Ingolf & Kübler, Dorothea & Maug, Ernst & Mechtenberg, Lydia, 2014. "Why votes have value: Instrumental voting with overconfidence and overestimation of others' errors," Games and Economic Behavior, Elsevier, vol. 84(C), pages 17-38.
    12. Leonard Wantchekon & Marko Klašnja & Natalija Novta, 2015. "Education and Human Capital Externalities: Evidence from Colonial Benin," The Quarterly Journal of Economics, Oxford University Press, vol. 130(2), pages 703-757.
    13. Charles N. Noussair & Stefan T. Trautmann & Gijs van de Kuilen, 2014. "Higher Order Risk Attitudes, Demographics, and Financial Decisions," Review of Economic Studies, Oxford University Press, vol. 81(1), pages 325-355.
    14. Alain Cohn & Jan Engelmann & Ernst Fehr & Michel André Maréchal, 2015. "Evidence for Countercyclical Risk Aversion: An Experiment with Financial Professionals," American Economic Review, American Economic Association, vol. 105(2), pages 860-885, February.
    15. Forsythe Robert & Horowitz Joel L. & Savin N. E. & Sefton Martin, 1994. "Fairness in Simple Bargaining Experiments," Games and Economic Behavior, Elsevier, vol. 6(3), pages 347-369, May.
    16. Bhattacharya, Sourav & Duffy, John & Kim, Sun-Tak, 2014. "Compulsory versus voluntary voting: An experimental study," Games and Economic Behavior, Elsevier, vol. 84(C), pages 111-131.
    17. Ockenfels, Axel & Selten, Reinhard, 2014. "Impulse balance in the newsvendor game," Games and Economic Behavior, Elsevier, vol. 86(C), pages 237-247.
    18. Cabral, Luis & Ozbay, Erkut Y. & Schotter, Andrew, 2014. "Intrinsic and instrumental reciprocity: An experimental study," Games and Economic Behavior, Elsevier, vol. 87(C), pages 100-121.
    19. Luba Petersen & Abel Winn, 2014. "Does Money Illusion Matter? Comment," American Economic Review, American Economic Association, vol. 104(3), pages 1047-1062, March.
    20. Chen, Yan & Li, Sherry Xin & Liu, Tracy Xiao & Shih, Margaret, 2014. "Which hat to wear? Impact of natural identities on coordination and cooperation," Games and Economic Behavior, Elsevier, vol. 84(C), pages 58-86.
    21. Karl Schlag, 2008. "A new method for constructing exact tests without making any assumptions," Economics Working Papers 1109, Department of Economics and Business, Universitat Pompeu Fabra.
    22. Andrea Isoni & Anders Poulsen & Robert Sugden & Kei Tsutsui, 2014. "Efficiency, Equality, and Labeling: An Experimental Investigation of Focal Points in Explicit Bargaining," American Economic Review, American Economic Association, vol. 104(10), pages 3256-3287, October.
    23. Michael Kosfeld & Devesh Rustagi, 2015. "Leader Punishment and Cooperation in Groups: Experimental Field Evidence from Commons Management in Ethiopia," American Economic Review, American Economic Association, vol. 105(2), pages 747-783, February.
    24. Filiz-Ozbay, Emel & Lopez-Vargas, Kristian & Ozbay, Erkut Y., 2015. "Multi-object auctions with resale: Theory and experiment," Games and Economic Behavior, Elsevier, vol. 89(C), pages 1-16.
    25. Cheung, Stephen L., 2012. "Risk Preferences Are Not Time Preferences: Comment," IZA Discussion Papers 6762, Institute for the Study of Labor (IZA).
    26. John Duffy & Daniela Puzzello, 2014. "Gift Exchange versus Monetary Exchange: Theory and Evidence," American Economic Review, American Economic Association, vol. 104(6), pages 1735-1776, June.
    27. Thomas Markussen & Louis Putterman & Jean-Robert Tyran, 2014. "Self-Organization for Collective Action: An Experimental Study of Voting on Sanction Regimes," Review of Economic Studies, Oxford University Press, vol. 81(1), pages 301-324.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Francesco, De Sinopoli & Giovanna, Iannantuoni & Valeria, Maggian & Stefania, Ottone, 2018. "A Two-Party System under the Proportional Rule is Possible: Strategic Voting in the Lab," Working Papers 381, University of Milano-Bicocca, Department of Economics, revised 16 May 2018.

    More about this item

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vie:viennp:1512. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Paper Administrator). General contact details of provider: http://www.univie.ac.at/vwl .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.