IDEAS home Printed from https://ideas.repec.org/a/eee/ecmode/v28y2011i4p1739-1746.html
   My bibliography  Save this article

Now, whose schools are really better (or weaker) than Germany's? A multiple testing approach

Author

Listed:
  • Hanck, Christoph

Abstract

Using PIRLS (Progress in International Reading Literacy Study) data, we investigate which countries' schools can be classified as significantly better or weaker than Germany's as regards the reading literacy of primary school children. The 'standard' approach is to conduct separate tests for each country relative to the reference country (Germany) and to reject the null of equally good schools for all those countries whose p-value satisfies pi [less-than-or-equals, slant] 0.05. We demonstrate that this approach ignores the multiple testing nature of the problem and thus overstates differences between schooling systems by producing unwarranted rejections of the null. We employ various multiple testing techniques to remedy this problem. The results suggest that the 'standard' approach may overstate the number of significantly different countries by up to 30%.

Suggested Citation

  • Hanck, Christoph, 2011. "Now, whose schools are really better (or weaker) than Germany's? A multiple testing approach," Economic Modelling, Elsevier, vol. 28(4), pages 1739-1746, July.
  • Handle: RePEc:eee:ecmode:v:28:y:2011:i:4:p:1739-1746
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0264999311000708
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yoav Benjamini & Abba M. Krieger & Daniel Yekutieli, 2006. "Adaptive linear step-up procedures that control the false discovery rate," Biometrika, Biometrika Trust, vol. 93(3), pages 491-507, September.
    2. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    3. Wo[ss]mann, Ludger & West, Martin, 2006. "Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS," European Economic Review, Elsevier, vol. 50(3), pages 695-736, April.
    4. Eric A. Hanushek & Ludger Wössmann, 2006. "Does Educational Tracking Affect Performance and Inequality? Differences- in-Differences Evidence Across Countries," Economic Journal, Royal Economic Society, vol. 116(510), pages 63-76, March.
    5. Robert J. Barro, 1991. "Economic Growth in a Cross Section of Countries," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(2), pages 407-443.
    6. Savin, N.E., 1984. "Multiple hypothesis testing," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 14, pages 827-879, Elsevier.
    7. John D. Storey & Jonathan E. Taylor & David Siegmund, 2004. "Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(1), pages 187-205, February.
    8. Spanos, Aris, 2010. "Statistical adequacy and the trustworthiness of empirical evidence: Statistical vs. substantive information," Economic Modelling, Elsevier, vol. 27(6), pages 1436-1452, November.
    9. Deckers, Thomas & Hanck, Christoph, 2009. "Multiple Testing Techniques in Growth Econometrics," MPRA Paper 17843, University Library of Munich, Germany.
    10. Romano, Joseph P. & Shaikh, Azeem M. & Wolf, Michael, 2008. "Formalized Data Snooping Based On Generalized Error Rates," Econometric Theory, Cambridge University Press, vol. 24(2), pages 404-447, April.
    11. Alan M. Taylor, 2002. "A Century Of Purchasing-Power Parity," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 139-150, February.
    12. Hendrik Jürges & Kerstin Schneider, 2008. "Fair ranking of teachers," Studies in Empirical Economics, in: Christian Dustmann & Bernd Fitzenberger & Stephen Machin (ed.), The Economics of Education and Training, pages 157-177, Springer.
    13. Woo, Jaejoon, 2003. "Economic, political, and institutional determinants of public deficits," Journal of Public Economics, Elsevier, vol. 87(3-4), pages 387-426, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Milda Norkute, 2015. "Can the sectoral New Keynesian Phillips curve explain inflation dynamics in the Euro Area?," Empirical Economics, Springer, vol. 49(4), pages 1191-1216, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hanck, Christoph, 2008. "Now, whose schools are really better (or weaker) than Germany's? A multiple testing approach," MPRA Paper 12008, University Library of Munich, Germany.
    2. Deckers, Thomas & Hanck, Christoph, 2009. "Multiple Testing Techniques in Growth Econometrics," MPRA Paper 17843, University Library of Munich, Germany.
    3. Nik Tuzov & Frederi Viens, 2011. "Mutual fund performance: false discoveries, bias, and power," Annals of Finance, Springer, vol. 7(2), pages 137-169, May.
    4. Smeekes, S., 2011. "Bootstrap sequential tests to determine the stationary units in a panel," Research Memorandum 003, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    5. Bajgrowicz, Pierre & Scaillet, Olivier, 2012. "Technical trading revisited: False discoveries, persistence tests, and transaction costs," Journal of Financial Economics, Elsevier, vol. 106(3), pages 473-491.
    6. Joseph Romano & Azeem Shaikh & Michael Wolf, 2008. "Control of the false discovery rate under dependence using the bootstrap and subsampling," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(3), pages 417-442, November.
    7. Hanck, Christoph, 2006. "For Which Countries did PPP hold? A Multiple Testing Approach," Technical Reports 2006,47, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    8. Georgios Sermpinis & Arman Hassanniakalager & Charalampos Stasinakis & Ioannis Psaradellis, 2018. "Technical Analysis and Discrete False Discovery Rate: Evidence from MSCI Indices," Papers 1811.06766, arXiv.org, revised Jun 2019.
    9. Joseph P. Romano & Azeem M. Shaikh & Michael Wolf, 2010. "Hypothesis Testing in Econometrics," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 75-104, September.
    10. Laurent Barras & Olivier Scaillet & Russ Wermers, 2010. "False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas," Journal of Finance, American Finance Association, vol. 65(1), pages 179-216, February.
    11. Christoph Hanck, 2009. "For which countries did PPP hold? A multiple testing approach," Empirical Economics, Springer, vol. 37(1), pages 93-103, September.
    12. Stephen A. Gorman & Frank J. Fabozzi, 2021. "The ABC’s of the alternative risk premium: academic roots," Journal of Asset Management, Palgrave Macmillan, vol. 22(6), pages 405-436, October.
    13. Martin Schlotter & Guido Schwerdt & Ludger Woessmann, 2011. "Econometric methods for causal evaluation of education policies and practices: a non-technical guide," Education Economics, Taylor & Francis Journals, vol. 19(2), pages 109-137.
    14. Charles R. Hulten & Robert M. Schwab, 1993. "Endogenous Growth, Public Capital, and the Convergence of Regional Manufacturing Industries," NBER Working Papers 4538, National Bureau of Economic Research, Inc.
    15. Christoph Hanck, 2013. "An Intersection Test for Panel Unit Roots," Econometric Reviews, Taylor & Francis Journals, vol. 32(2), pages 183-203, February.
    16. Felipe A. Dunsch & David K. Evans & Ezinne Eze-Ajoku & Mario Macis, 2017. "Management, Supervision, and Health Care: A Field Experiment," NBER Working Papers 23749, National Bureau of Economic Research, Inc.
    17. Ghosh Debashis, 2012. "Incorporating the Empirical Null Hypothesis into the Benjamini-Hochberg Procedure," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-21, July.
    18. Roller, Marcus & Steinberg, Daniel, 2020. "The distributional effects of early school stratification - non-parametric evidence from Germany," European Economic Review, Elsevier, vol. 125(C).
    19. David R. Bell & Olivier Ledoit & Michael Wolf, 2012. "A new portfolio formation approach to mispricing of marketing performance indicators with an application to customer satisfaction," ECON - Working Papers 079, Department of Economics - University of Zurich, revised Dec 2013.
    20. Sermpinis, Georgios & Hassanniakalager, Arman & Stasinakis, Charalampos & Psaradellis, Ioannis, 2021. "Technical analysis profitability and Persistence: A discrete false discovery approach on MSCI indices," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 73(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecmode:v:28:y:2011:i:4:p:1739-1746. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/30411 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.