IDEAS home Printed from https://ideas.repec.org/p/unu/wpaper/wp2013-068.html
   My bibliography  Save this paper

Evaluating Governance Indexes: Critical and Less Critical Questions

Author

Listed:
  • Gisselquist, Rachel M.

Abstract

Recent years have seen a proliferation of `composite indicators´ or `indexes´ of governance. Such measures can be useful tools for analysing governance, making public policy, building scientific knowledge, and even influencing ruling elites, but some are better tools than others and some are better suited to certain purposes than others. This paper provides a framework of ten questions to help users and producers of governance indexes to evaluate them and consider key components of index design. In reviewing these ten questions–only six of which, it argues, are critical–the paper offers examples from some of the best known measures of governance and related topics. It advances two broad arguments: First, more attention should be paid to the fundamentals of social science methodology, i.e., questions about concept formation, content validity, reliability, replicability, robustness, and the relevance of particular measures to underlying research questions. Second, less attention should be paid to some other issues commonly highlighted in the literature on governance measurement, i.e., questions about descriptive complexity, theoretical fit, the precision of estimates, and correct weighting. The paper builds upon a thorough review of the literature and the author´s three years of research in practice as co-author of a well-known governance index.

Suggested Citation

  • Gisselquist, Rachel M., 2013. "Evaluating Governance Indexes: Critical and Less Critical Questions," WIDER Working Paper Series 068, World Institute for Development Economic Research (UNU-WIDER).
  • Handle: RePEc:unu:wpaper:wp2013-068
    as

    Download full text from publisher

    File URL: https://www.wider.unu.edu/sites/default/files/WP2013-068.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Paolo Paruolo & Michaela Saisana & Andrea Saltelli, 2013. "Ratings and rankings: voodoo or science?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 609-634, June.
    2. Knoll, Martin & Zloczysti, Petra, 2012. "The Good Governance Indicators of the Millennium Challenge Account: How Many Dimensions are Really Being Measured?," World Development, Elsevier, vol. 40(5), pages 900-915.
    3. Matt Andrews & Roger Hay & Jerrett Myers, 2010. "Can Governance Indicators Make Sense? Towards a New Approach to Sector-Specific Measures of Governance," Oxford Development Studies, Taylor & Francis Journals, vol. 38(4), pages 391-410.
    4. Gingerich, Daniel W., 2013. "Governance Indicators and the Level of Analysis Problem: Empirical Findings from South America," British Journal of Political Science, Cambridge University Press, vol. 43(03), pages 505-540, July.
    5. Resnick, Danielle & Birner, Regina, 2006. "Does good governance contribute to pro-poor growth?: a review of the evidence from cross-country studies," DSGD discussion papers 30, International Food Policy Research Institute (IFPRI).
    6. M. Saisana & A. Saltelli & S. Tarantola, 2005. "Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 307-323.
    7. Round, Jeffery I., 2012. "Aid and Investment in Statistics for Africa," WIDER Working Paper Series 093, World Institute for Development Economic Research (UNU-WIDER).
    8. Gisselquist, Rachel M., 2012. "Good Governance as a Concept, and Why This Matters for Development Policy," WIDER Working Paper Series 030, World Institute for Development Economic Research (UNU-WIDER).
    9. repec:unu:wpaper:wp2012-30 is not listed on IDEAS
    10. Harttgen, Kenneth & Klasen, Stephan, 2012. "A Household-Based Human Development Index," World Development, Elsevier, vol. 40(5), pages 878-899.
    11. Elizabeth Stanton, 2007. "The Human Development Index: A History," Working Papers wp127, Political Economy Research Institute, University of Massachusetts at Amherst.
    12. Kaufmann, Daniel & Kraay, Aart & Zoido-Lobaton, Pablo, 1999. "Aggregating governance indicators," Policy Research Working Paper Series 2195, The World Bank.
    13. Charles P. Oman & Christiane Arndt, 2010. "Measuring Governance," OECD Development Centre Policy Briefs 39, OECD Publishing.
    14. Farhad Noorbakhsh, 1998. "The human development index: some technical issues and alternative indices," Journal of International Development, John Wiley & Sons, Ltd., vol. 10(5), pages 589-605.
    15. repec:cup:apsrev:v:89:y:1995:i:02:p:454-456_09 is not listed on IDEAS
    16. Matt Andrews, 2008. "The Good Governance Agenda: Beyond Indicators without Theory," Oxford Development Studies, Taylor & Francis Journals, vol. 36(4), pages 379-407.
    17. Williams, Andrew, 2009. "On the release of information by governments: Causes and consequences," Journal of Development Economics, Elsevier, vol. 89(1), pages 124-138, May.
    18. Andrea Saltelli, 2007. "Composite Indicators between Analysis and Advocacy," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 81(1), pages 65-77, March.
    19. Merilee S. Grindle, 2007. "Good Enough Governance Revisited," Development Policy Review, Overseas Development Institute, vol. 25(5), pages 533-574, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michener, Gregory, 2015. "Policy Evaluation via Composite Indexes: Qualitative Lessons from International Transparency Policy Indexes," World Development, Elsevier, vol. 74(C), pages 184-196.

    More about this item

    Keywords

    Index numbers (Economics); Public administration; Methodology (Qualitative research);

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:unu:wpaper:wp2013-068. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mauricio Roa Grisales). General contact details of provider: http://edirc.repec.org/data/widerfi.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.