IDEAS home Printed from https://ideas.repec.org/p/pui/dpaper/256.html

Do Publication Metrics Distort Research Effort? Bunching Evidence from Thailand’s 2019 Higher-education Reforms

Author

Listed:
  • Nattavudh Powdthavee

Abstract

Many universities in middle-income countries lack the peer-review infrastructure to assess research quality directly and instead tie financial rewards to publication in journals classified as Q1 under the SCImago Journal Rank system. By converting continuously varying journal quality into discrete institutional categories, these systems create sharp incentive discontinuities at quartile boundaries. In a preregistered study, we apply a bunching estimator — a method from public finance that detects excess concentration of observations around institutionally salient thresholds — to 149,402 Scopus-indexed publications from Thailand over 2016–2025, exploiting Thailand's 2019 higher-education reform as a source of temporal variation. We find no significant bunching before 2019 but substantial excess concentration immediately above the Q1 boundary afterwards — a pattern not observed in Singapore, whose publication environment is not organised around explicit quartile-based financial rewards. The post-reform excess mass corresponds to roughly 1,575 additional publications over 2020–2025, implying an estimated 39 million THB (approximately US$1.1 million) in cumulative institutional expenditure. These findings indicate that discrete quartile-based reward systems redirect research effort towards threshold optimisation rather than research quality, and that replacing binary quartile rewards with continuous percentile-based incentives would better align institutional evaluation with scientific output.

Suggested Citation

  • Nattavudh Powdthavee, 2026. "Do Publication Metrics Distort Research Effort? Bunching Evidence from Thailand’s 2019 Higher-education Reforms," PIER Discussion Papers 256, Puey Ungphakorn Institute for Economic Research.
  • Handle: RePEc:pui:dpaper:256
    as

    Download full text from publisher

    File URL: https://www.pier.or.th/files/dp/pier_dp_256.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Raj Chetty & John N. Friedman & Tore Olsen & Luigi Pistaferri, 2011. "Adjustment Costs, Firm Responses, and Micro vs. Macro Labor Supply Elasticities: Evidence from Danish Tax Records," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(2), pages 749-804.
    2. Hudson, Robert, 2024. "Responding to incentives or gaming the system? How UK business academics respond to the Academic Journal Guide," Research Policy, Elsevier, vol. 53(9).
    3. Zhuanlan Sun & Chenwei Zhang & Ka Lok Pang & Ying Tang & Yiwei Li, 2024. "Do Changes in Journal Rank Influence Publication Output? Evidence from China," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 7035-7054, November.
    4. Piotr Śpiewanowski & Oleksandr Talavera, 2021. "Journal rankings and publication strategy," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3227-3242, April.
    5. Strathern, Marilyn, 1997. "‘Improving ratings’: audit in the British University system," European Review, Cambridge University Press, vol. 5(3), pages 305-321, July.
    6. Andrew Gelman & Guido Imbens, 2019. "Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(3), pages 447-456, July.
    7. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    8. Franzoni, Chiara & Sauermann, Henry, 2014. "Crowd science: The organization of scientific research in open collaborative projects," Research Policy, Elsevier, vol. 43(1), pages 1-20.
    9. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Powdthavee, Nattavudh, 2026. "Do Publication Metrics Distort Research Effort? Bunching Evidence from Thailand’s 2019 Higher-Education Reforms," IZA Discussion Papers 18680, IZA Network @ LISER.
    2. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.
    3. Morgan-Thomas, Anna & Tsoukas, Serafeim & Dudau, Adina & Gąska, Paweł, 2024. "Beyond declarations: Metrics, rankings and responsible assessment," Research Policy, Elsevier, vol. 53(10).
    4. van den Besselaar, Peter & Heyman, Ulf & Sandström, Ulf, 2017. "Perverse effects of output-based research funding? Butler’s Australian case revisited," Journal of Informetrics, Elsevier, vol. 11(3), pages 905-918.
    5. Konstantin Fursov & Yana Roschina & Oksana Balmush, 2016. "Determinants of Research Productivity: An Individual-level Lens," Foresight and STI Governance, National Research University Higher School of Economics, vol. 10(2), pages 44-56.
    6. Alexander I. Terekhov, 2017. "Bibliometric spectroscopy of Russia’s nanotechnology: 2000–2014," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1217-1242, March.
    7. Dzieżyc, Maciej & Kazienko, Przemysław, 2022. "Effectiveness of research grants funded by European Research Council and Polish National Science Centre," Journal of Informetrics, Elsevier, vol. 16(1).
    8. Kulczycki, Emanuel & Korzeń, Marcin & Korytkowski, Przemysław, 2017. "Toward an excellence-based research funding system: Evidence from Poland," Journal of Informetrics, Elsevier, vol. 11(1), pages 282-298.
    9. repec:osf:socarx:h2fxp_v1 is not listed on IDEAS
    10. Jan Schulz, 2016. "Using Monte Carlo simulations to assess the impact of author name disambiguation quality on different bibliometric analyses," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1283-1298, June.
    11. David A. Pendlebury, 2019. "Charting a path between the simple and the false and the complex and unusable: Review of Henk F. Moed, Applied Evaluative Informetrics [in the series Qualitative and Quantitative Analysis of Scientific and Scholarly Communication, Wolfgang Glänzel, A," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 549-560, April.
    12. Nicky Agate & Rebecca Kennison & Stacy Konkiel & Christopher P. Long & Jason Rhody & Simone Sacchi & Penelope Weber, 2020. "The transformative power of values-enacted scholarship," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 7(1), pages 1-12, December.
    13. Liu, Xiaojuan & Xiang, Nannan & Huang, Mu-hsuan, 2026. "How highly cited papers affect LIS journal impact factor? An empirical study," Journal of Informetrics, Elsevier, vol. 20(1).
    14. Rafols, Ismael & Stirling, Andy, 2020. "Designing indicators for opening up evaluation. Insights from research assessment," SocArXiv h2fxp, Center for Open Science.
    15. Benedetto Lepori & Aldo Geuna & Antonietta Mira, 2019. "Scientific output scales with resources. A comparison of US and European universities," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-18, October.
    16. Cappelletti-Montano, Beniamino & Columbu, Silvia & Montaldo, Stefano & Musio, Monica, 2021. "New perspectives in bibliometric indicators: Moving from citations to citing authors," Journal of Informetrics, Elsevier, vol. 15(3).
    17. Emanuel Kulczycki & Raf Guns & Janne Pölönen & Tim C. E. Engels & Ewa A. Rozkosz & Alesia A. Zuccala & Kasper Bruun & Olli Eskola & Andreja Istenič Starčič & Michal Petr & Gunnar Sivertsen, 2020. "Multilingual publishing in the social sciences and humanities: A seven‐country European study," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(11), pages 1371-1385, November.
    18. Enrico Rubolino, 2022. "Taxing the Gender Gap: Labor Market Effects of a Payroll Tax Cut for Women in Italy," Cahiers de Recherches Economiques du Département d'économie 22.01, Université de Lausanne, Faculté des HEC, Département d’économie.
    19. Derek Messacar, 2022. "Loss-Averse Tax Manipulation and Tax-Preferred Savings," Cahiers de recherche / Working Papers 8, Institut sur la retraite et l'épargne / Retirement and Savings Institute.
    20. Marlo M. Vernon & Frances M. Yang, 2023. "Use of Latent Profile Analysis to Model the Translation of University Research into Health Practice and Policy: Exploration of Proposed Metrics," Research in Higher Education, Springer;Association for Institutional Research, vol. 64(7), pages 1058-1070, November.
    21. Su, Yang & Brooks, Chris & Lucey, Brian & Urquhart, Andrew, 2026. "Journal ratings changes: Implications for author diversity and research characteristics," Research Policy, Elsevier, vol. 55(2).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • O38 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Government Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pui:dpaper:256. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/pierbth.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.