IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v130y2025i4d10.1007_s11192-025-05272-8.html
   My bibliography  Save this article

The impact of innovation policies on scientometric indicators: a study through the combination of punctuated equilibrium theory and synthetic control method

Author

Listed:
  • Rodrigo Luz Barcellos

    (Universidade Federal do Rio Grande do Sul)

  • Daniel de Abreu Pereira Uhr

    (Universidade Federal de Pelotas)

  • Eliana Lemos Crestani

    (Universidade Federal do Rio Grande do Sul)

  • Lígia Mori Madeira

    (Universidade Federal do Rio Grande do Sul)

  • Marcelo Nogueira Cortimiglia

    (Universidade Federal do Rio Grande do Sul)

Abstract

The study considers the policies and budgetary review of the Brazilian innovation policies to suggest two significant punctuations during the period from 1995 to 2023. Supported by Punctuated Equilibrium Theory (PET), the analysis reveals an expansion phase from 2007 to 2014, followed by a reduction period from 2015 to 2023. To assess the impact of these policy changes, the Synthetic Control Method (SCM) is applied to measure the quantity and quality of documents published by Brazilian authors in the Web of Science database. Overall, the results confirm the expected effects on scientometric indicators: positive impacts during the expansion phase and negative impacts during the reduction phase. Notably, policy expansion had a more pronounced effect on publication volume than on citation impact. During the reduction period, collaborations with international and industrial partners strengthened as an alternative to limited national public funding. Additionally, the study highlights heterogeneous effects over time and demonstrates the efficacy of combining PET with SCM for evaluating public policies using panel data.

Suggested Citation

  • Rodrigo Luz Barcellos & Daniel de Abreu Pereira Uhr & Eliana Lemos Crestani & Lígia Mori Madeira & Marcelo Nogueira Cortimiglia, 2025. "The impact of innovation policies on scientometric indicators: a study through the combination of punctuated equilibrium theory and synthetic control method," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(4), pages 2185-2212, April.
    • Handle: RePEc:spr:scient:v:130:y:2025:i:4:d:10.1007_s11192-025-05272-8
    DOI: 10.1007/s11192-025-05272-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-025-05272-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-025-05272-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Hendrik P. Dalen, 2021. "How the publish-or-perish principle divides a science: the case of economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1675-1694, February.
    2. Vasile Cernat, 2024. "The unprincipled principal: how Romania’s inconsistent research reform impacted scientific output," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(9), pages 5557-5575, September.
    3. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2021. "The Augmented Synthetic Control Method," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1789-1803, October.
    4. Weishu Liu, 2019. "The data source of this study is Web of Science Core Collection? Not enough," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1815-1824, December.
    5. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    6. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    7. Fiammetta Menchetti & Fabrizio Cipollini & Fabrizia Mealli, 2023. "Combining counterfactual outcomes and ARIMA models for policy evaluation," The Econometrics Journal, Royal Economic Society, vol. 26(1), pages 1-24.
    8. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    9. Le Maux, Benoît & Necker, Sarah & Rocaboy, Yvon, 2019. "Cheat or perish? A theory of scientific customs," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    10. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    11. Rodrigo Arocena & Judith Sutz, 2000. "Looking At National Systems Of Innovation From The South," Industry and Innovation, Taylor & Francis Journals, vol. 7(1), pages 55-75.
    12. Concepta McManus & Abilio Afonso Baeta Neves, 2021. "Funding research in Brazil," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 801-823, January.
    13. Guanpeng Yan & Qiang Chen, 2023. "synth2: Synthetic control method with placebo tests, robustness test, and visualization," Stata Journal, StataCorp LLC, vol. 23(3), pages 597-624, September.
    14. Steven Callander & Gregory J. Martin, 2017. "Dynamic Policymaking with Decay," American Journal of Political Science, John Wiley & Sons, vol. 61(1), pages 50-67, January.
    15. Clemente, Jesus & Montanes, Antonio & Reyes, Marcelo, 1998. "Testing for a unit root in variables with a double change in the mean," Economics Letters, Elsevier, vol. 59(2), pages 175-182, May.
    16. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    17. Jones, Bryan D. & Sulkin, Tracy & Larsen, Heather A., 2003. "Policy Punctuations in American Political Institutions," American Political Science Review, Cambridge University Press, vol. 97(1), pages 151-169, February.
    18. James A. Cunningham & Matthias Menter & Chris Young, 2017. "A review of qualitative case methods trends and themes used in technology transfer research," The Journal of Technology Transfer, Springer, vol. 42(4), pages 923-956, August.
    19. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    20. repec:plo:pone00:0166149 is not listed on IDEAS
    21. William W. Hood & Concepción S. Wilson, 2001. "The Literature of Bibliometrics, Scientometrics, and Informetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 52(2), pages 291-314, October.
    22. Junwen Zhu & Weishu Liu, 2020. "A tale of two databases: the use of Web of Science and Scopus in academic papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 321-335, April.
    23. K. Jonkers & G. E. Derrick & C. Lopez-Illescas & P. Besselaar, 2014. "Measuring the scientific impact of e-research infrastructures: a citation based approach?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1179-1194, November.
    24. Jennifer L Woolley & Nydia MacGregor, 2022. "Science, technology, and innovation policy timing and nanotechnology entrepreneurship and innovation," PLOS ONE, Public Library of Science, vol. 17(3), pages 1-24, March.
    25. Jakob Edler & Jan Fagerberg, 2017. "Innovation policy: what, why, and how," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 33(1), pages 2-23.
    26. Abdullah Gök & John Rigby & Philip Shapira, 2016. "The impact of research funding on scientific outputs: Evidence from six smaller European countries," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(3), pages 715-730, March.
    27. Cesar H. Limaymanta & Rosalía Quiroz-de-García & Jesús A. Rivas-Villena & Andrea Rojas-Arroyo & Orlando Gregorio-Chaviano, 2022. "Relationship between collaboration and normalized scientific impact in South American public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6391-6411, November.
    28. Noémi Kreif & Richard Grieve & Dominik Hangartner & Alex James Turner & Silviya Nikolova & Matt Sutton, 2016. "Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units," Health Economics, John Wiley & Sons, Ltd., vol. 25(12), pages 1514-1528, December.
    29. Staša Milojević, 2012. "How Are Academic Age, Productivity and Collaboration Related to Citing Behavior of Researchers?," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-13, November.
    30. Jan Fagerberg, 2017. "Innovation Policy: Rationales, Lessons And Challenges," Journal of Economic Surveys, Wiley Blackwell, vol. 31(2), pages 497-512, April.
    31. Qiu, Shumin & Liu, Xielin & Gao, Taishan, 2017. "Do emerging countries prefer local knowledge or distant knowledge? Spillover effect of university collaborations on local firms," Research Policy, Elsevier, vol. 46(7), pages 1299-1311.
    32. Firpo Sergio & Possebom Vitor, 2018. "Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets," Journal of Causal Inference, De Gruyter, vol. 6(2), pages 1-26, September.
    33. Tevfik Murat Yildirim, 2022. "Stability and change in the public’s policy agenda: a punctuated equilibrium approach," Policy Sciences, Springer;Society of Policy Sciences, vol. 55(2), pages 337-350, June.
    34. Hyeri Choi & Hangjung Zo, 2019. "Assessing the efficiency of national innovation systems in developing countries," Science and Public Policy, Oxford University Press, vol. 46(4), pages 530-540.
    35. Leandro Alves Silva & Sara Gonçalves Antunes de Souza & Hérica Morais Righi & Wellington Marcelo Silva da Cruz, 2009. "University—industry interactions in an immature system of innovation: Evidence from Minas Gerais, Brazil," Science and Public Policy, Oxford University Press, vol. 36(5), pages 373-386, June.
    36. Guijie Zhang & Yuqiang Feng & Guang Yu & Luning Liu & Yanqiqi Hao, 2017. "Analyzing the time delay between scientific research and technology patents based on the citation distribution model," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1287-1306, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    2. Sarah Vogel & Cynthia Loftin & Joseph Zydlewski, 2025. "Publication patterns and collaborative dynamics of the fish and wildlife research partnership model," Environment Systems and Decisions, Springer, vol. 45(2), pages 1-17, June.
    3. Fernando Martin Mayoral & Carlos Andrés Rivera Lasso, 2025. "From Conflict to Prosperity: The Economic Impact of the Colombian Peace Agreements," Journal of International Development, John Wiley & Sons, Ltd., vol. 37(3), pages 758-772, April.
    4. Nuno Garoupa & Rok Spruk, 2024. "Populist Constitutional Backsliding and Judicial Independence: Evidence from Turkiye," Papers 2410.02439, arXiv.org.
    5. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    6. Maximiliano Marzetti & Rok Spruk, 2023. "Long-Term Economic Effects of Populist Legal Reforms: Evidence from Argentina," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 65(1), pages 60-95, March.
    7. Cummins Joseph & Miller Douglas L. & Smith Brock & Simon David, 2024. "Matching on Noise: Finite Sample Bias in the Synthetic Control Estimator," Journal of Econometric Methods, De Gruyter, vol. 13(1), pages 67-95, January.
    8. Pier Basaglia & Sophie M. Behr & Moritz A. Drupp, 2023. "De-Fueling Externalities: How Tax Salience and Fuel Substitution Mediate Climate and Health Benefits," Discussion Papers of DIW Berlin 2041, DIW Berlin, German Institute for Economic Research.
    9. Pier Basaglia & Sophie M. Behr & Moritz A. Drupp & Piero Basaglia, 2023. "De-Fueling Externalities: Causal Effects of Fuel Taxation and Mediating Mechanisms for Reducing Climate and Pollution Costs," CESifo Working Paper Series 10508, CESifo.
    10. Nuno Garoupa & Rok Spruk, 2025. "Revolutions as Structural Breaks: The Long-Term Economic and Institutional Consequences of the 1979 Iranian Revolution," Papers 2505.02425, arXiv.org.
    11. Mohammad Reza Farzanegan, 2023. "Years of life lost to revolution and war in Iran," Review of Development Economics, Wiley Blackwell, vol. 27(4), pages 2061-2103, November.
    12. Samuel Verevis & Murat Üngör, 2021. "What has New Zealand gained from The FTA with China?: Two counterfactual analyses†," Scottish Journal of Political Economy, Scottish Economic Society, vol. 68(1), pages 20-50, February.
    13. McCloud, Nadine, 2024. "What has inflation targeting done for household consumption?," International Review of Financial Analysis, Elsevier, vol. 95(PC).
    14. Irene Botosaru & Bruno Ferman, 2019. "On the role of covariates in the synthetic control method," The Econometrics Journal, Royal Economic Society, vol. 22(2), pages 117-130.
    15. Robert Messerle & Jonas Schreyögg, 2024. "Country-level effects of diagnosis-related groups: evidence from Germany’s comprehensive reform of hospital payments," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 25(6), pages 1013-1030, August.
    16. Michael Funke & Kadri Männasoo & Helery Tasane, 2023. "Regional Economic Impacts of the Øresund Cross-Border Fixed Link: Cui Bono?," CESifo Working Paper Series 10557, CESifo.
    17. Andrii Melnychuk, 2024. "Synthetic Controls with spillover effects: A comparative study," Papers 2405.01645, arXiv.org.
    18. M. A. Bautista & F. Gonz�lez & L. R. Mart�nez & P. Mu�oz & M. Prem, 2020. "Chile’s Missing Students: Dictatorship, Higher Education and Social Mobility," Documentos de Trabajo 18163, Universidad del Rosario.
    19. Tomasz Serwach, 2023. "The European Union and within‐country income inequalities. The case of the new member states," The World Economy, Wiley Blackwell, vol. 46(7), pages 1890-1939, July.
    20. Judith Nyulas & Ștefan Dezsi & Adrian-Florin Niță & Zsolt Magyari-Sáska & Marie-Luise Frey & Alpár Horváth, 2025. "Twenty-Five Years of Scientific Production on Geoparks from the Perspective of Bibliometric Analysis Using PRISMA," Sustainability, MDPI, vol. 17(5), pages 1-30, March.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:130:y:2025:i:4:d:10.1007_s11192-025-05272-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.