IDEAS home Printed from https://ideas.repec.org/p/osf/metaar/cj7xf.html
   My bibliography  Save this paper

Examining Psychological Science through Systematic Meta-Method Analysis: A Call for Research

Author

Listed:
  • Elson, Malte

    (Ruhr University Bochum)

Abstract

Research synthesis is based on the assumption that when the same association between constructs is observed repeatedly in a field, the relationship is probably real, even if its exact magnitude can be debated. Yet this probability is not only a function of recurring results, but also of the quality and consistency in the empirical procedures that produced those results and that any meta-analysis necessarily inherits. Standardized protocols in data collection, analysis, and interpretation are important empirical properties, and a healthy sign of a discipline's maturity. This manuscript proposes that meta-analyses as typically applied in psychology benefit from complementing their aggregates of observed effect sizes by systematically examining the standardization of methodology that deterministically produced them. Potential units of analyses are described and two examples are offered to illustrate the benefits of such efforts. Ideally, this synergetic approach emphasizes the role of methods in advancing theory by improving the quality of meta-analytic inferences.

Suggested Citation

  • Elson, Malte, 2019. "Examining Psychological Science through Systematic Meta-Method Analysis: A Call for Research," MetaArXiv cj7xf, Center for Open Science.
    • Handle: RePEc:osf:metaar:cj7xf
    DOI: 10.31219/osf.io/cj7xf
    as

    Download full text from publisher

    File URL: https://osf.io/download/5d0cc2d811bd080019a228a4/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/cj7xf?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sue Duval & Richard Tweedie, 2000. "Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis," Biometrics, The International Biometric Society, vol. 56(2), pages 455-463, June.
    2. Wiernik, Brenton M. & Kostal, Jack W. & Wilmot, Michael P. & Dilchert, Stephan & Ones, Deniz S., 2017. "Empirical Benchmarks for Interpreting Effect Size Variability in Meta-Analysis," Industrial and Organizational Psychology, Cambridge University Press, vol. 10(3), pages 472-479, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qian Li & Yan Chen & Shikun Sun & Muyuan Zhu & Jing Xue & Zihan Gao & Jinfeng Zhao & Yihe Tang, 2022. "Research on Crop Irrigation Schedules Under Deficit Irrigation—A Meta-analysis," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(12), pages 4799-4817, September.
    2. Bart Verkuil & Serpil Atasayi & Marc L Molendijk, 2015. "Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    3. Damiano Pizzol & Mike Trott & Igor Grabovac & Mario Antunes & Anna Claudia Colangelo & Simona Ippoliti & Cristian Petre Ilie & Anne Carrie & Nicola Veronese & Lee Smith, 2021. "Laparoscopy in Low-Income Countries: 10-Year Experience and Systematic Literature Review," IJERPH, MDPI, vol. 18(11), pages 1-11, May.
    4. Ünal, Zehra E. & Kartal, Gamze & Ulusoy, Serra & Ala, Aslı M. & Yilmaz, Munube & Geary, David C., 2023. "Relative contributions of g and basic domain-specific mathematics skills to complex mathematics competencies," Intelligence, Elsevier, vol. 101(C).
    5. Viktoria Maria Baumeister & Leonie Petra Kuen & Maike Bruckes & Gerhard Schewe, 2021. "The Relationship of Work-Related ICT Use With Well-being, Incorporating the Role of Resources and Demands: A Meta-Analysis," SAGE Open, , vol. 11(4), pages 21582440211, November.
    6. Gundula Krack, 2019. "How to make value-based health insurance designs more effective? A systematic review and meta-analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 841-856, August.
    7. Chuang Yuan & Jing Wang & Michael Ying, 2016. "Predictive Value of Carotid Distensibility Coefficient for Cardiovascular Diseases and All-Cause Mortality: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-15, April.
    8. Francisco Javier Blanco-Encomienda & Rocío García-Cantero & María José Latorre-Medina, 2020. "Association between Work-Related Rumination, Work Environment and Employee Well-Being: A Meta-Analytic Study of Main and Moderator Effects," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 150(3), pages 887-910, August.
    9. Pedro L. Cosio & Manuel Crespo-Posadas & Álvaro Velarde-Sotres & Mireia Pelaez, 2021. "Effect of Chronic Resistance Training on Circulating Irisin: Systematic Review and Meta-Analysis of Randomized Controlled Trials," IJERPH, MDPI, vol. 18(5), pages 1-16, March.
    10. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    11. Georgiou, George K. & Guo, Kan & Naveenkumar, Nithya & Vieira, Ana Paula Alves & Das, J.P., 2020. "PASS theory of intelligence and academic achievement: A meta-analytic review," Intelligence, Elsevier, vol. 79(C).
    12. Liliana R Demenescu & Rudie Kortekaas & Johan A den Boer & André Aleman, 2010. "Impaired Attribution of Emotion to Facial Expressions in Anxiety and Major Depression," PLOS ONE, Public Library of Science, vol. 5(12), pages 1-5, December.
    13. Lohwasser, Todor S., 2020. "Meta-analyzing the relative performance of venture capital-backed firms," Discussion Papers of the Institute for Organisational Economics 4/2020, University of Münster, Institute for Organisational Economics.
    14. Nelson, Jon Paul, 2020. "Fixed-effect versus random-effects meta-analysis in economics: A study of pass-through rates for alcohol beverage excise taxes," Economics Discussion Papers 2020-1, Kiel Institute for the World Economy (IfW Kiel).
    15. Michael Wolfowicz & Yael Litmanovitz & David Weisburd & Badi Hasisi, 2021. "Cognitive and behavioral radicalization: A systematic review of the putative risk and protective factors," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(3), September.
    16. Ruxiu Tie & Tiansong Zhang & Huarui Fu & Limengmeng Wang & Yebo Wang & Ying He & Binsheng Wang & Ni Zhu & Shan Fu & Xiaoyu Lai & Jimin Shi & He Huang, 2014. "Association between DNMT3A Mutations and Prognosis of Adults with De Novo Acute Myeloid Leukemia: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    17. Lianjie Liu & Zhuo Shao & Hang Yu & Wei Zhang & Hao Wang & Zubing Mei, 2020. "Is the platelet to lymphocyte ratio a promising biomarker to distinguish acute appendicitis? Evidence from a systematic review with meta-analysis," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-15, May.
    18. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    19. Nian-Feng Wan & Liwan Fu & Matteo Dainese & Yue-Qing Hu & Lars Pødenphant Kiær & Forest Isbell & Christoph Scherber, 2022. "Plant genetic diversity affects multiple trophic levels and trophic interactions," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    20. Kim, Yeolib & Kim, Seung Hyun & Peterson, Robert A. & Choi, Jeonghye, 2023. "Privacy concern and its consequences: A meta-analysis," Technological Forecasting and Social Change, Elsevier, vol. 196(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:cj7xf. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.