IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/fghcd.html
   My bibliography  Save this paper

Avoiding the eyeballing fallacy: Visualizing statistical differences between estimates using the pheatplot command

Author

Listed:
  • Brini, Elisa
  • Borgen, Solveig Topstad

    (University of Oslo)

  • Borgen, Nicolai T.

Abstract

Graphical representations of coefficients and confidence intervals of regression coefficients are increasingly used in scientific presentations and publications due to their easier and quicker readability compared to tables. However, in coefficient plots that include several estimated coefficients, researchers often use the confidence intervals to eyeball whether coefficients are statistically significant from each other, which results in an overly conservative test and increased risk of type II error. To avoid this eyeballing fallacy, we introduce the pheatplot postestimation command, designed to visualize the statistical significance across estimates of categorical variables in a regression model. It efficiently compares the significance level between point estimates and helps researchers avoid making wrong assumptions about whether estimates differ. Moreover, by representing p-values as a continuous measure rather than a binary threshold, it provides the flexibility to move beyond arbitrary cutoffs of statistical significance. This article provides a series of examples that illustrate the functionality of the pheatplot command.

Suggested Citation

  • Brini, Elisa & Borgen, Solveig Topstad & Borgen, Nicolai T., 2023. "Avoiding the eyeballing fallacy: Visualizing statistical differences between estimates using the pheatplot command," SocArXiv fghcd, Center for Open Science.
    • Handle: RePEc:osf:socarx:fghcd
    DOI: 10.31219/osf.io/fghcd
    as

    Download full text from publisher

    File URL: https://osf.io/download/651489683d9bde027a8d8415/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/fghcd?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Harvey Goldstein & Michael J. R. Healy, 1995. "The Graphical Presentation of a Collection of Means," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(1), pages 175-177, January.
    2. Afshartous, David & Preston, Richard A., 2010. "Confidence intervals for dependent data: Equating non-overlap with statistical significance," Computational Statistics & Data Analysis, Elsevier, vol. 54(10), pages 2296-2305, October.
    3. Alice Goisis & Peter Fallesen & Marta Seiz & Leire Salazar & Tatiana Eremenko & Marco Cozzani, 2023. "Educational gradients in the prevalence of Medically Assisted Reproduction (MAR) births in a comparative perspective," JRC Working Papers on Social Classes in the Digital Age 2023-06, Joint Research Centre.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giorgio E. Montanari & Marco Doretti, 2019. "Ranking Nursing Homes’ Performances Through a Latent Markov Model with Fixed and Random Effects," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 146(1), pages 307-326, November.
    2. Columbu, Silvia & Porcu, Mariano & Sulis, Isabella, 2021. "University choice and the attractiveness of the study area: Insights on the differences amongst degree programmes in Italy based on generalised mixed-effect models," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    3. Kimihiro Noguchi & Fernando Marmolejo-Ramos, 2016. "Assessing Equality of Means Using the Overlap of Range-Preserving Confidence Intervals," The American Statistician, Taylor & Francis Journals, vol. 70(4), pages 325-334, October.
    4. Elena Pirani & Daniele Vignoli, 2021. "Childbearing Across Partnerships in Italy: Prevalence, Demographic Correlates, Social Gradient," Econometrics Working Papers Archive 2021_15, Universita' degli Studi di Firenze, Dipartimento di Statistica, Informatica, Applicazioni "G. Parenti".
    5. Pickery, Jan, 2002. "Contextual effects on the vote in Germany: A multilevel analysis," Discussion Papers, Research Unit: Institutions and Social Change FS III 02-202, WZB Berlin Social Science Center.
    6. Omar Paccagnella, 2006. "Centering or Not Centering in Multilevel Models? The Role of the Group Mean and the Assessment of Group Effects," Evaluation Review, , vol. 30(1), pages 66-85, February.
    7. Tim Goedemé & Karel Van den Bosch & Lina Salanauskaite & Gerlinde Verbist, 2013. "Testing the Statistical Significance of Microsimulation Results: Often Easier than You Think. A Technical Note," ImPRovE Working Papers 13/10, Herman Deleeck Centre for Social Policy, University of Antwerp.
    8. Donata Marasini & Piero Quatto & Enrico Ripamonti, 2017. "Inferential confidence intervals for fuzzy analysis of teaching satisfaction," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(4), pages 1513-1529, July.
    9. Mikula, Stepan & Montag, Josef, 2023. "Roma and Bureaucrats: A Field Experiment on Ethnic and Socioeconomic Discrimination," IZA Discussion Papers 16218, Institute of Labor Economics (IZA).
    10. Yu-Cheng Ku & Tsun-Feng Chiang & Sheng-Mao Chang, 2017. "Is what you choose what you want?—outlier detection in choice-based conjoint analysis," Marketing Letters, Springer, vol. 28(1), pages 29-42, March.
    11. Bosker, R.J. & van de Loo, P.J.E. & van der Velden, R.K.W., 1997. "Differential effects of colleges on the labour market success of their graduates," ROA Research Memorandum 1E, Maastricht University, Research Centre for Education and the Labour Market (ROA).
    12. Carlijn Bussemakers & Gerbert Kraaykamp & Jochem Tolsma, 2022. "Variation in the educational consequences of parental death and divorce: The role of family and country characteristics," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 46(20), pages 581-618.
    13. Montanari, Giorgio E. & Doretti, Marco & Bartolucci, Francesco, 2017. "A multilevel latent Markov model for the evaluation of nursing homes' performance," MPRA Paper 80691, University Library of Munich, Germany.
    14. Bryony Hoskins & Michaela Saisana & Cynthia Villalba, 2015. "Civic Competence of Youth in Europe: Measuring Cross National Variation Through the Creation of a Composite Indicator," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 123(2), pages 431-457, September.
    15. S. Bacci & B. Bertaccini, 2021. "Assessment of the University Reputation Through the Analysis of the Student Mobility," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 156(2), pages 363-388, August.
    16. Eleonora Trappolini & Cristina Giudici, 2021. "Gendering health differences between nonmigrants and migrants by duration of stay in Italy," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 45(7), pages 221-258.
    17. Ku, Yu-Cheng & Wu, John, 2018. "Measuring respondent uncertainty in discrete choice experiments via utility suppression," Journal of choice modelling, Elsevier, vol. 27(C), pages 1-18.
    18. Giammarco Alderotti & Cecilia Tomassini & Daniele Vignoli, 2022. "‘Silver splits’ in Europe: The role of grandchildren and other correlates," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 46(21), pages 619-652.
    19. Nianbo Dong & Mark W. Lipsey, 2018. "Can Propensity Score Analysis Approximate Randomized Experiments Using Pretest and Demographic Information in Pre-K Intervention Research?," Evaluation Review, , vol. 42(1), pages 34-70, February.
    20. Kathleen Carey, 2000. "A multilevel modelling approach to analysis of patient costs under managed care," Health Economics, John Wiley & Sons, Ltd., vol. 9(5), pages 435-446, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:fghcd. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.