STAREG: Statistical replicability analysis of high throughput experiments with applications to spatial transcriptomic studies
Author
Abstract
Replicable signals from different yet conceptually related studies provide stronger scientific evidence and more powerful inference. We introduce STAREG, a statistical method for replicability analysis of high throughput experiments, and apply it to analyze spatial transcriptomic studies. STAREG uses summary statistics from multiple studies of high throughput experiments and models the the joint distribution of p-values accounting for the heterogeneity of different studies. It effectively controls the false discovery rate (FDR) and has higher power by information borrowing. Moreover, it provides different rankings of important genes. With the EM algorithm in combination with pool-adjacent-violator-algorithm (PAVA), STAREG is scalable to datasets with millions of genes without any tuning parameters. Analyzing two pairs of spatially resolved transcriptomic datasets, we are able to make biological discoveries that otherwise cannot be obtained by using existing methods.Author summary: Irreplicable research wastes time, money, and/or resources. Approximately $28 billion is estimated to be spent on preclinical research that cannot be replicated every year in the United States alone. Possible causes of irreplicable research may include experimental design, laboratory practices, and data analysis. We focus on data analysis. The past two decades have witnessed the expansion and increased availability of genomic data from high-throughput experiments. Due to privacy concerns or logistic reasons, raw data can be difficult to access but summary data such as p-values are readily available. We introduce STAREG, which jointly analyzes p-values from multiple genomic datasets that target the same scientific question with different populations or different technologies. This allows us to have more convincing and robust findings. STAREG is computationally scalable with solid statistical analysis. Moreover, it is versatile, platform-independent, and only requires p-values as input. By analyzing data sets from spatially resolved transcriptomic studies, we make biological discoveries that otherwise cannot be obtained with existing methods.
Suggested Citation
DOI: 10.1371/journal.pgen.1011423
Download full text from publisher
References listed on IDEAS
- C. Glenn Begley & Lee M. Ellis, 2012. "Raise standards for preclinical cancer research," Nature, Nature, vol. 483(7391), pages 531-533, March.
- Leonard P Freedman & Iain M Cockburn & Timothy S Simcoe, 2015. "The Economics of Reproducibility in Preclinical Research," PLOS Biology, Public Library of Science, vol. 13(6), pages 1-9, June.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018.
"Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015,"
Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
- Camerer, Colin F. & Dreber, Anna & Holzmeister, Felix & Ho, Teck-Hua & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas & Altmejd, Adam & But, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Munich Reprints in Economics 62818, University of Munich, Department of Economics.
- Camerer, Colin & Dreber, Anna & Holzmeister, Felix & Ho, Teck Hua & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," SocArXiv 4hmb6, Center for Open Science.
- Malika Ihle & Isabel S. Winney & Anna Krystalli & Michael Croucher, 2017. "Striving for transparent and credible research: practical guidelines for behavioral ecologists," Behavioral Ecology, International Society for Behavioral Ecology, vol. 28(2), pages 348-354.
- Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
- Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
- Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019.
"Replication studies in economics—How many and which papers are chosen for replication, and why?,"
EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 48(1), pages 62-83.
- Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.
- Frank Mueller-Langer & Benedikt Fecher & Dietmar Harhoff & Gert G. Wagner, 2018. "Replication Studies in Economics: How Many and Which Papers Are Chosen for Replication, and Why?," JRC Working Papers on Digital Economy 2018-01, Joint Research Centre.
- Joanna Chataway & Sarah Parks & Elta Smith, 2017. "How Will Open Science Impact on University-Industry Collaboration?," Foresight and STI Governance, National Research University Higher School of Economics, vol. 11(2), pages 44-53.
- Michaël Bikard & Matt Marx, 2020. "Bridging Academia and Industry: How Geographic Hubs Connect University Science and Corporate Technology," Management Science, INFORMS, vol. 66(8), pages 3425-3443, August.
- Michaël Bikard, 2018. "Made in Academia: The Effect of Institutional Origin on Inventors’ Attention to Science," Organization Science, INFORMS, vol. 29(5), pages 818-836, October.
- Kiri, Bralind & Lacetera, Nicola & Zirulia, Lorenzo, 2018.
"Above a swamp: A theory of high-quality scientific production,"
Research Policy, Elsevier, vol. 47(5), pages 827-839.
- Bralind Kiri & Nicola Lacetera & Lorenzo Zirulia, 2015. "Above a Swamp: A Theory of High-Quality Scientific Production," NBER Working Papers 21143, National Bureau of Economic Research, Inc.
- repec:plo:pone00:0147140 is not listed on IDEAS
- repec:plo:pone00:0215221 is not listed on IDEAS
- Hussinger, Katrin & Pellens, Maikel, 2019.
"Guilt by association: How scientific misconduct harms prior collaborators,"
Research Policy, Elsevier, vol. 48(2), pages 516-530.
- Hussinger, Katrin & Pellens, Maikel, 2017. "Guilt by association: How scientific misconduct harms prior collaborators," ZEW Discussion Papers 17-051, ZEW - Leibniz Centre for European Economic Research.
- Katrin Hussinger & Maikel Pellens, 2018. "Guilt by Association: How Scientific Misconduct Harms Prior Collaborators," DEM Discussion Paper Series 18-15, Department of Economics at the University of Luxembourg.
- Seibold, Heidi & Charlton, Alethea & Boulesteix, Anne-Laure & Hoffmann, Sabine, 2020. "Statisticians roll up your sleeves! There’s a crisis to be solved," MetaArXiv frta7, Center for Open Science.
- Watzinger, Martin & Schnitzer, Monika, 2019.
"Standing on the Shoulders of Science,"
Rationality and Competition Discussion Paper Series
215, CRC TRR 190 Rationality and Competition.
- Schnitzer, Monika & Krieger, Joshua & Watzinger, Martin, 2019. "Standing on the shoulders of science," CEPR Discussion Papers 13766, Centre for Economic Policy Research.
- Andreoli-Versbach, Patrick & Mueller-Langer, Frank, 2014.
"Open access to data: An ideal professed but not practised,"
Research Policy, Elsevier, vol. 43(9), pages 1621-1633.
- Patrick Andreoli-Versbach & Frank Mueller-Langer, 2013. "Open Access to Data: An Ideal Professed but not Practised," RatSWD Working Papers 215, German Data Forum (RatSWD).
- repec:plo:pbio00:3000763 is not listed on IDEAS
- Peter Harremoës, 2019. "Replication Papers," Publications, MDPI, vol. 7(3), pages 1-8, July.
- Bettina Bert & Céline Heinl & Justyna Chmielewska & Franziska Schwarz & Barbara Grune & Andreas Hensel & Matthias Greiner & Gilbert Schönfelder, 2019. "Refining animal research: The Animal Study Registry," PLOS Biology, Public Library of Science, vol. 17(10), pages 1-12, October.
- Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
- Nathalie Percie du Sert & Viki Hurst & Amrita Ahluwalia & Sabina Alam & Marc T Avey & Monya Baker & William J Browne & Alejandra Clark & Innes C Cuthill & Ulrich Dirnagl & Michael Emerson & Paul Garne, 2020. "The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-12, July.
- Vivian Leung & Frédérik Rousseau-Blass & Guy Beauchamp & Daniel S J Pang, 2018. "ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-13, May.
- Hajko, Vladimír, 2017. "The failure of Energy-Economy Nexus: A meta-analysis of 104 studies," Energy, Elsevier, vol. 125(C), pages 771-787.
- Oliver Braganza, 2020. "A simple model suggesting economically rational sample-size choice drives irreproducibility," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-19, March.
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pgen00:1011423. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form
Printed from https://ideas.repec.org/a/plo/pgen00/1011423.html