IDEAS home Printed from https://ideas.repec.org/a/bla/ecinqu/v63y2025i2p387-402.html

Underpowered studies and exaggerated effects: A replication and re‐evaluation of the magnitude of anchoring effects

Author

Listed:
  • Tongzhe Li
  • Collin Weigel
  • Paul Ferraro
  • Kent D. Messer

Abstract

We reconsider one of the most widely studied behavioral biases: anchoring effects. We estimate that study designs in this literature, including replication studies, routinely fail to achieve statistical power of more than 30%. This study replicates an anchoring study that reported an effect size of a 31% increase in participants' bids. In the replication, we increased the design's statistical power from 46% to 96%, reducing the average exaggeration of a statistically significant result by a factor of seven. Our replication results reject the size of the original estimated effects. We find an estimated effect of 3.4% (95% CI [−3.4%, 10%]).

Suggested Citation

  • Tongzhe Li & Collin Weigel & Paul Ferraro & Kent D. Messer, 2025. "Underpowered studies and exaggerated effects: A replication and re‐evaluation of the magnitude of anchoring effects," Economic Inquiry, Western Economic Association International, vol. 63(2), pages 387-402, April.
    • Handle: RePEc:bla:ecinqu:v:63:y:2025:i:2:p:387-402
    DOI: 10.1111/ecin.13279
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/ecin.13279
    Download Restriction: no
    File URL: https://libkey.io/10.1111/ecin.13279?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Konstantinos Ioannidis & Theo Offerman & Randolph Sloof, 2020. "On the effect of anchoring on valuations when the anchor is transparently uninformative," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 6(1), pages 77-94, June.
    2. Jonathan E. Alevy & Craig E. Landry & John A. List, 2015. "Field Experiments On The Anchoring Of Economic Valuations," Economic Inquiry, Western Economic Association International, vol. 53(3), pages 1522-1538, July.
    3. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    4. Drew Fudenberg & David K. Levine & Zacharias Maniadis, 2012. "On the Robustness of Anchoring Effects in WTP and WTA Experiments," American Economic Journal: Microeconomics, American Economic Association, vol. 4(2), pages 131-145, May.
    5. Dan Ariely & George Loewenstein & Drazen Prelec, 2003. ""Coherent Arbitrariness": Stable Demand Curves Without Stable Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(1), pages 73-106.
    6. Bergman, Oscar & Ellingsen, Tore & Johannesson, Magnus & Svensson, Cicek, 2010. "Anchoring and cognitive ability," Economics Letters, Elsevier, vol. 107(1), pages 66-68, April.
    7. Liang Guo, 2024. "Recovering the Anchoring of Economic Valuations," American Economic Journal: Microeconomics, American Economic Association, vol. 16(4), pages 192-228, November.
    8. Stefano DellaVigna & Elizabeth Linos, 2022. "RCTs to Scale: Comprehensive Evidence From Two Nudge Units," Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
    9. Zacharias Maniadis & Fabio Tufano & John A. List, 2014. "One Swallow Doesn't Make a Summer: New Evidence on Anchoring Effects," American Economic Review, American Economic Association, vol. 104(1), pages 277-290, January.
    10. Cristina Blanco-Perez & Abel Brodeur, 2020. "Publication Bias and Editorial Statement on Negative Findings," The Economic Journal, Royal Economic Society, vol. 130(629), pages 1226-1247.
    11. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Magdalena Brzozowicz & Michał Krawczyk, 2020. "Honey, Mugs and Caricatures: anchors on prices of consumer goods only hold hypothetically," Working Papers 2020-40, Faculty of Economic Sciences, University of Warsaw.
    2. Li, Lunzheng & Maniadis, Zacharias & Sedikides, Constantine, 2021. "Anchoring in Economics: A Meta-Analysis of Studies on Willingness-To-Pay and Willingness-To-Accept," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 90(C).
    3. Konstantinos Ioannidis & Theo Offerman & Randolph Sloof, 2020. "On the effect of anchoring on valuations when the anchor is transparently uninformative," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 6(1), pages 77-94, June.
    4. Sangsuk Yoon & Nathan M. Fong & Angelika Dimoka, 2019. "The robustness of anchoring effects on preferential judgments," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(4), pages 470-487, July.
    5. Ioannidis, Konstantinos, 2023. "Anchoring on valuations and perceived informativeness," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    6. Meyerhoff, Jürgen & Glenk, Klaus, 2015. "Learning how to choose—effects of instructional choice sets in discrete choice experiments," Resource and Energy Economics, Elsevier, vol. 41(C), pages 122-142.
    7. Meub, Lukas & Proeger, Till, 2014. "Are groups 'less behavioral'? The case of anchoring," University of Göttingen Working Papers in Economics 188, University of Goettingen, Department of Economics.
    8. Sugden, Robert & Zheng, Jiwei & Zizzo, Daniel John, 2013. "Not all anchors are created equal," Journal of Economic Psychology, Elsevier, vol. 39(C), pages 21-31.
    9. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    10. Susan Godlonton & Manuel A Hernandez & Mike Murphy, 2018. "Anchoring Bias in Recall Data: Evidence from Central America," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 100(2), pages 479-501.
    11. Jonathan E. Alevy & Craig E. Landry & John A. List, 2015. "Field Experiments On The Anchoring Of Economic Valuations," Economic Inquiry, Western Economic Association International, vol. 53(3), pages 1522-1538, July.
    12. Benjamin Enke & Uri Gneezy & Brian Hall & David Martin & Vadim Nelidov & Theo Offerman & Jeroen van de Ven, 2020. "Cognitive Biases: Mistakes or Missing Stakes?," CESifo Working Paper Series 8168, CESifo.
    13. Holm, Håkan J. & Samahita, Margaret & van Veldhuizen, Roel & Wengström, Erik, 2025. "Anchoring and subjective belief distributions," Journal of Economic Behavior & Organization, Elsevier, vol. 240(C).
    14. Carlos Alós-Ferrer & Georg D. Granic, 2023. "Does choice change preferences? An incentivized test of the mere choice effect," Experimental Economics, Springer;Economic Science Association, vol. 26(3), pages 499-521, July.
    15. Ferman, Bruno & Finamor, Lucas, 2025. "There must be an error here! Experimental evidence on coding errors' biases," I4R Discussion Paper Series 266, The Institute for Replication (I4R).
    16. Drew Fudenberg & David K. Levine & Zacharias Maniadis, 2012. "On the Robustness of Anchoring Effects in WTP and WTA Experiments," American Economic Journal: Microeconomics, American Economic Association, vol. 4(2), pages 131-145, May.
    17. Meub, Lukas & Proeger, Till E., 2015. "Anchoring in social context," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 55(C), pages 29-39.
    18. Christophe Hurlin & Jean-Edouard Colliard & Christophe Pérignon, 2024. "The Economics of Computational Reproducibility," Working Papers hal-04759240, HAL.
    19. Lukas Meub & Till Proeger, 2018. "Are groups ‘less behavioral’? The case of anchoring," Theory and Decision, Springer, vol. 85(2), pages 117-150, August.
    20. Catherine Roux & Christian Thöni, 2015. "Do control questions influence behavior in experiments?," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 185-194, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:ecinqu:v:63:y:2025:i:2:p:387-402. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/weaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.