IDEAS home Printed from https://ideas.repec.org/a/eee/bushor/v65y2022i4p437-446.html
   My bibliography  Save this article

It’s time to close the experimentation gap in advertising: Confronting myths surrounding ad testing

Author

Listed:
  • Campbell, Colin
  • Runge, Julian
  • Bates, Kenneth
  • Haefele, Stacey
  • Jayaraman, Neeraj

Abstract

Marketers know that running experiments is a proven way to improve results and gain competitive advantage against rivals. Despite this knowledge—and the fact that experiments are now easier to conduct than ever before—data shows that marketers consistently under-experiment. In this article, we examine why this gap exists and what can be done to close it. We do so by connecting with senior-level marketing professionals representing seven consumer-facing industries in two phases. First, through a series of interviews, we gain initial understanding of the concerns, challenges, and realities of those working in the industry. Following this phase, we surveyed a larger group to corroborate and extend our initial findings, comparing cases to identify challenges and the strategies used to overcome them. We present our findings as a series of experimentation myths before closing with a broader perspective on how organizations can infuse experimentation into their culture.

Suggested Citation

  • Campbell, Colin & Runge, Julian & Bates, Kenneth & Haefele, Stacey & Jayaraman, Neeraj, 2022. "It’s time to close the experimentation gap in advertising: Confronting myths surrounding ad testing," Business Horizons, Elsevier, vol. 65(4), pages 437-446.
    • Handle: RePEc:eee:bushor:v:65:y:2022:i:4:p:437-446
    DOI: 10.1016/j.bushor.2021.05.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0007681321000975
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.bushor.2021.05.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Eric M. Schwartz & Eric T. Bradlow & Peter S. Fader, 2017. "Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments," Marketing Science, INFORMS, vol. 36(4), pages 500-522, July.
    2. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    3. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    4. Joseph O. Eastlack, Jr. & Ambar G. Rao, 1989. "Advertising Experiments at the Campbell Soup Company," Marketing Science, INFORMS, vol. 8(1), pages 57-71.
    5. Erik Brynjolfsson & Kristina McElheran, 2016. "The Rapid Adoption of Data-Driven Decision-Making," American Economic Review, American Economic Association, vol. 106(5), pages 133-139, May.
    6. Daniel Zantedeschi & Eleanor McDonnell Feit & Eric T. Bradlow, 2017. "Measuring Multichannel Advertising Response," Management Science, INFORMS, vol. 63(8), pages 2706-2728, August.
    7. Thomke, Stefan & von Hippel, Eric & Franke, Roland, 1998. "Modes of experimentation: an innovation process--and competitive--variable," Research Policy, Elsevier, vol. 27(3), pages 315-332, July.
    8. Marc Fischer & Sönke Albers & Nils Wagner & Monika Frie, 2011. "Practice Prize Winner --Dynamic Marketing Budget Allocation Across Countries, Products, and Marketing Activities," Marketing Science, INFORMS, vol. 30(4), pages 568-585, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brett R Gordon & Kinshuk Jerath & Zsolt Katona & Sridhar Narayanan & Jiwoong Shin & Kenneth C Wilbur, 2019. "Inefficiencies in Digital Advertising Markets," Papers 1912.09012, arXiv.org, revised Feb 2020.
    2. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    3. Thomas W. Frick & Rodrigo Belo & Rahul Telang, 2023. "Incentive Misalignments in Programmatic Advertising: Evidence from a Randomized Field Experiment," Management Science, INFORMS, vol. 69(3), pages 1665-1686, March.
    4. Navdeep Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    5. Kirthi Kalyanam & John McAteer & Jonathan Marek & James Hodges & Lifeng Lin, 2018. "Cross channel effects of search engine advertising on brick & mortar retail sales: Meta analysis of large scale field experiments on Google.com," Quantitative Marketing and Economics (QME), Springer, vol. 16(1), pages 1-42, March.
    6. Gui Liberali & Alina Ferecatu, 2022. "Morphing for Consumer Dynamics: Bandits Meet Hidden Markov Models," Marketing Science, INFORMS, vol. 41(4), pages 769-794, July.
    7. George Gui & Harikesh Nair & Fengshi Niu, 2021. "Auction Throttling and Causal Inference of Online Advertising Effects," Papers 2112.15155, arXiv.org, revised Feb 2022.
    8. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2023. "Digital public health interventions at scale: The impact of social media advertising on beliefs and outcomes related to COVID vaccines," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(5), pages 2208110120-, January.
    9. Haupt, Johannes & Jacob, Daniel & Gubela, Robin M. & Lessmann, Stefan, 2019. "Affordable Uplift: Supervised Randomization in Controlled Exprtiments," IRTG 1792 Discussion Papers 2019-026, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    10. Michael Thomas, 2020. "Spillovers from Mass Advertising: An Identification Strategy," Marketing Science, INFORMS, vol. 39(4), pages 807-826, July.
    11. Hana Choi & Carl F. Mela & Santiago R. Balseiro & Adam Leary, 2020. "Online Display Advertising Markets: A Literature Review and Future Directions," Information Systems Research, INFORMS, vol. 31(2), pages 556-575, June.
    12. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    13. Navdeep S. Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    14. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2022. "Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement," Papers 2201.07055, arXiv.org, revised Oct 2022.
    15. Caio Waisman & Navdeep S. Sahni & Harikesh S. Nair & Xiliang Lin, 2019. "Parallel Experimentation and Competitive Interference on Online Advertising Platforms," Papers 1903.11198, arXiv.org, revised Feb 2024.
    16. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2023. "Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement," Marketing Science, INFORMS, vol. 42(4), pages 768-793, July.
    17. Ron Berman & Christophe Van den Bulte, 2022. "False Discovery in A/B Testing," Management Science, INFORMS, vol. 68(9), pages 6762-6782, September.
    18. Berman, Ron & Heller, Yuval, 2020. "Naive Analytics Equilibrium," MPRA Paper 103824, University Library of Munich, Germany.
    19. Randall Lewis & David Reiley, 2014. "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!," Quantitative Marketing and Economics (QME), Springer, vol. 12(3), pages 235-266, September.
    20. Ratchford, Brian & Soysal, Gonca & Zentner, Alejandro & Gauri, Dinesh K., 2022. "Online and offline retailing: What we know and directions for future research," Journal of Retailing, Elsevier, vol. 98(1), pages 152-177.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:bushor:v:65:y:2022:i:4:p:437-446. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/bushor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.