IDEAS home Printed from https://ideas.repec.org/a/eee/jbrese/v118y2020icp189-198.html
   My bibliography  Save this article

Running field experiments using Facebook split test

Author

Listed:
  • Orazi, Davide C.
  • Johnston, Allen C.

Abstract

Business researchers use experimental methods extensively due to their high internal validity. However, controlled laboratory and crowdsourcing settings often introduce issues of artificiality, data contamination, and low managerial relevance of the dependent variables. Field experiments can overcome these issues but are traditionally time- and resource-consuming. This primer presents an alternative experimental setting to conduct online field experiments in a time- and cost-effective way. It does so by introducing the Facebook A/B split test functionality, which allows for random assignment of manipulated variables embedded in ecologically-valid stimuli. We compare and contrast this method against laboratory settings and Amazon Mechanical Turk in terms of design flexibility, managerial relevance, data quality control, and sample representativeness. We then provide an empirical demonstration of how to set up, pre-test, run, and analyze FBST experiments.

Suggested Citation

  • Orazi, Davide C. & Johnston, Allen C., 2020. "Running field experiments using Facebook split test," Journal of Business Research, Elsevier, vol. 118(C), pages 189-198.
    • Handle: RePEc:eee:jbrese:v:118:y:2020:i:c:p:189-198
    DOI: 10.1016/j.jbusres.2020.06.053
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014829632030429X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jbusres.2020.06.053?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. J Jeffrey Inman & Margaret C Campbell & Amna Kirmani & Linda L Price, 2018. "Our Vision for the Journal of Consumer Research: It’s All about the Consumer," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(5), pages 955-959.
    2. repec:cup:judgdm:v:5:y:2010:i:5:p:411-419 is not listed on IDEAS
    3. Dean Eckles & Brett R. Gordon & Garrett A. Johnson, 2018. "Field studies of psychologically targeted ads face threats to internal validity," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 115(23), pages 5254-5255, June.
    4. Orazi, Davide C. & Pizzetti, Marta, 2015. "Revisiting fear appeals: A structural re-inquiry of the protection motivation model," International Journal of Research in Marketing, Elsevier, vol. 32(2), pages 223-225.
    5. Smith, Scott M. & Roster, Catherine A. & Golden, Linda L. & Albaum, Gerald S., 2016. "A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples," Journal of Business Research, Elsevier, vol. 69(8), pages 3139-3148.
    6. Arthur Schram, 2005. "Artificiality: The tension between internal and external validity in economic experiments," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 225-237.
    7. Orazi, Davide C. & Lei, Jing & Bove, Liliana L., 2015. "The nature and framing of gambling consequences in advertising," Journal of Business Research, Elsevier, vol. 68(10), pages 2049-2056.
    8. Kirsten Passyn & Mita Sujan, 2006. "Self-Accountability Emotions and Fear Appeals: Motivating Behavior," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 32(4), pages 583-589, March.
    9. Daly, Timothy M. & Nataraajan, Rajan, 2015. "Swapping bricks for clicks: Crowdsourcing longitudinal data on Amazon Turk," Journal of Business Research, Elsevier, vol. 68(12), pages 2603-2609.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wallach, Karen Anne & Popovich, Deidre, 2023. "When Big Is Less than Small: Why dominant brands lack authenticity in their sustainability initiatives," Journal of Business Research, Elsevier, vol. 158(C).
    2. Lukas Maier & Christian V. Baccarella & Jörn H. Block & Timm F. Wagner & Kai-Ingo Voigt, 2023. "The Legitimization Effect of Crowdfunding Success: A Consumer Perspective," Entrepreneurship Theory and Practice, , vol. 47(4), pages 1389-1420, July.
    3. Chawla, Yash & Chodak, Grzegorz, 2021. "Social media marketing for businesses: Organic promotions of web-links on Facebook," Journal of Business Research, Elsevier, vol. 135(C), pages 49-65.
    4. Manuela Fritz & Michael Grimm & Ingmar Weber & Elad Yom-Tov & Benedictus Praditya, 2022. "Uncover your risk! Using Facebook to increase personal risk awareness and screening of type 2 diabetes in Indonesia," Working Papers 221, Bavarian Graduate Program in Economics (BGPE).
    5. Krallman, Alexandra & Barnes, Donald C. & Lastner, Matthew M. & Collier, Joel E., 2023. "You can’t touch this: Driving purchase justification for hedonic online purchases," Journal of Business Research, Elsevier, vol. 155(PB).
    6. Zhang, Jin & Xu, Xiaobing & Keh, Hean Tat, 2022. "I implement, they deliberate: The matching effects of point of view and mindset on consumer attitudes," Journal of Business Research, Elsevier, vol. 139(C), pages 397-410.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thieme, Lutz & Winkelhake, Olaf & Hartmann, Ulrich, 2014. "Fairness als universelle Norm? Empirische Evidenz ohne Manna [Fairness as a universal norm? Empiric evidence without manna]," Working Papers of the European Institute for Socioeconomics 12, European Institute for Socioeconomics (EIS), Saarbrücken.
    2. Bouma, J.A. & Nguyen, Binh & van der Heijden, Eline & Dijk, J.J., 2018. "Analysing Group Contract Design Using a Lab and a Lab-in-the-Field Threshold Public Good Experiment," Discussion Paper 2018-049, Tilburg University, Center for Economic Research.
    3. Anastasia Stathopoulou & Tommy Kweku Quansah & George Balabanis, 2022. "The Blinding Effects of Team Identification on Sports Corruption: Cross-Cultural Evidence from Sub-Saharan African Countries," Journal of Business Ethics, Springer, vol. 179(2), pages 511-529, August.
    4. Zizzo, Daniel John, 2013. "Claims and confounds in economic experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 93(C), pages 186-195.
    5. Shir-Way Siew & Michael S. Minor & Reto Felix, 2018. "The influence of perceived strength of brand origin on willingness to pay more for luxury goods," Journal of Brand Management, Palgrave Macmillan, vol. 25(6), pages 591-605, November.
    6. Efrat Dressler & Yevgeny Mugerman, 2023. "Doing the Right Thing? The Voting Power Effect and Institutional Shareholder Voting," Journal of Business Ethics, Springer, vol. 183(4), pages 1089-1112, April.
    7. Ellen, B. Parker & Mackey, Jeremy D. & McAllister, Charn P. & Mercer, Ian S., 2022. "Are small measures big problems? A meta-analytic investigation of brief measures of the Big Five," Journal of Business Research, Elsevier, vol. 151(C), pages 579-592.
    8. Castillo, Geoffrey, 2020. "The attraction effect and its explanations," Games and Economic Behavior, Elsevier, vol. 119(C), pages 123-147.
    9. Kazi, Oishi & Miller, Steven R. & Malone, Trey & Wolf, Christopher A., 2022. "The Changing Role of Fat Perceptions in Fluid Milk Labeling: Would the Dairy Industry Sell More if 2% Milk Was Called “98% Fat Free”?," Journal of Food Distribution Research, Food Distribution Research Society, vol. 53(2), July.
    10. Casenave, Eric & Klarmann, Martin, 2020. "The accountability paradox: How holding marketers accountable hinders alignment with short-term marketing goals," Journal of Business Research, Elsevier, vol. 112(C), pages 95-108.
    11. Lukas Maier & Christian V. Baccarella & Jörn H. Block & Timm F. Wagner & Kai-Ingo Voigt, 2023. "The Legitimization Effect of Crowdfunding Success: A Consumer Perspective," Entrepreneurship Theory and Practice, , vol. 47(4), pages 1389-1420, July.
    12. Lancellotti, Matthew P. & Thomas, Sunil, 2018. "Men hate it, women love it: Guilty pleasure advertising messages," Journal of Business Research, Elsevier, vol. 85(C), pages 271-280.
    13. van Esch, Patrick & Black, J. Stewart, 2019. "Factors that influence new generation candidates to engage with and complete digital, AI-enabled recruiting," Business Horizons, Elsevier, vol. 62(6), pages 729-739.
    14. Mwesiumo, Deodat & Halpern, Nigel & Budd, Thomas & Suau-Sanchez, Pere & Bråthen, Svein, 2021. "An exploratory and confirmatory composite analysis of a scale for measuring privacy concerns," Journal of Business Research, Elsevier, vol. 136(C), pages 63-75.
    15. Vieira, Valter Afonso & Rafael, Diego Nogueira & Agnihotri, Raj, 2022. "Augmented reality generalizations: A meta-analytical review on consumer-related outcomes and the mediating role of hedonic and utilitarian values," Journal of Business Research, Elsevier, vol. 151(C), pages 170-184.
    16. Tiziana Medda & Vittorio Pelligra & Tommaso Reggiani, 2021. "Lab-Sophistication: Does Repeated Participation in Laboratory Experiments Affect Pro-Social Behaviour?," Games, MDPI, vol. 12(1), pages 1-14, February.
    17. Sharifi, Seyed Shahin & Aghazadeh, Hashem, 2016. "Discount reference moderates customers' reactions to discount frames after online service failure," Journal of Business Research, Elsevier, vol. 69(10), pages 4074-4080.
    18. Xinyu (Judy) Hu & Larissa K. Barber & Alecia M. Santuzzi, 2021. "Does Active Leisure Improve Worker Well-Being? An Experimental Daily Diary Approach," Journal of Happiness Studies, Springer, vol. 22(5), pages 2003-2029, June.
    19. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2023. "Digital public health interventions at scale: The impact of social media advertising on beliefs and outcomes related to COVID vaccines," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(5), pages 2208110120-, January.
    20. Chidambaram, Bhuvanachithra & Janssen, Marco A. & Rommel, Jens & Zikos, Dimitrios, 2014. "Commuters’ mode choice as a coordination problem: A framed field experiment on traffic policy in Hyderabad, India," Transportation Research Part A: Policy and Practice, Elsevier, vol. 65(C), pages 9-22.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jbrese:v:118:y:2020:i:c:p:189-198. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jbusres .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.