IDEAS home Printed from https://ideas.repec.org/a/zag/market/v28y2016i1p93-109.html
   My bibliography  Save this article

Open-Rate Controlled Experiment in E-Mail Marketing Campaigns

Author

Listed:
  • Antun Biloš

    (Faculty of Economics, University of Osijek)

  • Davorin Turkalj

    (Faculty of Economics, University of Osijek)

  • Ivan Kelić

    (Faculty of Economics, University of Osijek)

Abstract

Purpose – The main purpose of this paper is to test the controlled experiment (A/B split) methodology in B2C oriented e-mail marketing campaigns. Design/Methodology/Approach – E-mail marketing techniques have been a substantial part of e-marketing methodology since the early Internet days of the mid-1990s. From the very beginning of Internet utilization for business purposes, e-mail was one of the most widely used communication techniques in B2B and B2C markets alike. Due to high volumes of spamming and progression of online communication clutter, some practitioners began to question the usability of e-mail as a marketing communication channel, while others embarked on working on improving the message itself. Efforts were invested into improving message quality, as well as into better understanding user expectations. One of the most commonly used techniques to test specific e-mail message elements is the controlled experiment. Findings and implications – This paper explores several types of controlled experiments in a specific Croatian B2C market. Tests were run to determine subscriber behavior towards several newsletter components, including sending time, sending day, sender’s name, and subject line. Open and click rates for tested campaigns, and several other metrics were investigated using MailChimp software. An N − 1 two-proportion test using an adjusted Wald confidence interval around the difference in the proportions was used for comparing the open-rate measure in the controlled experiments between subjects. Limitation – Controlled experiments (A/B split tests) showed a lot of potential as a way of measuring behavior and preferences of subscribers, although several apparent limitations (the data-set scope, comparability issues) indicated a clear need for standardization on a managerial and scientific level. Originality – This paper provides an up-to-date e-mail marketing effectiveness literature review, describes and tests the methodology and metrics for e-mail campaigns measurement, and suggests several important guidelines for further research.

Suggested Citation

  • Antun Biloš & Davorin Turkalj & Ivan Kelić, 2016. "Open-Rate Controlled Experiment in E-Mail Marketing Campaigns," Tržište/Market, Faculty of Economics and Business, University of Zagreb, vol. 28(1), pages 93-109.
    • Handle: RePEc:zag:market:v:28:y:2016:i:1:p:93-109
    as

    Download full text from publisher

    File URL: http://hrcak.srce.hr/file/236637
    Download Restriction: None
    ---><---

    References listed on IDEAS

    as
    1. Ellis-Chadwick, Fiona & Doherty, Neil F., 2012. "Web advertising: The role of e-mail marketing," Journal of Business Research, Elsevier, vol. 65(6), pages 843-848.
    2. De Bruyn, Arnaud & Lilien, Gary L., 2008. "A multi-stage model of word-of-mouth influence through viral marketing," International Journal of Research in Marketing, Elsevier, vol. 25(3), pages 151-163.
    3. André Bonfrer & Xavier Drèze, 2009. "Real-Time Evaluation of E-mail Campaign Performance," Marketing Science, INFORMS, vol. 28(2), pages 251-263, 03-04.
    4. Pantea Carmen & Pop Nicolae Al., 2010. "Email Marketing Campaigns: The Easiest Path From Organizations To Consumers – An Exploratory Assessment," Annals of Faculty of Economics, University of Oradea, Faculty of Economics, vol. 1(1), pages 737-742, July.
    5. Andrea L. Micheaux, 2011. "Managing e-mail advertising frequency from the consumer perspective," Post-Print hal-02504326, HAL.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lorente-Páramo, Ángel J. & Chaparro-Peláez, Julián & Hernández-García, Ángel, 2020. "How to improve e-mail click-through rates – A national culture approach," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    2. Julián Chaparro-Peláez & Ángel Hernández-García & Ángel-José Lorente-Páramo, 2022. "May I have your attention, please? An investigation on opening effectiveness in e-mail marketing," Review of Managerial Science, Springer, vol. 16(7), pages 2261-2284, October.
    3. Lorente-Páramo, Ángel J. & Hernández-García, Ángel & Chaparro-Peláez, Julián, 2020. "Influence of cultural dimensions on promotional e-mail effectiveness," Technological Forecasting and Social Change, Elsevier, vol. 150(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lorente-Páramo, Ángel J. & Hernández-García, Ángel & Chaparro-Peláez, Julián, 2020. "Influence of cultural dimensions on promotional e-mail effectiveness," Technological Forecasting and Social Change, Elsevier, vol. 150(C).
    2. Kumar, Ashish, 2021. "An empirical examination of the effects of design elements of email newsletters on consumers’ email responses and their purchase," Journal of Retailing and Consumer Services, Elsevier, vol. 58(C).
    3. Lorente-Páramo, Ángel J. & Chaparro-Peláez, Julián & Hernández-García, Ángel, 2020. "How to improve e-mail click-through rates – A national culture approach," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    4. Julián Chaparro-Peláez & Ángel Hernández-García & Ángel-José Lorente-Páramo, 2022. "May I have your attention, please? An investigation on opening effectiveness in e-mail marketing," Review of Managerial Science, Springer, vol. 16(7), pages 2261-2284, October.
    5. Ralf van der Lans & Gerrit van Bruggen & Jehoshua Eliashberg & Berend Wierenga, 2010. "A Viral Branching Model for Predicting the Spread of Electronic Word of Mouth," Marketing Science, INFORMS, vol. 29(2), pages 348-365, 03-04.
    6. Yadav, Manjit S. & de Valck, Kristine & Hennig-Thurau, Thorsten & Hoffman, Donna L. & Spann, Martin, 2013. "Social Commerce: A Contingency Framework for Assessing Marketing Potential," Journal of Interactive Marketing, Elsevier, vol. 27(4), pages 311-323.
    7. Romain Cadario, 2015. "The impact of online word-of-mouth on television show viewership: An inverted U-shaped temporal dynamic," Marketing Letters, Springer, vol. 26(4), pages 411-422, December.
    8. Peres, Renana & Muller, Eitan & Mahajan, Vijay, 2010. "Innovation diffusion and new product growth models: A critical review and research directions," International Journal of Research in Marketing, Elsevier, vol. 27(2), pages 91-106.
    9. Fang Di & Richards Timothy J. & Grebitus Carola, 2019. "Modeling Product Choices in a Peer Network," Forum for Health Economics & Policy, De Gruyter, vol. 22(1), pages 1-13, June.
    10. Rodriguez, Virginie & Sangle-Ferriere, Marion, 2023. "Do supermarkets’ emails have any value for their customers? The effect of emails’ content and interestingness on customers’ attitude and engagement," Journal of Retailing and Consumer Services, Elsevier, vol. 75(C).
    11. Christina Katsikari & Leonidas Hatzithomas & Thomas Fotiadis & Dimitrios Folinas, 2020. "Push and Pull Travel Motivation: Segmentation of the Greek Market for Social Media Marketing in Tourism," Sustainability, MDPI, vol. 12(11), pages 1-18, June.
    12. Zahra MajlesiRad & Abdol Hamid Haji pour Shoushtari, 2020. "Analysis of the impact of social network sites and eWOM marketing, considering the reinforcing dimensions of the concept of luxury, on tendency toward luxury brand," Future Business Journal, Springer, vol. 6(1), pages 1-19, December.
    13. Adarsh Anand & Mohini Agarwal & Gunjan Bansal & A. H. S. Garmabaki, 2016. "Studying product diffusion based on market coverage," Journal of Marketing Analytics, Palgrave Macmillan, vol. 4(4), pages 135-146, December.
    14. Kim, Juran & Kang, Seungmook & Lee, Ki Hoon, 2021. "Evolution of digital marketing communication: Bibliometric analysis and network visualization from key articles," Journal of Business Research, Elsevier, vol. 130(C), pages 552-563.
    15. Claus, Bart & Geyskens, Kelly & Millet, Kobe & Dewitte, Siegfried, 2012. "The referral backfire effect: The identity-threatening nature of referral failure," International Journal of Research in Marketing, Elsevier, vol. 29(4), pages 370-379.
    16. Kim, Jikyung (Jeanne) & Kim, Sanghwa & Choi, Jeonghye, 2020. "Purchase now and consume later: Do online and offline environments drive online social interactions and sales?," Journal of Business Research, Elsevier, vol. 120(C), pages 274-285.
    17. Yen-Liang Chen & Chia-Ling Chang & An-Qiao Sung, 2021. "Predicting eWOM’s Influence on Purchase Intention Based on Helpfulness, Credibility, Information Quality and Professionalism," Sustainability, MDPI, vol. 13(13), pages 1-19, July.
    18. Agha Mohammad Ali Kermani, Mehrdad & Fatemi Ardestani, Seyed Farshad & Aliahmadi, Alireza & Barzinpour, Farnaz, 2017. "A novel game theoretic approach for modeling competitive information diffusion in social networks with heterogeneous nodes," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 466(C), pages 570-582.
    19. Colicev, Anatoli & Kumar, Ashish & O'Connor, Peter, 2019. "Modeling the relationship between firm and user generated content and the stages of the marketing funnel," International Journal of Research in Marketing, Elsevier, vol. 36(1), pages 100-116.
    20. Su, Hung Jen & Huang, Yu-An & Brodowsky, Glen & Kim, Hyun Jeong, 2011. "The impact of product placement on TV-induced tourism: Korean TV dramas and Taiwanese viewers," Tourism Management, Elsevier, vol. 32(4), pages 805-814.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zag:market:v:28:y:2016:i:1:p:93-109. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tanja Komarac (email available below). General contact details of provider: https://edirc.repec.org/data/fefzghr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.