IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2010.05117.html
   My bibliography  Save this paper

Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments

Author

Listed:
  • George Z. Gui

Abstract

Randomized controlled trials generate experimental variation that can credibly identify causal effects, but often suffer from limited scale, while observational datasets are large, but often violate desired identification assumptions. To improve estimation efficiency, I propose a method that leverages imperfect instruments - pretreatment covariates that satisfy the relevance condition but may violate the exclusion restriction. I show that these imperfect instruments can be used to derive moment restrictions that, in combination with the experimental data, improve estimation efficiency. I outline estimators for implementing this strategy, and show that my methods can reduce variance by up to 50%; therefore, only half of the experimental sample is required to attain the same statistical precision. I apply my method to a search listing dataset from Expedia that studies the causal effect of search rankings on clicks, and show that the method can substantially improve the precision.

Suggested Citation

  • George Z. Gui, 2020. "Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments," Papers 2010.05117, arXiv.org, revised Dec 2023.
  • Handle: RePEc:arx:papers:2010.05117
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2010.05117
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Guido W. Imbens & Tony Lancaster, 1994. "Combining Micro and Macro Data in Microeconometric Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 655-680.
    2. Susan Athey & Raj Chetty & Guido Imbens, 2020. "Combining Experimental and Observational Data to Estimate Treatment Effects on Long Term Outcomes," Papers 2006.09676, arXiv.org.
    3. Raluca M. Ursu, 2018. "The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions," Marketing Science, INFORMS, vol. 37(4), pages 530-552, August.
    4. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    5. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    6. Randall A. Lewis & Justin M. Rao, 2015. "The Unfavorable Economics of Measuring the Returns to Advertising," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(4), pages 1941-1973.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2023. "Digital public health interventions at scale: The impact of social media advertising on beliefs and outcomes related to COVID vaccines," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(5), pages 2208110120-, January.
    2. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    3. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    4. Brett R Gordon & Kinshuk Jerath & Zsolt Katona & Sridhar Narayanan & Jiwoong Shin & Kenneth C Wilbur, 2019. "Inefficiencies in Digital Advertising Markets," Papers 1912.09012, arXiv.org, revised Feb 2020.
    5. Ron Berman & Christophe Van den Bulte, 2022. "False Discovery in A/B Testing," Management Science, INFORMS, vol. 68(9), pages 6762-6782, September.
    6. Andrey Simonov & Shawndra Hill, 2021. "Competitive Advertising on Brand Search: Traffic Stealing and Click Quality," Marketing Science, INFORMS, vol. 40(5), pages 923-945, September.
    7. Berman, Ron & Heller, Yuval, 2020. "Naive Analytics Equilibrium," MPRA Paper 103824, University Library of Munich, Germany.
    8. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2022. "The Effectiveness of Digital Interventions on COVID-19 Attitudes and Beliefs," Papers 2206.10214, arXiv.org.
    9. Thomas W. Frick & Rodrigo Belo & Rahul Telang, 2023. "Incentive Misalignments in Programmatic Advertising: Evidence from a Randomized Field Experiment," Management Science, INFORMS, vol. 69(3), pages 1665-1686, March.
    10. Bradley T. Shapiro, 2020. "Advertising in Health Insurance Markets," Marketing Science, INFORMS, vol. 39(3), pages 587-611, May.
    11. Randall Lewis & Dan Nguyen, 2015. "Display advertising’s competitive spillovers to consumer search," Quantitative Marketing and Economics (QME), Springer, vol. 13(2), pages 93-115, June.
    12. Jacob LaRiviere & Mikolaj Czajkowski & Nick Hanley & Katherine Simpson, 2016. "What is the Causal Impact of Knowledge on Preferences in Stated Preference Studies?," Working Papers 2016-12, Faculty of Economic Sciences, University of Warsaw.
    13. Tobias Regner, 2021. "Crowdfunding a monthly income: an analysis of the membership platform Patreon," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 45(1), pages 133-142, March.
    14. Wei Zhou & Zidong Wang, 2020. "Competing for Search Traffic in Query Markets: Entry Strategy, Platform Design, and Entrepreneurship," Working Papers 20-12, NET Institute.
    15. Uddin, Main & Wang, Liang Choon & Smyth, Russell, 2021. "Do government-initiated energy comparison sites encourage consumer search and lower prices? Evidence from an online randomized controlled experiment in Australia," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 167-182.
    16. Adena, Maja & Hager, Anselm, 2024. "Does online fundraising increase charitable giving? A nationwide field experiment on Facebook," Discussion Papers, Research Unit: Economics of Change SP II 2020-302r2, WZB Berlin Social Science Center, revised 2024.
    17. Xiang Hui & Meng Liu, 2022. "Quality Certificates Alleviate Consumer Aversion to Sponsored Search Advertising," CESifo Working Paper Series 9886, CESifo.
    18. Bart J. Bronnenberg & Jean-Pierre Dubé & Chad Syverson, 2022. "Marketing Investment and Intangible Brand Capital," Journal of Economic Perspectives, American Economic Association, vol. 36(3), pages 53-74, Summer.
    19. George Gui & Harikesh Nair & Fengshi Niu, 2021. "Auction Throttling and Causal Inference of Online Advertising Effects," Papers 2112.15155, arXiv.org, revised Feb 2022.
    20. Naoki Aizawa & You Suk Kim, 2020. "Public and Private Provision of Information in Market-Based Public Programs: Evidence from Advertising in Health Insurance Marketplaces," NBER Working Papers 27695, National Bureau of Economic Research, Inc.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2010.05117. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.