IDEAS home Printed from https://ideas.repec.org/p/trr/wpaper/202003.html
   My bibliography  Save this paper

A Study of Discontinuity Effects in Regression Inference based on Web-Augmented Mixed Mode Surveys

Author

Listed:
  • Jan Pablo Burgard
  • Joscha Krause
  • Ralf Münnich

Abstract

We consider a situation where the sample design of a survey is modified over time in order to save resources. The former design is a classical large-scale survey. The new design is a mixed mode survey where a smaller classical sample is augmented by records of an online survey. For the online survey no inclusion probabilities are available. We study how this change of data collection affects regression coefficient estimation when the model remains constant in the population over time. Special emphasis is placed on situations where the online records are selective with respect to the model. We develop a statistical framework to quantify so-called survey discontinuities in regression analysis. The term refers to differences between coefficient estimates that solely stem from the survey redesign. For this purpose, we apply hypothesis tests to identify whether observed differences in estimates are significant. Further, we discuss propensity estimation and calibration as potential methods to reduce selection biases stemming from the web survey. A Monte Carlo simulation study is conducted to test the methods under different degrees of selectivity. We find that even mild informativeness significantly impairs regression inference relative to the former survey despite bias correction.

Suggested Citation

  • Jan Pablo Burgard & Joscha Krause & Ralf Münnich, 2020. "A Study of Discontinuity Effects in Regression Inference based on Web-Augmented Mixed Mode Surveys," Research Papers in Economics 2020-03, University of Trier, Department of Economics.
    • Handle: RePEc:trr:wpaper:202003
    as

    Download full text from publisher

    File URL: http://www.uni-trier.de/fileadmin/fb4/prof/VWL/EWF/Research_Papers/2020-03.pdf
    File Function: First version, 2020
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jan Pablo Burgard & Ralf Münnich & Martin Rupp, 2019. "A Generalized Calibration Approach Ensuring Coherent Estimates with Small Area Constraints," Research Papers in Economics 2019-10, University of Trier, Department of Economics.
    2. Denis Devaud & Yves Tillé, 2019. "Deville and Särndal’s calibration: revisiting a 25-years-old successful optimization problem," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 28(4), pages 1033-1065, December.
    3. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555, January.
    4. Denis Devaud & Yves Tillé, 2019. "Rejoinder on: Deville and Särndal’s calibration: revisiting a 25-year-old successful optimization problem," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 28(4), pages 1087-1091, December.
    5. Jan van den Brakel & Xichuan (Mark) Zhang & Siu‐Ming Tam, 2020. "Measuring Discontinuities in Time Series Obtained with Repeated Sample Surveys," International Statistical Review, International Statistical Institute, vol. 88(1), pages 155-175, April.
    6. Emilio Zagheni & Ingmar Weber, 2015. "Demographic research with non-representative internet data," International Journal of Manpower, Emerald Group Publishing Limited, vol. 36(1), pages 13-25, April.
    7. Jan Pablo Burgard & Jan-Philipp Kolb & Hariolf Merkle & Ralf Münnich, 2017. "Synthetic data for open and reproducible methodological research in social sciences and official statistics," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 11(3), pages 233-244, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anne Konrad & Jan Pablo Burgard & Ralf Münnich, 2021. "A Two‐level GREG Estimator for Consistent Estimation in Household Surveys," International Statistical Review, International Statistical Institute, vol. 89(3), pages 635-656, December.
    2. Alessio Guandalini & Claudio Ceccarelli, 2022. "Impact measurement and dimension reduction of auxiliary variables in calibration estimator using the Shapley decomposition," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 31(4), pages 759-784, October.
    3. Zhifeng Gao & Ted C. Schroeder, 2009. "Consumer responses to new food quality information: are some consumers more sensitive than others?," Agricultural Economics, International Association of Agricultural Economists, vol. 40(3), pages 339-346, May.
    4. Cheng, Leilei & Yin, Changbin & Chien, Hsiaoping, 2015. "Demand for milk quantity and safety in urban China: evidence from Beijing and Harbin," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 59(2), April.
    5. Johannes Buggle & Thierry Mayer & Seyhun Orcan Sakalli & Mathias Thoenig, 2023. "The Refugee’s Dilemma: Evidence from Jewish Migration out of Nazi Germany," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(2), pages 1273-1345.
    6. Christelis, Dimitris & Dobrescu, Loretti I. & Motta, Alberto, 2020. "Early life conditions and financial risk-taking in older age," The Journal of the Economics of Ageing, Elsevier, vol. 17(C).
    7. Ortega, David L. & Wang, H. Holly & Wu, Laping & Hong, Soo Jeong, 2015. "Retail channel and consumer demand for food quality in China," China Economic Review, Elsevier, vol. 36(C), pages 359-366.
    8. Doyle, Orla & Fidrmuc, Jan, 2006. "Who favors enlargement?: Determinants of support for EU membership in the candidate countries' referenda," European Journal of Political Economy, Elsevier, vol. 22(2), pages 520-543, June.
    9. Tovar, Jorge, 2012. "Consumers’ Welfare and Trade Liberalization: Evidence from the Car Industry in Colombia," World Development, Elsevier, vol. 40(4), pages 808-820.
    10. Pereira, Pedro & Ribeiro, Tiago, 2011. "The impact on broadband access to the Internet of the dual ownership of telephone and cable networks," International Journal of Industrial Organization, Elsevier, vol. 29(2), pages 283-293, March.
    11. Simon P. Anderson & André de Palma, 2012. "Competition for attention in the Information (overload) Age," RAND Journal of Economics, RAND Corporation, vol. 43(1), pages 1-25, March.
    12. Mtimet, Nadhem & Ujiie, Kiyokazu & Kashiwagi, Kenichi & Zaibet, Lokman & Nagaki, Masakazu, 2011. "The effects of Information and Country of Origin on Japanese Olive Oil Consumer Selection," 2011 International Congress, August 30-September 2, 2011, Zurich, Switzerland 114642, European Association of Agricultural Economists.
    13. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M. & Mjelde, James W., 2020. "Product availability in discrete choice experiments with private goods," Journal of choice modelling, Elsevier, vol. 36(C).
    14. Doherty, Edel & Campbell, Danny, 2011. "Demand for improved food safety and quality: a cross-regional comparison," 85th Annual Conference, April 18-20, 2011, Warwick University, Coventry, UK 108791, Agricultural Economics Society.
    15. Abdurrahman B. Aydemir & Erkan Duman, 2021. "Migrant Networks and Destination Choice: Evidence from Moves across Turkish Provinces," Koç University-TUSIAD Economic Research Forum Working Papers 2109, Koc University-TUSIAD Economic Research Forum.
    16. Brown, Sarah & Greene, William H. & Harris, Mark N. & Taylor, Karl, 2015. "An inverse hyperbolic sine heteroskedastic latent class panel tobit model: An application to modelling charitable donations," Economic Modelling, Elsevier, vol. 50(C), pages 228-236.
    17. Divine Ikenwilo & Sebastian Heidenreich & Mandy Ryan & Colette Mankowski & Jameel Nazir & Verity Watson, 2018. "The Best of Both Worlds: An Example Mixed Methods Approach to Understand Men’s Preferences for the Treatment of Lower Urinary Tract Symptoms," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 11(1), pages 55-67, February.
    18. Paleti, Rajesh, 2018. "Generalized multinomial probit Model: Accommodating constrained random parameters," Transportation Research Part B: Methodological, Elsevier, vol. 118(C), pages 248-262.
    19. Grzybowski, Lukasz & Hasbi, Maude & Liang, Julienne, 2018. "Transition from copper to fiber broadband: The role of connection speed and switching costs," Information Economics and Policy, Elsevier, vol. 42(C), pages 1-10.
    20. Filiz-Ozbay, Emel & Guryan, Jonathan & Hyndman, Kyle & Kearney, Melissa & Ozbay, Erkut Y., 2015. "Do lottery payments induce savings behavior? Evidence from the lab," Journal of Public Economics, Elsevier, vol. 126(C), pages 1-24.

    More about this item

    Keywords

    Calibration; hypothesis test; informative sampling; propensity score estimation;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:trr:wpaper:202003. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Matthias Neuenkirch (email available below). General contact details of provider: https://edirc.repec.org/data/petride.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.