IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v185y2022i3p916-932.html
   My bibliography  Save this article

Using a responsive survey design to innovate self‐administered mixed‐mode surveys

Author

Listed:
  • Tobias Gummer
  • Pablo Christmann
  • Sascha Verhoeven
  • Christof Wolf

Abstract

Implementing innovations in surveys often results in uncertainty concerning how different design decisions will affect key performance indicators such as response rates, nonresponse bias, or survey costs. Thus, responsive survey designs have been developed to better cope with such situations. In the present study, we propose a responsive survey design that relies on experimentation in the earlier phases of the survey to decide between different design choices of which—prior to data collection—their impact on performance indicators is uncertain. We applied this design to the European Values Study 2017/2018 in Germany that advanced its general social survey‐type design away from the traditional face‐to‐face mode to self‐administered modes. These design changes resulted in uncertainty as to how different incentive strategies and mode choice sequences would affect response rates, nonresponse bias, and survey costs. We illustrate the application and operation of the proposed responsive survey design, as well as an efficiency issue that accompanies it. We also compare the performance of the responsive survey design to a traditional survey design that would have kept all design characteristics static during the field period.

Suggested Citation

  • Tobias Gummer & Pablo Christmann & Sascha Verhoeven & Christof Wolf, 2022. "Using a responsive survey design to innovate self‐administered mixed‐mode surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 916-932, July.
    • Handle: RePEc:bla:jorssa:v:185:y:2022:i:3:p:916-932
    DOI: 10.1111/rssa.12835
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12835
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssa.12835?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Eleanor Singer & Cong Ye, 2013. "The Use and Effects of Incentives in Surveys," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 112-141, January.
    2. William Axinn & Cynthia Link & Robert Groves, 2011. "Responsive Survey Design, Demographic Data Collection, and Models of Demographic Behavior," Demography, Springer;Population Association of America (PAA), vol. 48(3), pages 1127-1149, August.
    3. Robert M. Groves & Steven G. Heeringa, 2006. "Responsive design for household surveys: tools for actively controlling survey errors and costs," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(3), pages 439-457, July.
    4. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    5. Felderer, Barbara & Müller, Gerrit & Kreuter, Frauke & Winter, Joachim, 2018. "The Effect of Differential Incentives on Attrition Bias: Evidence from the PASS Wave 3 Incentive Experiment," Munich Reprints in Economics 62837, University of Munich, Department of Economics.
    6. Barry Schouten & Fannie Cobben & Peter Lundquist & James Wagner, 2016. "Does more balanced survey response imply less non-response bias?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(3), pages 727-748, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wagner James & West Brady T. & Elliott Michael R. & Coffey Stephanie, 2020. "Comparing the Ability of Regression Modeling and Bayesian Additive Regression Trees to Predict Costs in a Responsive Survey Design Context," Journal of Official Statistics, Sciendo, vol. 36(4), pages 907-931, December.
    2. Särndal Carl-Erik & Lundquist Peter, 2017. "Inconsistent Regression and Nonresponse Bias: Exploring Their Relationship as a Function of Response Imbalance," Journal of Official Statistics, Sciendo, vol. 33(3), pages 709-734, September.
    3. Brick J. Michael & Tourangeau Roger, 2017. "Responsive Survey Designs for Reducing Nonresponse Bias," Journal of Official Statistics, Sciendo, vol. 33(3), pages 735-752, September.
    4. Burger Joep & Perryck Koen & Schouten Barry, 2017. "Robustness of Adaptive Survey Designs to Inaccuracy of Design Parameters," Journal of Official Statistics, Sciendo, vol. 33(3), pages 687-708, September.
    5. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    6. Carl-Erik Särndal & Imbi Traat & Kaur Lumiste, 2018. "Interaction Between Data Collection And Estimation Phases In Surveys With Nonresponse," Statistics in Transition New Series, Polish Statistical Association, vol. 19(2), pages 183-200, June.
    7. Roberts Caroline & Herzing Jessica M.E. & Vandenplas Caroline, 2020. "A Validation of R-Indicators as a Measure of the Risk of Bias using Data from a Nonresponse Follow-Up Survey," Journal of Official Statistics, Sciendo, vol. 36(3), pages 675-701, September.
    8. Holly Matulewicz & Eric Grau & Arif Mamun & Gina Livermore, "undated". "Promoting Readiness of Minors in Supplemental Security Income (PROMISE): PROMISE 60-Month Sampling and Survey Plan," Mathematica Policy Research Reports be402161c12e402392af9182e, Mathematica Policy Research.
    9. van Berkel Kees & van der Doef Suzanne & Schouten Barry, 2020. "Implementing Adaptive Survey Design With an Application to the Dutch Health Survey," Journal of Official Statistics, Sciendo, vol. 36(3), pages 609-629, September.
    10. Stephanie Coffey, PhD. & Jaya Damineni & John Eltinge, PhD. & Anup Mathur, PhD. & Kayla Varela & Allison Zotti, 2023. "Some Open Questions on Multiple-Source Extensions of Adaptive-Survey Design Concepts and Methods," Working Papers 23-03, Center for Economic Studies, U.S. Census Bureau.
    11. Beaumont Jean-Francois & Bocci Cynthia & Haziza David, 2014. "An Adaptive Data Collection Procedure for Call Prioritization," Journal of Official Statistics, Sciendo, vol. 30(4), pages 1-15, December.
    12. J. N. K. Rao, 2021. "On Making Valid Inferences by Integrating Data from Surveys and Other Sources," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 83(1), pages 242-272, May.
    13. Jamie C. Moore & Gabriele B. Durrant & Peter W. F. Smith, 2021. "Do coefficients of variation of response propensities approximate non‐response biases during survey data collection?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 301-323, January.
    14. Särndal Carl-Erik & Traat Imbi & Lumiste Kaur, 2018. "Interaction Between Data Collection And Estimation Phases In Surveys With Nonresponse," Statistics in Transition New Series, Polish Statistical Association, vol. 19(2), pages 183-200, June.
    15. McCarthy Jaki & Wagner James & Sanders Herschel Lisette, 2017. "The Impact of Targeted Data Collection on Nonresponse Bias in an Establishment Survey: A Simulation Study of Adaptive Survey Design," Journal of Official Statistics, Sciendo, vol. 33(3), pages 857-871, September.
    16. Ronald R. Rindfuss & Minja K. Choe & Noriko O. Tsuya & Larry L. Bumpass & Emi Tamaki, 2015. "Do low survey response rates bias results? Evidence from Japan," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 32(26), pages 797-828.
    17. Early Kirstin & Mankoff Jennifer & Fienberg Stephen E., 2017. "Dynamic Question Ordering in Online Surveys," Journal of Official Statistics, Sciendo, vol. 33(3), pages 625-657, September.
    18. Chun Asaph Young & Schouten Barry & Wagner James, 2017. "JOS Special Issue on Responsive and Adaptive Survey Design: Looking Back to See Forward – Editorial: In Memory of Professor Stephen E. Fienberg, 1942–2016," Journal of Official Statistics, Sciendo, vol. 33(3), pages 571-577, September.
    19. Reza C. Daniels, 2012. "A Framework for Investigating Micro Data Quality, with Application to South African Labour Market Household Surveys," SALDRU Working Papers 90, Southern Africa Labour and Development Research Unit, University of Cape Town.
    20. Reist, Benjamin M. & Rodhouse, Joseph B. & Ball, Shane T. & Young, Linda J., 2019. "Subsampling of Nonrespondents in the 2017 Census of Agriculture," NASS Research Reports 322826, United States Department of Agriculture, National Agricultural Statistics Service.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:185:y:2022:i:3:p:916-932. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.