IDEAS home Printed from https://ideas.repec.org/a/vrs/offsta/v33y2017i3p687-708n6.html
   My bibliography  Save this article

Robustness of Adaptive Survey Designs to Inaccuracy of Design Parameters

Author

Listed:
  • Burger Joep

    (Statistics Netherlands, Department of Process Development and Methodology, CBS-weg 11, P.O. Box 4481, 6401 CZ Heerlen, The Netherlands.)

  • Perryck Koen
  • Schouten Barry

    (Statistics Netherlands, Department of Process Development and Methodology, The Hague, The Netherlands.)

Abstract

Adaptive survey designs (ASDs) optimize design features, given 1) the interactions between the design features and characteristics of sampling units and 2) a set of constraints, such as a budget and a minimum number of respondents. Estimation of the interactions is subject to both random and systematic error. In this article, we propose and evaluate four viewpoints to assess robustness of ASDs to inaccuracy of design parameter estimates: the effect of both imprecision and bias on both ASD structure and ASD performance. We additionally propose three distance measures to compare the structure of ASDs. The methodology is illustrated using a simple simulation study and a more complex but realistic case study on the Dutch Travel Survey. The proposed methodology can be applied to other ASD optimization problems. In our simulation study and case study, the ASD was fairly robust to imprecision, but not to realistic dynamics in the design parameters. To deal with the sensitivity of ASDs to changing design parameters, we recommend to learn and update the design parameters.

Suggested Citation

  • Burger Joep & Perryck Koen & Schouten Barry, 2017. "Robustness of Adaptive Survey Designs to Inaccuracy of Design Parameters," Journal of Official Statistics, Sciendo, vol. 33(3), pages 687-708, September.
    • Handle: RePEc:vrs:offsta:v:33:y:2017:i:3:p:687-708:n:6
    DOI: 10.1515/jos-2017-0032
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jos-2017-0032
    Download Restriction: no
    File URL: https://libkey.io/10.1515/jos-2017-0032?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Robert M. Groves & Steven G. Heeringa, 2006. "Responsive design for household surveys: tools for actively controlling survey errors and costs," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(3), pages 439-457, July.
    2. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    3. Calinescu, Melania & Bhulai, Sandjai & Schouten, Barry, 2013. "Optimal resource allocation in survey designs," European Journal of Operational Research, Elsevier, vol. 226(1), pages 115-121.
    4. Barry Schouten & Fannie Cobben & Peter Lundquist & James Wagner, 2016. "Does more balanced survey response imply less non-response bias?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(3), pages 727-748, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tobias Gummer & Pablo Christmann & Sascha Verhoeven & Christof Wolf, 2022. "Using a responsive survey design to innovate self‐administered mixed‐mode surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 916-932, July.
    2. Särndal Carl-Erik & Lundquist Peter, 2017. "Inconsistent Regression and Nonresponse Bias: Exploring Their Relationship as a Function of Response Imbalance," Journal of Official Statistics, Sciendo, vol. 33(3), pages 709-734, September.
    3. McCarthy Jaki & Wagner James & Sanders Herschel Lisette, 2017. "The Impact of Targeted Data Collection on Nonresponse Bias in an Establishment Survey: A Simulation Study of Adaptive Survey Design," Journal of Official Statistics, Sciendo, vol. 33(3), pages 857-871, September.
    4. Brick J. Michael & Tourangeau Roger, 2017. "Responsive Survey Designs for Reducing Nonresponse Bias," Journal of Official Statistics, Sciendo, vol. 33(3), pages 735-752, September.
    5. Chun Asaph Young & Schouten Barry & Wagner James, 2017. "JOS Special Issue on Responsive and Adaptive Survey Design: Looking Back to See Forward – Editorial: In Memory of Professor Stephen E. Fienberg, 1942–2016," Journal of Official Statistics, Sciendo, vol. 33(3), pages 571-577, September.
    6. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    7. Carl-Erik Särndal & Imbi Traat & Kaur Lumiste, 2018. "Interaction Between Data Collection And Estimation Phases In Surveys With Nonresponse," Statistics in Transition New Series, Polish Statistical Association, vol. 19(2), pages 183-200, June.
    8. Roberts Caroline & Herzing Jessica M.E. & Vandenplas Caroline, 2020. "A Validation of R-Indicators as a Measure of the Risk of Bias using Data from a Nonresponse Follow-Up Survey," Journal of Official Statistics, Sciendo, vol. 36(3), pages 675-701, September.
    9. van Berkel Kees & van der Doef Suzanne & Schouten Barry, 2020. "Implementing Adaptive Survey Design With an Application to the Dutch Health Survey," Journal of Official Statistics, Sciendo, vol. 36(3), pages 609-629, September.
    10. Stephanie Coffey, PhD. & Jaya Damineni & John Eltinge, PhD. & Anup Mathur, PhD. & Kayla Varela & Allison Zotti, 2023. "Some Open Questions on Multiple-Source Extensions of Adaptive-Survey Design Concepts and Methods," Working Papers 23-03, Center for Economic Studies, U.S. Census Bureau.
    11. Wagner James & West Brady T. & Elliott Michael R. & Coffey Stephanie, 2020. "Comparing the Ability of Regression Modeling and Bayesian Additive Regression Trees to Predict Costs in a Responsive Survey Design Context," Journal of Official Statistics, Sciendo, vol. 36(4), pages 907-931, December.
    12. J. N. K. Rao, 2021. "On Making Valid Inferences by Integrating Data from Surveys and Other Sources," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 83(1), pages 242-272, May.
    13. Jamie C. Moore & Gabriele B. Durrant & Peter W. F. Smith, 2021. "Do coefficients of variation of response propensities approximate non‐response biases during survey data collection?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 301-323, January.
    14. Lynn, Peter, 2013. "Targeted response inducement strategies on longitudinal surveys," Understanding Society Working Paper Series 2013-02, Understanding Society at the Institute for Social and Economic Research.
    15. Särndal Carl-Erik & Traat Imbi & Lumiste Kaur, 2018. "Interaction Between Data Collection And Estimation Phases In Surveys With Nonresponse," Statistics in Transition New Series, Polish Statistical Association, vol. 19(2), pages 183-200, June.
    16. Eltinge John L. & Biemer Paul P. & Holmberg Anders, 2013. "A Potential Framework for Integration of Architecture and Methodology to Improve Statistical Production Systems," Journal of Official Statistics, Sciendo, vol. 29(1), pages 125-145, March.
    17. Early Kirstin & Mankoff Jennifer & Fienberg Stephen E., 2017. "Dynamic Question Ordering in Online Surveys," Journal of Official Statistics, Sciendo, vol. 33(3), pages 625-657, September.
    18. Reza C. Daniels, 2012. "A Framework for Investigating Micro Data Quality, with Application to South African Labour Market Household Surveys," SALDRU Working Papers 90, Southern Africa Labour and Development Research Unit, University of Cape Town.
    19. Reist, Benjamin M. & Rodhouse, Joseph B. & Ball, Shane T. & Young, Linda J., 2019. "Subsampling of Nonrespondents in the 2017 Census of Agriculture," NASS Research Reports 322826, United States Department of Agriculture, National Agricultural Statistics Service.
    20. Lewis Taylor, 2017. "Univariate Tests for Phase Capacity: Tools for Identifying When to Modify a Survey’s Data Collection Protocol," Journal of Official Statistics, Sciendo, vol. 33(3), pages 601-624, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vrs:offsta:v:33:y:2017:i:3:p:687-708:n:6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.sciendo.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.