IDEAS home Printed from https://ideas.repec.org/p/onb/oenbwp/253.html
   My bibliography  Save this paper

Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)

Author

Listed:
  • Elisabeth Beckmann

    (Oesterreichische Nationalbank, Foreign Research Division)

  • Lukas Olbrich

    (Institute for Employment Research (IAB))

  • Joseph Sakshaug

    (University of Mannheim)

Abstract

Interviewers have long been identified as a source of error in face-to-face surveys. However, previous studies have typically focused on a single source of interviewer-related error and single-country cross-sectional surveys. We extend this literature by investigating the influence of interviewers from multiple dimensions in the Oesterreichische Nationalbank (OeNB) Euro Survey, a cross-national survey conducted annually in ten Central, Eastern, and Southeastern European countries. Using data from ten rounds (i.e., 100 country-years), we first analyze the extent of interviewer variance in financial literacy measures and how these effects compare to other questionnaire items. Building on the previous literature, we also evaluate the stability of these estimates over time and across countries. Second, we apply several data quality indicators on various dimensions of interviewer-related error and investigate country-years with particularly exceptional patterns. Finally, we use a multivariate tree-based outlier detection method (isolation forest) that flags country-years and interviewers with outlying values and combine it with methods from the interpretable machine learning literature to identify the respective exceptional feature values.

Suggested Citation

  • Elisabeth Beckmann & Lukas Olbrich & Joseph Sakshaug, 2024. "Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)," Working Papers 253, Oesterreichische Nationalbank (Austrian Central Bank).
  • Handle: RePEc:onb:oenbwp:253
    as

    Download full text from publisher

    File URL: https://www.oenb.at/dam/jcr:072228b5-6cc5-49f1-8711-6b6db15973b4/wp-253.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ian Brunton-Smith & Patrick Sturgis & George Leckie, 2017. "Detecting and understanding interviewer effects on survey data by using a cross-classified mixed effects location–scale model," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(2), pages 551-568, February.
    2. Thomas F. Crossley & Tobias Schmidt & Panagiota Tzamourani & Joachim K. Winter, 2021. "Interviewer effects and the measurement of financial literacy," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 150-178, January.
    3. Annamaria Lusardi & Olivia S. Mitchell, 2008. "Planning and Financial Literacy: How Do Women Fare?," American Economic Review, American Economic Association, vol. 98(2), pages 413-417, May.
    4. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    5. Elisabeth Beckmann & Sarah Reiter, 2020. "How financially literate is CESEE? Insights from the OeNB Euro Survey," Focus on European Economic Integration, Oesterreichische Nationalbank (Austrian Central Bank), issue Q3/20, pages 36-59.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elisabeth Beckmann & Christa Hainz & Sarah Reiter, 2022. "Third-Party Loan Guarantees: Measuring Literacy and its Effect on Financial Decisions (Elisabeth Beckmann, Christa Hainz, Sarah Reiter)," Working Papers 237, Oesterreichische Nationalbank (Austrian Central Bank).
    2. Thomas F. Crossley & Tobias Schmidt & Panagiota Tzamourani & Joachim K. Winter, 2021. "Interviewer effects and the measurement of financial literacy," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 150-178, January.
    3. Rebecca Walcott & Isabelle Cohen & Denise Ferris, 2024. "When Who Matters: Interviewer Effects and Survey Modality," Evaluation Review, , vol. 48(6), pages 1024-1049, December.
    4. Olbrich, Lukas & Kosyakova, Yuliya & Sakshaug, Joseph W., 2022. "The reliability of adult self-reported height: The role of interviewers," Economics & Human Biology, Elsevier, vol. 45(C).
    5. Boggio, Cecilia & Coda Moscarola, Flavia & Gallice, Andrea, 2020. "What is good for the goose is good for the gander?," Economics of Education Review, Elsevier, vol. 75(C).
    6. Aileen Heinberg & Angela Hung & Arie Kapteyn & Annamaria Lusardi & Anya Savikhin Samek & Joanne Yoong, 2014. "Five steps to planning success: experimental evidence from US households," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 30(4), pages 697-724.
    7. Yoshihiko Kadoya & Mostafa Saidur Rahim Khan, 2018. "Can financial literacy reduce anxiety about life in old age?," Journal of Risk Research, Taylor & Francis Journals, vol. 21(12), pages 1533-1550, December.
    8. Chang Sik Kim & Jonathan Cairns & Valentina Quarantotti & Bogumil Kaczkowski & Yinhai Wang & Peter Konings & Xiang Zhang, 2024. "A statistical simulation model to guide the choices of analytical methods in arrayed CRISPR screen experiments," PLOS ONE, Public Library of Science, vol. 19(8), pages 1-16, August.
    9. Madeleine Seale & Annamaria Kiss & Simone Bovio & Ignazio Maria Viola & Enrico Mastropaolo & Arezki Boudaoud & Naomi Nakayama, 2022. "Dandelion pappus morphing is actuated by radially patterned material swelling," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    10. John Y. Campbell, 2016. "Restoring Rational Choice: The Challenge of Consumer Financial Regulation," American Economic Review, American Economic Association, vol. 106(5), pages 1-30, May.
    11. JANSSENS, Jochen & DE CORTE, Annelies & SÖRENSEN, Kenneth, 2016. "Water distribution network design optimisation with respect to reliability," Working Papers 2016007, University of Antwerp, Faculty of Business and Economics.
    12. Marion Chatelain, 2023. "Endogeic Earthworms Avoid Soil Mimicking Metal Pollution Levels in Urban Parks," Sustainability, MDPI, vol. 15(15), pages 1-17, July.
    13. Bottazzi, Laura & Lusardi, Annamaria, 2021. "Stereotypes in financial literacy: Evidence from PISA," Journal of Corporate Finance, Elsevier, vol. 71(C).
    14. Silva Larson & Anne (Giger)-Dray & Tina Cornioley & Manithaythip Thephavanh & Phomma Thammavong & Sisavan Vorlasan & John G. Connell & Magnus Moglia & Peter Case & Kim S. Alexander & Pascal Perez, 2020. "A Game-Based Approach to Exploring Gender Differences in Smallholder Decisions to Change Farming Practices: White Rice Production in Laos," Sustainability, MDPI, vol. 12(16), pages 1-22, August.
    15. Shunsuke Ono & Pattaphol Yuktadatta & Takafumi Taniguchi & Tomoe Iitsuka & Masafumi Noguchi & Sawa Tanaka & Haruka Ito & Kousei Nakamura & Nanako Yasuhara & Chihiro Miyawaki & Katsumi Mikura & Mostafa, 2021. "Financial Literacy and Exercise Behavior: Evidence from Japan," Sustainability, MDPI, vol. 13(8), pages 1-15, April.
    16. Oberrauch, Luis & Kaiser, Tim, 2020. "Economic competence in early secondary school: Evidence from a large-scale assessment in Germany," International Review of Economics Education, Elsevier, vol. 35(C).
    17. Teruaki Kido & Yuko Yotsumoto & Masamichi J. Hayashi, 2025. "Hierarchical representations of relative numerical magnitudes in the human frontoparietal cortex," Nature Communications, Nature, vol. 16(1), pages 1-15, December.
    18. Sławomir Kujawski & Agnieszka Kujawska & Mariusz Kozakiewicz & Djordje G. Jakovljevic & Błażej Stankiewicz & Julia L. Newton & Kornelia Kędziora-Kornatowska & Paweł Zalewski, 2022. "Effects of Sitting Callisthenic Balance and Resistance Exercise Programs on Cognitive Function in Older Participants," IJERPH, MDPI, vol. 19(22), pages 1-18, November.
    19. Raymond Hernandez & Elizabeth A. Pyatak & Cheryl L. P. Vigen & Haomiao Jin & Stefan Schneider & Donna Spruijt-Metz & Shawn C. Roll, 2021. "Understanding Worker Well-Being Relative to High-Workload and Recovery Activities across a Whole Day: Pilot Testing an Ecological Momentary Assessment Technique," IJERPH, MDPI, vol. 18(19), pages 1-17, October.
    20. Christopher Hassall & Michael Nisbet & Evan Norcliffe & He Wang, 2024. "The Potential Health Benefits of Urban Tree Planting Suggested through Immersive Environments," Land, MDPI, vol. 13(3), pages 1-12, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:onb:oenbwp:253. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Markus Knell (email available below). General contact details of provider: https://edirc.repec.org/data/oenbbat.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.