IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v41y2012i4p491-534.html
   My bibliography  Save this article

Panel Conditioning in Longitudinal Social Science Surveys

Author

Listed:
  • John Robert Warren
  • Andrew Halpern-Manners

Abstract

Social scientists usually assume that the attitudes, behaviors, and statuses of respondents to longitudinal surveys are not altered by the act of measuring them. If this assumption is false—or even if the quality of survey participants’ responses change because of measurement—then social scientists risk mischaracterizing the existence, magnitude, and correlates of changes across survey waves in respondents’ characteristics. In this article, we make the case that social scientists ought to worry more about panel conditioning biases. We also describe and demonstrate empirical strategies for estimating the magnitude of such biases in longitudinal surveys, and we provide illustrative empirical results that are germane to social science research. We end by outlining a research agenda that would generate specific information about the nature and degree of panel conditioning in specific longitudinal surveys as well as a broader understanding of the circumstances in which panel conditioning is most likely to occur.

Suggested Citation

  • John Robert Warren & Andrew Halpern-Manners, 2012. "Panel Conditioning in Longitudinal Social Science Surveys," Sociological Methods & Research, , vol. 41(4), pages 491-534, November.
    • Handle: RePEc:sae:somere:v:41:y:2012:i:4:p:491-534
    DOI: 10.1177/0049124112460374
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124112460374
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124112460374?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kremer, Michael R. & Karlan, D. S. & Hornbeck, Richard A. & Gine, X. & Duflo, E. & Pariente, W. & Null, C. & Miguel, E. & Devoto, F. & Crepon, B. & Banerjee, A. & Zwane, A. P. & Zinman, J. & Van Dusen, 2011. "Being Surveyed Can Change Later Behavior and Related Parameter Estimates," Scholarly Articles 11339433, Harvard University Department of Economics.
    2. Sharad Borle & Utpal M. Dholakia & Siddharth S. Singh & Robert A. Westbrook, 2007. "The Impact of Survey Participation on Subsequent Customer Behavior: An Empirical Investigation," Marketing Science, INFORMS, vol. 26(5), pages 711-726, 09-10.
    3. Wu C. & Sitter R. R, 2001. "A Model-Calibration Approach to Using Complete Auxiliary Information From Survey Data," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 185-193, March.
    4. Fitzsimons, Gavan J & Morwitz, Vicki G, 1996. "The Effect of Measuring Intent on Brand-Level Purchase Behavior," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 23(1), pages 1-11, June.
    5. Marcel Das & Vera Toepoel & Arthur van Soest, 2011. "Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys," Sociological Methods & Research, , vol. 40(1), pages 32-56, February.
    6. Das, J.W.M. & Toepoel, V. & van Soest, A.H.O., 2007. "Can I use a Panel? Panel Conditioning and Attrition Bias in Panel Surveys," Other publications TiSEM 9407cc7a-23f1-49b9-990d-8, Tilburg University, School of Economics and Management.
    7. Simmons, Carolyn J & Bickart, Barbara A & Lynch, John G, Jr, 1993. "Capturing and Creating Public Opinion in Survey Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 20(2), pages 316-329, September.
    8. Christopher Mann, 2005. "Unintentional voter mobilization: Does participation in pre-election surveys increase voter turnout?," Natural Field Experiments 00305, The Field Experiments Website.
    9. Wilson, Sven E. & Howell, Benjamin L., 2005. "Do panel surveys make people sick? US arthritis trends in the Health and Retirement Study," Social Science & Medicine, Elsevier, vol. 60(11), pages 2623-2627, June.
    10. Gert G. Wagner & Joachim R. Frick & Jürgen Schupp, 2007. "The German Socio-Economic Panel Study (SOEP) – Scope, Evolution and Enhancements," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 127(1), pages 139-169.
    11. Bartels, Larry M., 1999. "Panel Effects in the American National Election Studies," Political Analysis, Cambridge University Press, vol. 8(1), pages 1-20, January.
    12. Alan Gerber & Anton Orlich & Jennifer Smith, 2003. "Self-prophecy effects and voter turnout: An experimental replication," Natural Field Experiments 00333, The Field Experiments Website.
    13. Wilson, Sven & Howell, Benjamin L., 2007. "Disease prevalence and survey design effects: A response to Weir and Smith," Social Science & Medicine, Elsevier, vol. 65(6), pages 1078-1081, September.
    14. Gavan J. Fitzsimons & Joseph C. Nunes & Patti Williams, 2007. "License to Sin: The Liberating Role of Reporting Expectations," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 34(1), pages 22-31, April.
    15. Meurs, Henk & Van Wissen, Leo & Visser, Jacqueline, 1989. "Measurement Biases in Panel Data," University of California Transportation Center, Working Papers qt00q1x266, University of California Transportation Center.
    16. Morwitz, Vicki G & Johnson, Eric J & Schmittlein, David C, 1993. "Does Measuring Intent Change Behavior?," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 20(1), pages 46-61, June.
    17. Granberg, Donald & Holmberg, Sören, 1992. "The Hawthorne Effect in Election Studies: The Impact of Survey Participation on Voting," British Journal of Political Science, Cambridge University Press, vol. 22(2), pages 240-247, April.
    18. Solon, Gary, 1986. "Effects of Rotation Group Bias on Estimation of Unemployment," Journal of Business & Economic Statistics, American Statistical Association, vol. 4(1), pages 105-109, January.
    19. Anthony Onwuegbuzie & Nancy Leech, 2007. "Validity and Qualitative Research: An Oxymoron?," Quality & Quantity: International Journal of Methodology, Springer, vol. 41(2), pages 233-249, April.
    20. Dholakia, Utpal M & Morwitz, Vicki G, 2002. "The Scope and Persistence of Mere-Measurement Effects: Evidence from a Field Study of Customer Satisfaction Measurement," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 29(2), pages 159-167, September.
    21. Robert Voogt & Hetty Van Kempen, 2002. "Nonresponse Bias and Stimulus Effects in the Dutch National Election Study," Quality & Quantity: International Journal of Methodology, Springer, vol. 36(4), pages 325-345, November.
    22. Vicki G. Morwitz, 2005. "The effect of survey measurement on respondent behaviour," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 21(4‐5), pages 451-455, July.
    23. Meurs, Henk & Van Wissen, Leo & Visser, Jacqueline, 1989. "Measurement Biases in Panel Data," University of California Transportation Center, Working Papers qt4095q216, University of California Transportation Center.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Henna Busk & Christine Dauth & Elke J. Jahn, 2017. "Do Changes in Regulation Affect Temporary Agency Workers’ Job Satisfaction?," Industrial Relations: A Journal of Economy and Society, Wiley Blackwell, vol. 56(3), pages 514-544, July.
    2. Sara Yeatman & Christie Sennott & Steven Culpepper, 2013. "Young Women’s Dynamic Family Size Preferences in the Context of Transitioning Fertility," Demography, Springer;Population Association of America (PAA), vol. 50(5), pages 1715-1737, October.
    3. S. C. Noah Uhrig & Nicole Watson, 2020. "The Impact of Measurement Error on Wage Decompositions: Evidence From the British Household Panel Survey and the Household, Income and Labour Dynamics in Australia Survey," Sociological Methods & Research, , vol. 49(1), pages 43-78, February.
    4. Kroh, Martin & Winter, Florin & Schupp, Jürgen, 2016. "Using Person-Fit Measures to Assess the Impact of Panel Conditioning on Reliability," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 80(4), pages 914-942.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrew Halpern-Manners & John Warren, 2012. "Panel Conditioning in Longitudinal Studies: Evidence From Labor Force Items in the Current Population Survey," Demography, Springer;Population Association of America (PAA), vol. 49(4), pages 1499-1519, November.
    2. Bach, Ruben L. & Eckman, Stephanie, 2017. "Does participating in a panel survey change respondents' labor market behavior?," IAB-Discussion Paper 201715, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    3. Omar Merlo & Andreas B. Eisingerich & Wayne D. Hoyer, 2024. "Immunizing customers against negative brand-related information," Journal of the Academy of Marketing Science, Springer, vol. 52(1), pages 140-163, January.
    4. Toepoel, V. & Das, J.W.M. & van Soest, A.H.O., 2008. "Design Effects in Web Surveys : Comparing Trained and Fresh Respondents," Discussion Paper 2008-51, Tilburg University, Center for Economic Research.
    5. Xiaojing Dong & Ramkumar Janakiraman & Ying Xie, 2014. "The Effect of Survey Participation on Consumer Behavior: The Moderating Role of Marketing Communication," Marketing Science, INFORMS, vol. 33(4), pages 567-585, July.
    6. Thomas F. Crossley & Jochem Bresser & Liam Delaney & Joachim Winter, 2017. "Can Survey Participation Alter Household Saving Behaviour?," Economic Journal, Royal Economic Society, vol. 127(606), pages 2332-2357, November.
    7. Sachin Gathani & Maria Paula Gomez & Ricardo Sabates & Dimitri Stoelinga, 2015. "The Effect of Monitoring," Evaluation Review, , vol. 39(6), pages 555-586, December.
    8. Marcel Das & Vera Toepoel & Arthur van Soest, 2011. "Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys," Sociological Methods & Research, , vol. 40(1), pages 32-56, February.
    9. Rene Segers & Philip Hans Franses, 2014. "Panel design effects on response rates and response quality," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 68(1), pages 1-24, February.
    10. Das, J.W.M. & Toepoel, V. & van Soest, A.H.O., 2007. "Can I use a Panel? Panel Conditioning and Attrition Bias in Panel Surveys," Discussion Paper 2007-56, Tilburg University, Center for Economic Research.
    11. Sun, Baohong & Morwitz, Vicki G., 2010. "Stated intentions and purchase behavior: A unified model," International Journal of Research in Marketing, Elsevier, vol. 27(4), pages 356-366.
    12. Marianne Bertrand & Dean S. Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2005. "What's Psychology Worth? A Field Experiment in the Consumer Credit Market," Working Papers 918, Economic Growth Center, Yale University.
    13. Burton, Jonathan & Laurie, Heather & Uhrig, S.C. Noah, 2010. "Understanding Society Innovation Panel Wave 2: results from methodological experiments," Understanding Society Working Paper Series 2010-04, Understanding Society at the Institute for Social and Economic Research.
    14. Amar Cheema & Dipankar Chakravarti & Atanu R. Sinha, 2012. "Bidding Behavior in Descending and Ascending Auctions," Marketing Science, INFORMS, vol. 31(5), pages 779-800, September.
    15. Choi, James J. & Haisley, Emily & Kurkoski, Jennifer & Massey, Cade, 2017. "Small cues change savings choices," Journal of Economic Behavior & Organization, Elsevier, vol. 142(C), pages 378-395.
    16. Agarwal Sanjeev & Teas R. Kenneth, 2005. "Measurement Context Effects in Telephone-Survey-Based Tests of Causal Models," Review of Marketing Science, De Gruyter, vol. 3(1), pages 1-24, July.
    17. Christopher B. Mann, 2005. "Unintentional Voter Mobilization: Does Participation in Preelection Surveys Increase Voter Turnout?," The ANNALS of the American Academy of Political and Social Science, , vol. 601(1), pages 155-168, September.
    18. Tami Kim & Leslie K. John & Todd Rogers & Michael I. Norton, 2019. "Procedural Justice and the Risks of Consumer Voting," Management Science, INFORMS, vol. 65(11), pages 5234-5251, November.
    19. Dikla Perez & Gal Oestreicher-Singer & Lior Zalmanson & Matthew Matan Rubin, 2023. "“No, Thanks”: How Do Requests for Feedback Affect the Consumption Behavior of Non-Compliers?," Marketing Letters, Springer, vol. 34(1), pages 83-97, March.
    20. Auspurg, Katrin & Burton, Jonathan & Cullinane, Carl & Delavande, Adeline & Laura, Fumagalli & Iacovou, Maria & Jäckle, Annette & Kaminska, Olena & Lynn, Peter & Mathews, Paul & Nicolaas, Gerry & Nic, 2013. "Understanding Society Innovation Panel Wave 5: results from methodological experiments," Understanding Society Working Paper Series 2013-06, Understanding Society at the Institute for Social and Economic Research.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:41:y:2012:i:4:p:491-534. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.