IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0225432.html
   My bibliography  Save this article

Whose data can we trust: How meta-predictions can be used to uncover credible respondents in survey data

Author

Listed:
  • Sonja Radas
  • Drazen Prelec

Abstract

Many areas of economics use subjective data, although it had been known to present problems regarding its reliability. To improve data quality, researchers may use scoring rules that reward respondents so that it is most profitable for them to tell the truth. However, if the subjects are not well informed about the topic or if they do not pay sufficient attention, they will produce data that could not be dependably used for decision-making even though subjects gave their honest answer. In this paper we show how meta-predictions (respondents’ predictions about choices of others) can be used for identification of respondents who produce dependable data. We use purchase intention survey, a popular method to elicit early adoption forecasts for a new concept, as a test bed for our approach. We present results from three online experiments, demonstrating that corrected purchase intentions are closer to the real outcomes.

Suggested Citation

  • Sonja Radas & Drazen Prelec, 2019. "Whose data can we trust: How meta-predictions can be used to uncover credible respondents in survey data," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-16, December.
    • Handle: RePEc:plo:pone00:0225432
    DOI: 10.1371/journal.pone.0225432
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0225432
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0225432&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0225432?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. M. Galesic & W. Bruine de Bruin & M. Dumas & A. Kapteyn & J. E. Darling & E. Meijer, 2018. "Asking about social circles improves election predictions," Nature Human Behaviour, Nature, vol. 2(3), pages 187-193, March.
    2. S. Balasubramanian & S. Gupta & W. Kamakura & M. Wedel, 1998. "Modeling large data sets in marketing," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 52(3), pages 303-323, November.
    3. Dražen Prelec & H. Sebastian Seung & John McCoy, 2017. "A solution to the single-question crowd wisdom problem," Nature, Nature, vol. 541(7638), pages 532-535, January.
    4. Manski, Charles F, 1990. "Nonparametric Bounds on Treatment Effects," American Economic Review, American Economic Association, vol. 80(2), pages 319-323, May.
    5. Leisch, Friedrich, 2004. "FlexMix: A General Framework for Finite Mixture Models and Latent Class Regression in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 11(i08).
    6. Manohar U. Kalwani & Alvin J. Silk, 1982. "On the Reliability and Predictive Validity of Purchase Intention Measures," Marketing Science, INFORMS, vol. 1(3), pages 243-286.
    7. David Court & Benjamin Gillen & Jordi McKenzie & Charles R. Plott, 2018. "Two information aggregation mechanisms for predicting the opening weekend box office revenues of films: Boxoffice Prophecy and Guess of Guesses," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 65(1), pages 25-54, January.
    8. Morwitz, Vicki G. & Steckel, Joel H. & Gupta, Alok, 2007. "When do purchase intentions predict sales?," International Journal of Forecasting, Elsevier, vol. 23(3), pages 347-364.
    9. Michel Wedel & Wayne DeSarbo, 1995. "A mixture likelihood approach for generalized linear models," Journal of Classification, Springer;The Classification Society, vol. 12(1), pages 21-55, March.
    10. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    11. Michel Wedel & Wagner Kamakura, 2001. "Factor analysis with (mixed) observed and latent variables in the exponential family," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 515-530, December.
    12. Sun, Baohong & Morwitz, Vicki G., 2010. "Stated intentions and purchase behavior: A unified model," International Journal of Research in Marketing, Elsevier, vol. 27(4), pages 356-366.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Baillon, Aurélien & Bleichrodt, Han & Granic, Georg D., 2022. "Incentives in surveys," Journal of Economic Psychology, Elsevier, vol. 93(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dong, Changgui & Sigrin, Benjamin, 2019. "Using willingness to pay to forecast the adoption of solar photovoltaics: A “parameterization + calibration” approach," Energy Policy, Elsevier, vol. 129(C), pages 100-110.
    2. Domenico Piccolo & Rosaria Simone, 2019. "The class of cub models: statistical foundations, inferential issues and empirical evidence," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 28(3), pages 389-435, September.
    3. Dai, Min & Jia, Yanwei & Kou, Steven, 2021. "The wisdom of the crowd and prediction markets," Journal of Econometrics, Elsevier, vol. 222(1), pages 561-578.
    4. Horstmann, Felix, 2017. "Measuring the shopper's attitude toward the point of sale display: Scale development and validation," Journal of Retailing and Consumer Services, Elsevier, vol. 36(C), pages 112-123.
    5. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    6. Salvatore Ingrassia & Simona Minotti & Giorgio Vittadini, 2012. "Local Statistical Modeling via a Cluster-Weighted Approach with Elliptical Distributions," Journal of Classification, Springer;The Classification Society, vol. 29(3), pages 363-401, October.
    7. Salvatore Ingrassia & Antonio Punzo, 2020. "Cluster Validation for Mixtures of Regressions via the Total Sum of Squares Decomposition," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 526-547, July.
    8. Cecere, Grazia & Corrocher, Nicoletta & Guerzoni, Marco, 2018. "Price or performance? A probabilistic choice analysis of the intention to buy electric vehicles in European countries," Energy Policy, Elsevier, vol. 118(C), pages 19-32.
    9. Nguyen, Cathy & Faulkner, Margaret & Yang, Song & Williams, John & Tong, Luqiong, 2022. "Mind the gap: Understanding the gap between intentions and behaviour in the charity context," Journal of Business Research, Elsevier, vol. 148(C), pages 216-224.
    10. Chen, Cathy W.S. & Chan, Jennifer S.K. & So, Mike K.P. & Lee, Kevin K.M., 2011. "Classification in segmented regression problems," Computational Statistics & Data Analysis, Elsevier, vol. 55(7), pages 2276-2287, July.
    11. Gerhard Tutz & Micha Schneider & Maria Iannario & Domenico Piccolo, 2017. "Mixture models for ordinal responses to account for uncertainty of choice," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 11(2), pages 281-305, June.
    12. Wirtz, Jochen & Xiao, Ping & Chiang, Jeongwen & Malhotra, Naresh, 2014. "Contrasting the Drivers of Switching Intent and Switching Behavior in Contractual Service Settings," Journal of Retailing, Elsevier, vol. 90(4), pages 463-480.
    13. Decker, Reinhold & Trusov, Michael, 2010. "Estimating aggregate consumer preferences from online product reviews," International Journal of Research in Marketing, Elsevier, vol. 27(4), pages 293-307.
    14. Ville A. Satopää & Marat Salikhov & Philip E. Tetlock & Barbara Mellers, 2021. "Bias, Information, Noise: The BIN Model of Forecasting," Management Science, INFORMS, vol. 67(12), pages 7599-7618, December.
    15. Arts, Joep W.C. & Frambach, Ruud T. & Bijmolt, Tammo H.A., 2011. "Generalizations on consumer innovation adoption: A meta-analysis on drivers of intention and behavior," International Journal of Research in Marketing, Elsevier, vol. 28(2), pages 134-144.
    16. Sonja Radas & Dražen Prelec, 2021. "Predicted preference conjoint analysis," PLOS ONE, Public Library of Science, vol. 16(8), pages 1-16, August.
    17. Tom Frans Wilderjans & Eva Gaer & Henk A. L. Kiers & Iven Mechelen & Eva Ceulemans, 2017. "Principal Covariates Clusterwise Regression (PCCR): Accounting for Multicollinearity and Population Heterogeneity in Hierarchically Organized Data," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 86-111, March.
    18. Roberto Mari & Salvatore Ingrassia & Antonio Punzo, 2023. "Local and Overall Deviance R-Squared Measures for Mixtures of Generalized Linear Models," Journal of Classification, Springer;The Classification Society, vol. 40(2), pages 233-266, July.
    19. Satopää, Ville A., 2021. "Improving the wisdom of crowds with analysis of variance of predictions of related outcomes," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1728-1747.
    20. Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0225432. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.