IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/qznpk.html

Experiments on Crowdsourcing Policy Assessment

Author

Listed:
  • Prpić, John

Abstract

Can Crowds serve as useful allies in policy design? How do non-expert Crowds perform relative to experts in the assessment of policy measures? Does the geographic location of non-expert Crowds, with relevance to the policy context, alter the performance of non-experts Crowds in the assessment of policy measures? In this work, we investigate these questions by undertaking experiments designed to replicate expert policy assessments with non-expert Crowds recruited from Virtual Labor Markets. We use a set of ninety six climate change adaptation policy measures previously evaluated by experts in the Netherlands as our control condition to conduct experiments using two discrete sets of non-expert Crowds recruited from Virtual Labor Markets. We vary the composition of our non-expert Crowds along two conditions: participants recruited from a geographical location directly relevant to the policy context and participants recruited at-large. We discuss our research methods in detail and provide the findings of our experiments. Prpić, J., Taeihagh, A., & Melton, J. (2014). Experiments on Crowdsourcing Policy Assessment. Oxford Internet Institute, University of Oxford - IPP 2014 - Crowdsourcing for Politics and Policy.

Suggested Citation

  • Prpić, John, 2017. "Experiments on Crowdsourcing Policy Assessment," SocArXiv qznpk, Center for Open Science.
    • Handle: RePEc:osf:socarx:qznpk
    DOI: 10.31219/osf.io/qznpk
    as

    Download full text from publisher

    File URL: https://osf.io/download/589d6bab594d9001fcddd15b/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/qznpk?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ben Taieb, Souhaib & Hyndman, Rob J., 2014. "A gradient boosting approach to the Kaggle load forecasting competition," International Journal of Forecasting, Elsevier, vol. 30(2), pages 382-394.
    2. Kelly, Charlotte & May, Anthony D. & Jopson, Ann, 2008. "The development of an option generation tool to identify potential transport policy packages," Transport Policy, Elsevier, vol. 15(6), pages 361-371, November.
    3. Karim R. Lakhani & Jill A. Panetta, 2007. "The Principles of Distributed Innovation," Innovations: Technology, Governance, Globalization, MIT Press, vol. 2(3), pages 97-112, July.
    4. Gabriele Paolacci & Jesse Chandler & Panagiotis G. Ipeirotis, 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(5), pages 411-419, August.
    5. Taeihagh, Araz & Bañares-Alcántara, René & Givoni, Moshe, 2014. "A virtual environment for the formulation of policy packages," Transportation Research Part A: Policy and Practice, Elsevier, vol. 60(C), pages 53-68.
    6. Kietzmann, Jan H. & Hermkens, Kristopher & McCarthy, Ian P. & Silvestre, Bruno S., 2011. "Social media? Get serious! Understanding the functional building blocks of social media," Business Horizons, Elsevier, vol. 54(3), pages 241-251, May.
    7. Triparna de Vreede & Cuong Nguyen & Gert-Jan de Vreede & Imed Boughzala & Onook Oh & Roni Reiter-Palmon, 2013. "A theoretical model of user engagement in crowdsourcing," Grenoble Ecole de Management (Post-Print) hal-01244682, HAL.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Prpić, John & Shukla, Prashant P. & Kietzmann, Jan H. & McCarthy, Ian P., 2015. "How to work a crowd: Developing crowd capital through crowdsourcing," Business Horizons, Elsevier, vol. 58(1), pages 77-85.
    2. repec:osf:socarx:jer9k_v1 is not listed on IDEAS
    3. Prpić, John, 2017. "A Framework for Policy Crowdsourcing," SocArXiv pmfdx, Center for Open Science.
    4. Prpić, John, 2017. "MOOCs and Crowdsourcing: Massive Courses and Massive Resources," SocArXiv uwess, Center for Open Science.
    5. Prpić, John, 2017. "How To Work A Crowd: Developing Crowd Capital Through Crowdsourcing," SocArXiv jer9k, Center for Open Science.
    6. Araz Taeihagh, 2017. "Crowdsourcing: a new tool for policy-making?," Policy Sciences, Springer;Society of Policy Sciences, vol. 50(4), pages 629-647, December.
    7. Prpić, John, 2017. "The Fundamentals of Policy Crowdsourcing," SocArXiv wdtvh, Center for Open Science.
    8. repec:osf:socarx:pmfdx_v1 is not listed on IDEAS
    9. Araz Taeihagh, 2017. "Network-centric policy design," Policy Sciences, Springer;Society of Policy Sciences, vol. 50(2), pages 317-338, June.
    10. Täuscher, Karl, 2017. "Leveraging collective intelligence: How to design and manage crowd-based business models," Business Horizons, Elsevier, vol. 60(2), pages 237-245.
    11. Moeinaddini, Amin & Habibian, Meeghat, 2024. "Acceptability of transportation demand management policy packages considering interactions and socio-economic heterogeneity," Research in Transportation Economics, Elsevier, vol. 103(C).
    12. Anna Dreber & Tore Ellingsen & Magnus Johannesson & David Rand, 2013. "Do people care about social context? Framing effects in dictator games," Experimental Economics, Springer;Economic Science Association, vol. 16(3), pages 349-371, September.
    13. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    14. repec:bcp:journl:v:4:y:2020:i:6:p:338-345 is not listed on IDEAS
    15. Kenneth D. Nguyen & Heather Rosoff & Richard S. John, 2017. "Valuing Equal Protection in Aviation Security Screening," Risk Analysis, John Wiley & Sons, vol. 37(12), pages 2405-2419, December.
    16. Frederiks, Arjan J. & Englis, Basil G. & Ehrenhard, Michel L. & Groen, Aard J., 2019. "Entrepreneurial cognition and the quality of new venture ideas: An experimental approach to comparing future-oriented cognitive processes," Journal of Business Venturing, Elsevier, vol. 34(2), pages 327-347.
    17. Nguyen Hoang Thuan & Pedro Antunes & David Johnstone, 2016. "Factors influencing the decision to crowdsource: A systematic literature review," Information Systems Frontiers, Springer, vol. 18(1), pages 47-68, February.
    18. Martín-Rojas, Rodrigo & García-Morales, Victor J. & Garrido-Moreno, Aurora & Salmador-Sánchez, Maria Paz, 2021. "Social Media Use and the Challenge of Complexity: Evidence from the Technology Sector," Journal of Business Research, Elsevier, vol. 129(C), pages 621-640.
    19. Ciaran Heavey & Zeki Simsek & Christina Kyprianou & Marten Risius, 2020. "How do strategic leaders engage with social media? A theoretical framework for research and practice," Strategic Management Journal, Wiley Blackwell, vol. 41(8), pages 1490-1527, August.
    20. Uttara M. Ananthakrishnan & Beibei Li & Michael D. Smith, 2020. "A Tangled Web: Should Online Review Portals Display Fraudulent Reviews?," Information Systems Research, INFORMS, vol. 31(3), pages 950-971, September.
    21. Smith, Andrew N. & Fischer, Eileen & Yongjian, Chen, 2012. "How Does Brand-related User-generated Content Differ across YouTube, Facebook, and Twitter?," Journal of Interactive Marketing, Elsevier, vol. 26(2), pages 102-113.
    22. Varun Dutt & Cleotilde Gonzalez, 2013. "Enabling Eco-Friendly Choices by Relying on the Proportional-Thinking Heuristic," Sustainability, MDPI, vol. 5(1), pages 1-15, January.
    23. Logan S. Casey & Jesse Chandler & Adam Seth Levine & Andrew Proctor & Dara Z. Strolovitch, 2017. "Intertemporal Differences Among MTurk Workers: Time-Based Sample Variations and Implications for Online Data Collection," SAGE Open, , vol. 7(2), pages 21582440177, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:qznpk. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.