Easier Crowdsourcing Is Better: Designing Crowdsourcing Systems to Increase Information Quality and User Participation

Crowdsourcing promises to expand organizational knowledge and “sensor” networks dramatically, making it possible to engage ordinary people in large-scale data collection, often at much lower cost than that of traditional approaches to gathering data. A major challenge in crowdsourcing is ensuring that the data that crowds provide is of sufficient quality to be usable in organizational decision-making and analysis. We refer to this challenge as the Problem of Crowd Information Quality (Crowd IQ). We need to increase quality while giving contributors the flexibility to contribute data based on their individual perceptions. The design science research project produced several artifacts, including a citizen science information system (NLNature), design principles (guidelines) for the development of crowdsourcing projects, and an instance-based crowdsourcing design theory. We also made several methodological contributions related to the process of design science research and behavioral research in information systems. Over the course of the project, we addressed several challenges in designing crowdsourcing systems, formulating design principles, and conducting rigorous design science research. Specifically, we showed that: design choices can have a sizable impact in the real world; it can be unclear how to implement design principles; and design features that are unrelated to design principles can confound efforts to evaluate artifacts. During the project, we also experienced challenges for which no adequate solution was found, reaffirming that design is an iterative process.