Adaptive Idea Screening Using Consumers

Following a successful idea generation exercise, a company might easily be left with hundreds of ideas generated by experts, employees, or consumers. The next step is to screen these ideas and identify those with the highest potential. In this paper we propose a practical approach to involving consumers in idea screening. Although the number of ideas may potentially be very large, it would be unreasonable to ask each consumer to evaluate more than a few ideas. This raises the challenge of efficiently selecting the ideas to be evaluated by each consumer. We describe several idea-screening algorithms that perform this selection adaptively based on the evaluations made by previous consumers. We use simulations to compare and analyze the performance of the algorithms as well as to understand their behavior. The best-performing algorithm focuses on the ideas that are the most likely to have been misclassified (as “top” or “bottom” ideas) based on the previous evaluations, and avoids discarding ideas too fast by adding random perturbations to the misclassification probabilities. We demonstrate the convergent validity of this algorithm using a field experiment, which also confirms the convergence pattern predicted by simulations.