Exploiting Meta-cognitive Features for a Machine-Learning-Based One-Shot Group-Decision Aggregation

The outcome of collective decision-making often relies on the procedure through which the perspectives of its members are aggregated. Popular aggregation methods, such as the majority rule, often fail to produce the optimal result, especially in high-complexity tasks. Methods that rely on meta-cognitive information, such as confidence-based methods and the Surprisingly Popular answer, have succeeded in various tasks. However, there are still scenarios that result in choosing the incorrect answer. We aim to exploit meta-cognitive information and learn from it, to enhance the group’s ability to produce a correct answer. Specifically, we propose two different feature-representation approaches: Response-Centered feature Representation (RCR), which focuses on the characteristics of the individual response, and Answer-Centered feature Representation (ACR), which focuses on the characteristics of each of the potential answers. Using these two feature-representation approaches, we train machine-learning models to predict the correctness of a response and an answer. The trained models are used in our two proposed aggregation approaches: (1) The Response-Prediction (RP) approach aggregates the results of the group’s votes by exploiting the RCR feature-engineering approach; (2) The Answer-Prediction (AP) approach aggregates the results of the group’s votes by exploiting the ACR feature-engineering approach. To evaluate our methodology, we collected 2514 responses for different tasks. The results show a significant increase in the success rate compared to standard rule-based aggregation methods.