Evaluating competencies with spoken comments and machine learning

Employees are frequently evaluated on a numerical scale by their supervisors. These numerical assessments inform far-reaching managerial decisions, such as promotions, training opportunities, and dismissals. Yet, they often lack accuracy, are subject to supervisor bias, and do not provide justification for the ratings. In this paper, we address the limitations of numerical ratings by letting individuals provide spoken assessments of others and use a Random Forest algorithm to convert the spoken assessments into numbers (algorithmic ratings). Through this method, we combine the advantages of qualitative feedback and numerical ratings while potentially mitigating common biases. Our results suggest that the algorithmic ratings more accurately reflect the distribution of competencies (as measured by psychometric tests) than assigned numerical ratings (assigned ratings). The algorithmic ratings are considerably more nuanced and less skewed compared to the assigned ratings. Our findings highlight the potential of combining spoken comments with a machine learning model to enhance the accuracy of employee assessments in organizational settings.