Informal feedback rather than performance measurements – user-centred evaluation in Scrum projects

The main features of the Scrum process are intense communication between different stakeholders, and rapid feedback based on regular delivery of working software. The integration of traditional user-centred evaluation activities in this context is challenging. Hence, this paper presents an interview study of 21 informants, categorised in four different professional roles. The main contribution of the paper is an overview of the types of user-centred evaluation conducted by information technology professionals in various Scrum projects. Results show that various forms of feedback are indeed gathered on the usability and user experience of the software, system or service being developed. However, the user-centred evaluations conducted typically are informal with few users, gathering empirical qualitative data and performed during short unplanned sessions. Performance measurements gathering quantitative data are seldom used. The informants in the business specialist role merely ask users about their opinion, whereas the other roles use several user-centred evaluation activities to gather feedback on their design. Generally, feedback is gathered throughout the whole project, but often evaluation is conducted early in the project or even before the actual development starts. Finally, these results are discussed in relation to previous studies in the area.