A pragmatic approach for identifying and managing design science research goals and evaluation criteria

The effectiveness of a Design Science Research (DSR) project is judged both by the fitness of the designed artifact as a solution in the application environment and by the level of new research contributions. An important and understudied challenge is how to translate DSR project research goals into discrete and measurable evaluation criteria for use in the DSR processes. This position paper proposes an inclusive approach for articulating DSR goals and then identifying project evaluation criteria for these goals. The goals are organized hierarchically as utilitarian goals, safety goals, interaction and communication goals, cognitive and aesthetic goals, innovation goals, and evolution goals. Goals in a DSR project are identified pragmatically by considering the components of the context coupled with the hierarchy of goals. Based on the identified goals, the associated evaluation criteria are determined and organized along the same hierarchy. These criteria measure the ability of the artifact to meet its goals in itscontext (immediate fitness). Moreover, our approach also supports the innovation and research contributions of the project. The apex of the goal hierarchy addresses the identification of criteria measuring the fitness for evolution of the designed artifact, to accommodate for changes in goals or context.