Designing journal peer review: diverse expectations, procedures and concerns

Peer review plays a central role in the quality assurance of scholarly publishing. Scholarly journals use independent review as a key ingredient in the assessment and improvement of ‘quality’, leading to editorial decisions of what articles should be published and under which conditions. Although the role of peer review has been historically contested, it has become a widely shared practice among scholarly journals worldwide. However, review procedures are now more diverse than ever and continue to develop. Increasingly radical innovations are suggested to address flaws in the peer review system. In addition, peer review is now embedded in increasingly complex editorial assessment procedures, involving both technical and research integrity related concerns. In this chapter, we discuss a range of design options for peer review procedures, in light of contrasting expectations and concerns, the challenges of involving stakeholders such as authors and reviewers, and the specific mission and rationale of a scholarly journal. We give concrete examples of how the designs can be implemented, and pointers to past experience of what can go wrong. The chapter is organised around key questions of who counts as a ‘peer’, how to establish ‘independence’, what counts as ‘quality’, and how to resolve disagreement.