Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research

Background: A systematic review of core outcome sets (COS) for research is updated annually to populate an online database. It is a resource intensive review to do annually but automation techniques have potential to aid the process. The production of guidance and standards in COS development means that there is now an expectation that COS are being developed and reported to a higher standard. This is the fifth update to the systematic review and will explore these issues. Methods: Searches were carried out to identify studies published or indexed in 2018. Automated screening methods were used to rank the citations in order of relevance. The cut-off for screening was set to the top 25% in ranked priority order, following development and validation of the algorithm. Studies were eligible for inclusion if they reported the development of a COS, regardless of any restrictions by age, health condition or setting. COS were assessed against each of the Core Outcome Set-STAndards for Development (COS-STAD). Results: Thirty studies describing the development of 44 COS were included in this update. Six COS (20%) were deemed to have met all 12 criteria representing the 11 minimum standards for COS development (range = 4 to 12 criteria, median = 10 criteria). All 30 COS studies met all four minimum standards for scope. Twenty-one (70%) COS met all three minimum standards for stakeholders. Twenty-three studies (77%) included patients with the condition or their representatives. The number of countries involved in the development of COS ranged from 1 to 39 (median = 10). Six studies (20%) met all four minimum standards [five criteria] for the consensus process. Conclusion: Automated ranking was successfully used to assist the screening process and reduce the workload of this systematic review update. With the provision of guidelines, COS are better reported and being developed to a higher standard.