Community monitoring interventions to curb corruption and increase access and quality of service delivery in low‐ and middle‐income countries: a systematic review

This Campbell systematic review assesses the effectiveness of community monitoring interventions in reducing corruption. The review summarises findings from 15 studies, of which seven are from Asia, six from Africa and two from Latin America. Community monitoring interventions can reduce corruption. They also improve use of health services, but no significant effect is found on school enrolments or dropouts. There is no improvement in health service waiting times, but there is an improvement in weight for age, though not child mortality. There are beneficial effects on education outcomes as measured by test scores. Community monitoring interventions appear to be more effective in improving outcomes when they promote direct contact between citizens and providers or politicians, and when they include tools for citizens to monitor the performance of providers and politicians. In all cases, findings are based on a small number of studies. There is heterogeneity in the findings with respect to health and education. Hence it is difficult to provide any strong, overall conclusions about intervention effectiveness. PLAIN LANGUAGE SUMMARY COMMUNITY MONITORING INTERVENTIONS CAN REDUCE CORRUPTION AND MAY IMPROVE SERVICES Community monitoring interventions (CMIs) can reduce corruption. In some cases, but not all, there are positive effects on health and education outcomes. Further research is needed to understand contexts and designs for effective interventions. WHAT DID THE REVIEW STUDY? Corruption and inefficient allocation of resources in service delivery are widespread in low‐ and middle‐income countries. Community monitoring interventions (CMIs) are intended to address this problem. The community is given the opportunity to participate in monitoring service delivery: observing and assessing providers' performance to provide feedback to providers and politicians. This review assesses the evidence on the effects of community monitoring interventions on corruption and access and quality of service delivery outcomes. The review also considers the mechanism through which CMIs effect a change in corruption and service delivery outcomes, and possible moderating factors such as geographic region, income level or length of exposure to interventions. WHAT STUDIES ARE INCLUDED? To assess the effect on corruption included studies had to have either an experimental or a quasi‐experimental design. Qualitative studies were included to assess mechanisms and moderators. The review assesses 15 studies of 23 different programmes' intervention effects. The studies were conducted in Africa (6), Asia (7) and Latin America (2). Most studies focused on programmes in the education sector (9), followed by health (3), infrastructure (2) and employment promotion (1). What is the aim of this review? This Campbell systematic review assesses the effectiveness of community monitoring interventions in reducing corruption. The review summarises findings from 15 studies, of which seven are from Asia, six from Africa and two from Latin America. WHAT ARE THE MAIN RESULTS OF THIS REVIEW? Community monitoring interventions can reduce corruption. They also improve use of health services, but no significant effect is found on school enrolments or dropouts. There is no improvement in health service waiting times, but there is an improvement in weight for age, though not child mortality. There are beneficial effects on education outcomes as measured by test scores. Community monitoring interventions appear to be more effective in improving outcomes when they promote direct contact between citizens and providers or politicians, and when they include tools for citizens to monitor the performance of providers and politicians. In all cases, findings are based on a small number of studies. There is heterogeneity in the findings with respect to health and education. Hence it is difficult to provide any strong, overall conclusions about intervention effectiveness.. WHAT DO THE FINDINGS OF THIS REVIEW MEAN? The evidence identifies CMIs as promising. That is, there is evidence that they are effective. But the evidence base is thin, the interventions do no work in all contexts, and some approaches appear more promising than others. Future studies should assess the effectiveness of different types of community monitoring interventions in different contexts, sectors and time frames to identify when and how such programmes may be most effective in improving outcomes. There is a need for adequate information and tools to assist citizens in the process of monitoring. Research about these mechanisms and their moderation of the effectiveness of CMIs should be a priority for further research in the area. HOW UP‐TO‐DATE IS THIS REVIEW? The review authors searched for studies published until November 2013. This Campbell systematic review was published in November 2016. Executive summary/Abstract 1.1 Background In many low‐ and middle‐income countries (L&MICs) corruption and mismanagement of resources are prevalent in the public sector. Community monitoring interventions (CMIs) aim to address such issues and have become common in recent years. Such programmes seek to involve communities in the monitoring of public service providers to increase their accountability to users. However, their effectiveness in reducing corruption and improving access and quality of services remain unclear. 1.2 Objectives This review aims to assess and synthesise the evidence on the effects of CMI interventions on access to and quality of service delivery and corruption outcomes in L&MICs. More specifically, the review aims to answer three main questions: 1) What are the effects of CMIs on access to and quality of service delivery and corruption outcome measures in L&MICs relative to no formal community monitoring or CMIs with less community representation? 2) What are the mechanisms through which CMIs effect a change in service delivery and corruption outcomes? 3) Do factors such as geographic region, income level or length of exposure to interventions moderate final or intermediate outcomes? 1.3 Search Methods We searched for relevant studies across a broad range of online databases, websites and knowledge repositories, which allowed the identification of both peer reviewed and grey literature. Keywords for searching were translated into Spanish, French, and Portuguese and relevant non‐English language literature was included. We also conducted reference snowballing and contacted experts and practitioners to identify additional studies. We used Endnote software to manage citations, abstracts, and documents. First stage results were screened against the inclusion criteria by two independent reviewers, with additional supervision by a third. 1.4 Selection Criteria We included studies of CMI in countries that were classified as L&MICs according to the World Bank definition at the time the intervention being studied was carried out. We included quantitative studies with either experimental or quasi‐experimental design to address question 1. In addition, both quantitative and qualitative studies were eligible for inclusion to address questions 2 and 3. 1.5 Data Collection and Analysis Two reviewers independently coded and extracted data on study details, design and relevant results from the included studies. Studies were critically appraised for potential bias using a predefined set of criteria. To prepare the data for meta‐analysis we calculated standardised mean differences and 95 per cent confidence intervals (CI) for continuous outcome variables and risk ratios and risk differences and 95% CI for dichotomous outcome variables. We then synthesised results using statistical meta‐analysis. Where possible we also extracted data on intermediate outcomes such as citizen participation and public officials and service providers' responsiveness. 1.6 Results Our search strategy returned 109,017 references. Of these 36,955 were eliminated as duplicates and a further 71,283 were excluded at the title screening stage. The remaining 787 papers were included for abstract screening and 181 studies were included for full text screening. Fifteen studies met the inclusion criteria for addressing question 1. Of these, ten used randomised assignment and five used quasi‐experimental methodologies. An additional six sibling papers were also included to address questions 2 and 3. Included studies were conducted in Africa (6), Asia (7) and Latin America (2). The 15 studies included for quantitative analysis evaluated the effects of 23 different CMIs in the areas of Information Campaigns (10), Scorecards (3), Social Audits (5), and combined Information campaigns and Scorecards (2). Most studies focused on interventions in the education sector (9), followed by health (3), infrastructure (2) and employment promotion (1). Corruption outcomes Included studies on the effects of CMI on corruption outcomes were implemented in infrastructure, education and employment assistance programmes. The overall effect of CMI as measured by forensic economic estimates in two studies suggest a reduction in corruption (SMD=0.15, 95% CI [0.01, 0.29). Three studies (comprising four interventions) measured perception of corruption as an outcome measure. A meta‐analysis of two of these studies showed evidence for a reduction in the perception of corruption among the intervention group (risk difference (RD) 0.08, 95% CI [0.02, 0.13]). Another study, which was not included in the meta‐analysis due to a lack of comparability in outcome, suggests an increase in perceptions of corruption in the intervention group (SMD ‐0.23, 95% CI [‐0.38, ‐0.07]). Access to services A number of different outcome measures were included as proxies for access to service delivery. One study examined the effects of an information campaign and a combined information and scorecard campaign on health care utilisation. The information campaign showed no significant effect in the short term, but the information campaign and score card combined resulted in an increase in utilisation both in the short term (SMD 2.13, 95% CI [0.79, 3.47]) and the medium term (SMD 0.34, 95% CI [0.12, 0.55]). The overall effects of two CMI interventions on immunisation outcomes suggest a positive effect in the short term (Risk Ratio (RR): 1.56, 95% CI [1.39, 1.73]). However, the medium term effect reported from one of these interventions is smaller and less precise (RR 1.04, 95% CI [‐0.52, 2.61]). Another study reporting on a range of measures of access to health services suggests an overall positive effect (RR 1.43, 95% CI [1.29, 1.58]). Meta‐analysis of four studies which evaluated the effects of CMI on school enrolment showed an overall positive effect, but the estimate cross the line of no effect (SMD 0.09, 95% CI [‐0.03, 0.21]). The overall effect across on drop‐out across four studies is no different from zero (SMD 0.0, 95% CI [‐0.10, 0.10]). Quality of services For health related interventions child death and anthropometric outcomes were considered proxies for quality of service. A meta‐analysis of two studies which examined the short term effects of a score card and a combined score card and information campaign using child deaths as an outcome is not clear (RR 0.76 [0.42, 1.11]). For the score card and information campaign intervention data was available on the medium term effects and the estimate is similarly imprecise (RR 0.79, 95% CI [0.57, 1.08]). The average effect on weight for age, based on the same two studies, suggests an overall beneficial effect (RR 1.20, 95% CI [1.02, 1.38]). For the combined score card and information campaign intervention with data on medium term effects the results suggest the benefits were sustained (RR 1.29, 95% CI [1.01, 1.64]). The same two studies also looked at waiting times for services and the results suggest no difference in this outcome (RR 0.99, 95% CI [.80, 1.17]). In education interventions test scores were used as a proxy outcome measure for quality of service. The overall effect across six studies was 0.16 (SMD, 95% CI [0.04, 0.29]). The limited number of studies included in our review, and the limited number of included studies with information on intermediate outcomes in particular limited our ability to answer our second and third research questions regarding the mechanisms through which CMIs effect change and whether contextual factors such as geographic region, income level or length of exposure to interventions moderate final or intermediate outcomes. Nonetheless, some exploratory evidence is provided in response to these questions, which may inform further research in the area. Some likely important moderators of the effect of CMI are having an accountability mechanism for ensuring citizen participation, availability of information and tools for citizens engaged in the monitoring process and pre‐existing beliefs regarding the responsiveness of providers to citizen's needs 1.7 Authors' conclusions This review identified and analysed available evidence regarding the effects of CMIs on both access to and quality of service delivery and on corruption outcome measures in L&MICs. Overall, our findings were heterogeneous making it difficult to provide any strong, overall conclusions as to the effectiveness of CMIs. However, the results suggest CMIs may have a positive effect on corruption measures and some service delivery measures. We found the overall effect of CMIs on both forensic and perception based measures of corruption to be positive. In improving access to public sector services results were more variable. Effects on utilization of health services are not clear, but we observe an improvement in immunization rates. In the education sector, we did not find evidence of an effect on proxy access measures such as school enrollment and dropout. We used child anthropometric measurements and deaths and waiting times for services as proxy measures for service quality in the health sector and test scores in the education sector. The evidence from two studies suggests improvements in weight for height, but no difference in child deaths or in waiting times for services. The results suggest an improvement of quality of services, as measured by improvements in test scores. Despite limitations in our ability to synthesise evidence on the mechanisms which moderate the effects of CMIs, some important preliminary evidence was uncovered. Firstly, we identified a lack of accountability in ensuring the involvement of citizens in CMIs as an important potential bottleneck to effectiveness. Secondly, we identified the need for adequate information and tools to assist citizens in the process of monitoring. Further research on these mechanisms and their moderating effect on the effectiveness of CMIs should be a priority for further research in the area.