Farmer Field Schools for Improving Farming Practices and Farmer Outcomes: A Systematic Review

This Campbell systematic review examines the effectiveness of farmer field schools in improving intermediate outcomes (such as knowledge and pesticide use) and final outcomes (such as agricultural yields, incomes and empowerment) in low‐ and middle‐income countries (LMICs), as well as implementation factors associated with programme success and failure. The review sythesises evidence from 92 impact evaluations, of which 15 were of sufficient quality for policy‐oriented findings, and 20 qualitative studies. Farmer field schools improve farmers' knowledge and adoption of beneficial practices, and reduce overuse of pesticides. This leads to positive outcomes for farmers: on average, a 13% increase in agricultural yields and a 20% increase in income. Farmer field schools also reduce pesticide use and environmental degradation. However, the evidence for these outcomes comes from short‐term evaluations of pilot programmes, and no studies with a low risk of bias are available. In programmes that were delivered at a national scale, studies conducted more than two years after implementation did not show any positive outcomes from the programme. For large‐scale programmes, recruiting and training appropriate facilitators was problematic. Authors' conclusions Farmer field schools (FFS) are a common approach used to transfer specialist knowledge, promote skills and empower farmers around the world. At least 10 million farmers in 90 countries have attended such schools. FFS are implemented by facilitators using participatory “discovery‐based” learning based on adult education principles. Many different implementing bodies have been involved. Field schools have a range of objectives, including tackling overuse of pesticides and other harmful practices, improving agricultural and environmental outcomes, and empowering disadvantaged farmers such as women. We conducted a systematic review of evidence on FFS implementation to investigate whether FFS make a difference, to which farmers, and why or why not. We synthesised quantitative evidence on intervention effects using statistical meta‐analysis, and qualitative evidence on the barriers and enablers of effectiveness using a theory of change framework. The results of statistical meta‐analysis provide evidence that FFS are beneficial in improving intermediate outcomes relating to knowledge learned and adoption of beneficial practices, as well as final outcomes relating to agricultural production and farmers' incomes. The findings suggest this to be the case for FFS promoting integrated pest management (IPM) technology, as well as other techniques. However, the rigorous impact evaluation evidence base is small and there are no studies that we were able to identify as having a low risk of bias. There is no evidence that neighbouring non‐participant farmers benefit from diffusion of IPM knowledge from FFS participants. Therefore, they do not experience improvements in IPM adoption and agriculture outcomes. The evidence of positive effects on agricultural outcomes is largely limited to short‐term evaluations of pilot programmes. In the few examples where FFS have been scaled up, the evidence does not suggest they have been effective in improving agricultural outcomes among participating farmers or neighbouring non‐participants. Although empowerment is a major objective of many FFS, very few studies have collected information on this outcome in a rigorous manner. A few studies suggest farmers feel greater self‐confidence. What explains the lack of scalable effects among FFS participants, or diffusion of IPM practices among the community? FFS differ from standard agricultural extension interventions, which tend to focus on disseminating knowledge of more simple practices such as application of fertiliser and pesticides, or adoption of improved seeds. The experiential nature of the training, and the need for the benefits of the FFS technology to be observed, are barriers to spontaneous diffusion. Furthermore, the effectiveness of scaled‐up interventions has been hampered by problems in recruiting and training appropriate facilitators at scale. The review provides implications for policy, practice and research. Executive Summary BACKGROUND After almost three decades of decline in public support, agriculture is now back on the development agenda. Since the late 1980s, support to agriculture has shifted from top‐down approaches to those identifying technologies and methods of communicating technologies which are suitable to support farmers' livelihoods in a sustainable manner, including participatory approaches based on the notion of creating spaces for farmer self‐learning. One such approach is the farmer field school (FFS), an adult education intervention which uses intensive “discovery‐based” learning methods with the objectives of providing skills in such areas as integrated pest management (IPM) and empowering farmers and communities. FFS have been implemented in 90 countries worldwide, reaching an estimated 10‐15 million farmers. Farmer field schools may appear to be the latest tool, but what does the evidence say regarding their effectiveness? OBJECTIVES This systematic review synthesises evidence on interventions identified as “farmer field schools” conducted in low‐ and middle‐income countries. The review aims to provide answers to the following research questions: Review question (1): a)What are the effects of farmer field schools on final outcomes such as yields, net revenues and farmer empowerment? b)What are the effects of farmer field schools on intermediate outcomes such as knowledge and adoption of improved practices (e.g. reduced use of pesticides)? c)What are the effects on outcomes for non‐participating neighbouring farmers living in the same communities as FFS farmers? Review question (2): What are the enablers of and barriers to FFS effectiveness, diffusion and sustainability? STUDY SELECTION CRITERIA Studies included in the review satisfied the following criteria. Eligible participants included farmers growing arable crops, living in low‐ or middle‐income countries at the time of the intervention. The review included those participating directly in the field school and also non‐participant neighbour farmers who may benefit through spillover effects or more formal dissemination methods. Eligible interventions were those identified as “farmer field schools,” regardless of the design or implementation, including FFS programmes providing training in IPM and other techniques. Studies combining FFS with other intervention components, such as input or marketing support, were also included. Comparisons eligible for the effectiveness review were farmers who received no intervention, or access to agricultural extension services from another source, including IPM (or equivalent) training. All outcomes reported were eligible for the review.primary Primary outcomes were agricultural outcomes, including yields and profits (net revenues). Secondary outcomes included other final outcomes such as environmental outcomes, health status and empowerment; and intermediate outcomes, including farmer knowledge and adoption of practices. Qualitative evidence on barriers to and enablers of effectiveness and sustainability were also included, including process and implementation information and measures of beneficiaries' attitudes and experiences with FFS. Eligible study designs for the effectiveness synthesis (review question 1) were measurable using counterfactual impact evaluations, including experimental or quasi‐experimental study designs and methods of analysis. Studies eligible for the synthesis of barriers and enablers (review question 2) were based on primary data collected from FFS participants, extension agents or experts, analysed using qualitative methods or descriptive statistics. The qualitative studies needed to report at least some information on the research question, procedures for collecting data, sampling and recruitment, and at least two sample characteristics. SEARCH STRATEGY The search included electronic academic databases, internet search engines, websites and theses, as well as handsearches of key journals and literature snowballing. Searches included general social science sources as well as agriculture subject‐specific sources of published and unpublished literature. All searches were updated in October 2012. The farmer field schools evaluation community has generated a large number of evaluations. We screened the titles and abstracts of over 28,000 papers, the majority of which were irrelevant to the topic. Four‐hundred‐sixty (460) relevant papers on FFS were assessed for inclusion based on full text. After the final screen by two authors, 134 quasi‐experimental studies comprising 92 distinct evaluations meeting the inclusion criteria were eligible for the review. The impact evaluations provide quantitative estimates of effects on outcomes for 71 FFS projects. However, only 15 of the impact evaluations meeting the inclusion criteria were judged to be of sufficient internal validity to make predictions for policy. The review also includes 20 qualitative evaluations meeting the inclusion criteria, which discuss the barriers to and enablers of change in 20 FFS projects. A portfolio review of 337 project documents was also conducted. DATA COLLECTION AND ANALYSIS Two independent reviewers assessed the full text papers against the inclusion criteria; discrepancies were resolved by consensus or by a third author if needed. Two reviewers extracted data from included studies. Quantitative impact evaluation studies were critically appraised according to the likely risk of bias according to threats to internal validity (causal identification), external validity (generalisability) and file‐drawer effects (publication bias). Qualitative evaluations were assessed according to adequacy of reporting, data collection, presentation, analysis and conclusions drawn. We used a hypothesised programme theory of change (White, 2009) as the framework for integrating the evidence. We collected data on programme design, implementation, targeting and contextual factors, and linked individual studies by programme in order to assess whether heterogeneous programme effects were correlated with study design, implementation and context. For the quantitative synthesis (review question 1), we extracted effect size estimates from included studies, calculating standard errors and 95 per cent confidence intervals using data provided in the studies, where possible. We used random effects meta‐analysis, estimating average effects of farmer field schools on the different outcomes, and examining heterogeneity. The results of the publication bias analysis suggested under‐reporting of small sample studies with negative or insignificant findings for studies reporting evidence on agricultural yields, which is evidence for possible publication bias. For the synthesis of qualitative evidence (review question 2), we used a thematic approach (Thomas & Harden, 2008), combining predetermined themes based on the links and assumptions in the theory of change model, as well as any other themes emerging from the detailed coding of the included studies. In the final stage of analysis, we used an iterated approach in which some effect moderators identified during the qualitative synthesis were tested in meta‐analysis and meta‐regression. RESULTS Review question (1) No studies with a low risk of bias were identified for the review of effects and only 15 (out of 92) quasi‐experimental studies were assessed as being of medium risk of bias and therefore policy‐actionable. The results of these medium‐risk‐of‐bias studies (reported in Summary of Findings Table 1) suggest farmer field schools impact positively on intermediate and final outcomes for participating farmers in the short to medium term. Findings for intermediate outcomes were as follows: There was a significant increase of 0.21 standard deviations on knowledge about beneficial practices among farmer field school participants over comparison farmers (SMD=0.21, 95% confidence interval (CI)=0.07, 0.35; Q=5, Tau‐sq=0.008, I‐sq=55%; evidence from 3 studies). There was a significant reduction in pesticide use by 23 per cent for IPM and IPPM FFS participants over comparison farmers (RR=0.77, 95% CI=0.61, 0.97; Q=40, Tau‐sq=0.07, I‐sq=83%; 8 studies). Effects on pesticide use were particularly large and consistent for cotton IPM projects in Asia. There was a significant increase in indices of adoption of other beneficial practices by 0.22 standard deviations over comparison farmers (SMD=0.22, 95% CI=0.06, 0.38; Q=10, Tau‐sq=0.02, I‐sq=80%; 3 studies). For final outcomes, the findings were as follows: A significant increase in agricultural yields was estimated among FFS participants, by 13 per cent over comparison farmers (RR=1.13, 95% CI=1.04, 1.22; Q=53, Tau‐sq=0.008, I‐sq=81%; 11 studies). A significant increase in profits (net revenues) was estimated, by 19 per cent among FFS participants over comparison farmers (RR=1.19, 95% CI=1.11, 1.27; Q=1, Tau‐sq=0, I‐sq=0%; 2 studies). The increase in profits was higher for FFS projects which also included complementary interventions involving input or marketing support (RR=2.51, 95% CI=1.51, 4.16, Q=1, Tau‐sq=0, I‐sq=0%; 2 studies). There was a 39 per cent reduction in estimated environmental impact quotient (EIQ) score as a result of reduced pesticide use among FFS farmers over comparison farmers (RR=0.61, 95% CI=0.48, 0.78; Q=3, Tau‐sq=0.01, I‐sq=33%; 3 studies). We could not identify any studies which provided valid estimates of impacts on farmer health outcomes. Very few studies assessed empowerment using quantitative counterfactual methods, and only one provided estimates of statistical precision. However, there is no evidence of effects on outcomes over the longer term (follow‐up surveys greater than two years after implementation) in programmes which have been scaled up nationwide. For IPM farmer field schools, there is no evidence that diffusion from FFS participants to non‐participating neighbour farmers usually happens: Overall, studies found no significant change in knowledge among FFS neighbours over comparison farmers. There was also no evidence for improvements among neighbours on pesticide use, yields or environmental impact quotient. When relatively better‐educated farmers are targeted to participate in the IPM field schools, diffusion may occur for simple practices (such as reduced pesticide use) and yields. However, even in a few cases where diffusion appeared to occur, the evidence does not suggest diffusion to non‐participants is sustained over time. Review question (2) Qualitative evaluations (reported in Summary of Findings Table 2) in the review helped us to understand the different types of farmer field schools implemented around the world, the reasons for heterogeneous impacts among FFS participants, and the limited diffusion to non‐participating neighbour farmers. FFS use discovery‐based learning methods which differ from agricultural extension interventions that tend to focus on disseminating knowledge of more simple practices, for instance application of fertiliser and pesticides, or adoption of improved seeds. However, there are several barriers to spontaneous diffusion of knowledge and practices. The FFS curriculum is complex and the training should be experience‐based, so that farmers are able to observe that FFS practices have a relative advantage over conventional farmer practices. Existing levels of social capital, the reach of social networks, and approaches to targeting FFS participants were found to be potentially important factors in influencing diffusion. More generally, the studies identify some of the more common problems in implementation, notably where a top‐down “transfer of technology” approach has been implemented for an intervention which is intended to be based on a “bottom‐up” participatory approach. All qualitative evaluations presented some evidence of use of triangulation to verify their findings, although most studies had weaknesses in reporting on sampling, analysis, and presentation of data, making quality appraisal of this evidence base challenging. IMPLICATIONS FOR POLICY AND PROGRAMMES Farmer field schools can have beneficial effects for participating farmers, in pilot programmes in the short term. The impacts on agricultural outcomes may be of substantial importance to farmers, in the region of a 10 per cent increase in yields and 20 per cent increase in profits (net revenues). The effects are particularly large when FFS are implemented alongside complementary upstream or downstream interventions (access to seeds and other inputs, assistance in marketing produce) for cash crops. However, the few studies of scaled‐up programmes measuring outcomes over the longer term (more than two years post‐training) do not find any evidence of effects of FFS. Farmers may also feel more confident, but again very few studies have assessed empowerment outcomes rigorously. There is little evidence of diffusion of improved practices or outcomes from FFS participants to non‐participating neighbour farmers. Field schools targeting more educated farmers may be better able to diffuse simple practices, such as on reduced pesticide use, than field schools that target less educated farmers. However, there is no evidence that any diffusion of practices is sustained over time, nor any evidence for adoption of more complex IPM practices via diffusion. As a method of rural adult education, FFS appear suited for gradual scale‐up provided there is a clear focus on ensuring local institutionalisation (i.e. favouring intensiveness of coverage in each community over geographical breadth of coverage). On the other hand, FFS seem unsuited to solve the problems of large‐scale extension. The approach may not be cost‐effective compared with agricultural extension in many contexts, except where existing farming practices are particularly damaging, for example due to overuse of pesticides. This is because of the highly intensive (and therefore relatively costly) nature of the training programme, the relative successes in targeting more educated farmers as compared with disadvantaged groups, and failures in promoting diffusion of IPM practices. Targeting FFS participants: Proponents of FFS have recommended targeting more highly educated farmers, those with greater land endowments, younger farmers and women, favouring those with relatively low opportunity costs of labour or farmers with relatively high pesticide costs. Problems were highlighted in targeting women who lived in household where they were not in a decision‐making position, and youth who were unable to dedicate sufficient time to the FFS plot or their fields. Where the aim is to include women and disadvantaged members of the community, implementers may need to tailor the intervention to enable their participation in the programme. The curriculum needs to be relevant and consistent with the needs and opportunities of women and the poor. Most obviously, in contexts where women are primarily responsible for growing subsistence crops, a curriculum that covers only commercial crops is unlikely to attract women participants. More generally, the curriculum and crops covered in FFS should also be adapted according to the local agricultural system and the needs of the farmers targeted by the programme. Curricula need to deal with the major challenges facing farmers. In most cases, these challenges will be multifaceted, highlighting the need to balance comprehensiveness with being able to cover all issues in sufficient depth to ensure appropriate learning. A cumulative approach over several seasons, including exchanges between field schools, may be preferable. FFS facilitators: The evidence also suggests that appropriate targeting and training of FFS facilitators is important. The theory of change suggests FFS should be delivered according to a participatory and discovery‐based approach to learning, including opportunities for farmers to experiment and observe new practices, particularly if farmers are to be empowered with lifelong skills capacity development. Attempts to target facilitators based on education or literacy levels may be less effective than targeting based on ability to communicate, and appropriate training which enables facilitators to use a bottom‐up approach. This is most obviously a barrier in scaled‐up programmes where FFS facilitators are recruited from extension staff who previously used more top‐down agricultural extension methods. Recruitment of facilitators should take into account personal attitude, maturity, literacy, leadership skills, knowledge in local language and experience with farming. In many contexts the gender of the facilitator should be carefully considered. Facilitators should have access to ongoing support and backstopping from supervisors and technical experts connected to local research centres. Regular monitoring of facilitators may help to identify schools where additional support is required. Complementary policies: Institutional actors involved in FFS should consider farmers' needs and interests in the design and implementation of the FFS programme. In some contexts stronger policies and regulatory measures may be necessary to counteract the activities of the pesticide industry, including the promotion and sale of pesticides by extension workers who are promoting FFS. New policies facilitating participatory agricultural extension approaches, replacing earlier extension policies aimed at promoting off‐the‐shelf technologies and input packages, may also be necessary. Local institutionalisation: Formal support and encouragement of FFS alumni, including technical assistance and backstopping, may be important for the sustainability of FFS practices and related activities. Given the skills‐based nature of the practices promoted in FFS, formal community‐building activities, support and successful attempts to institutionalise the approach, to encourage FFS graduates to train other farmers, are likely to be needed for any broader diffusion to non‐participating neighbour farmers, although the evidence base does not indicate that such attempts have been successful in the past. IMPLICATIONS FOR RESEARCH The majority of FFS impact evaluations (68 out of 92) use designs of questionable internal validity, and are therefore of limited value in determining whether farmer field schools have made a difference to outcomes. We were not able to locate any completed evaluations which used randomised assignment, an approach which is feasible for FFS. In three‐quarters of evaluations, no serious attempts were made to control for confounding through statistical matching or other statistical analysis, and in one‐third of cases statistical significance tests were not reported. The likely consequence, as indicated in the meta‐analysis, was the systematic overestimation of effects for all outcomes. The extent of resources that has been devoted to farmer field schools evaluations might therefore be usefully re‐allocated to conducting fewer but more rigorous impact evaluations, particularly those based on a solid counterfactual, with prospective cluster‐level assignment (randomised or otherwise) to allow measurement of community‐wide diffusion and to assess effects on agriculture and empowerment outcomes in the medium to longer term (three years or more). Evaluations should report information on both intervention design and implementation processes so that it is possible to assess whether programme causal chains break down because the intervention design is simply not appropriate for the context or because of poor implementation. Many qualitative evaluations need to report aspects of the research process in greater detail to allow users to assess their credibility and applicability. In particular, clear reporting on objectives, on methods of sampling, data collection and analysis should be provided. Greater use of structured abstracts will facilitate easier access to quantitative and especially qualitative research. Future studies should include data on views and experiences of FFS facilitators and agricultural extension workers. Summary of Findings Tables Summary of Findings Table 1: Effectiveness studies (review question 1) Outcomes Summary of findings No. of studies (participants) Relative effect size (95% CI) Percentage change compared with control group Quality assessment3 Statement Final outcomes ‐ all farmer field school participants (review question 1a) Yields (primary outcome) 11 (3,198) 1.13 RR1 (1.04, 1.22) 13% increase in yields of FFS participants on average relative to comparison group (4%, 22%) ++oo Low Moderate risk of bias and publication bias strongly suspected FFS may increase yields of FFS participants by an average of 13% relative to comparison group, though there is notable variation across populations and contexts Net revenues (primary outcome) 2 (488) 1.19 RR (1.11, 1.27) 19% increase in net revenue of FFS participants on average relative to comparison group (11%, 27%) ++oo Low Moderate risk of bias and small number of studies FFS may increase net revenues (profits) of FFS participants by an average of 19% relative to comparison group Empowerment 1 (200) 2.13 RR (1.46, 3.12) FFS participants 1.13 more likely to report positive empowerment outcomes relative to comparison group (0.46, 2.12) +ooo Very low Moderate risk of bias, serious indirectness and very serious imprecision The evidence on the impact of FFS on empowerment for FFS participants is inconclusive Environmental outcomes (environmental impact quotient) 3 (1,149) 0.61 RR (0.48, 0.77) 39% reduction in environmental impact quotient of FFS participants on average relative to comparison group (52%, 23%) ++oo Low Moderate risk of bias and small number of studies FFS may reduce the environmental impact quotient by 39% on average relative to comparison group Intermediate outcomes ‐ farmer field school participants (review question 1b) Knowledge test scores 3 (426) 0.21 SMD2 (0.07, 0.35) The knowledge test scores achieved by FFS participants are on average 0.21 standard deviations greater than in the comparison group (0.07, 0.35) ++oo Low Moderate risk of bias and small number of studies FFS may increase knowledge of FFS participants by 0.21 standard deviations on average relative to comparison group Pesticide use (IPM/IPPM FFS only) 9 (2,335) 0.83 RR(0.66, 1.04) 17% decrease in pesticide use by FFS participants on average relative to comparison group (‐34%, 4%) ++oo Low Moderate risk of bias and serious imprecision FFS may decrease pesticide use of IPM/IPPM FFS participants by 17% on average relative to comparison group though there is notable variation across populations and contexts Adoption of beneficial practices 3 (794) 0.22 SMD (0.06, 0.38) The number of practices adopted by FFS participants is on average 0.22 standard deviations greater than in the comparison group +ooo Very low Moderate risk of bias, serious inconsistency and small number of studies Evidence on the effect of FFS on the adoption of beneficial practices is inconclusive Diffusion to neighbour farmers (review question 1c) Pesticide demand neighbours (pesticide use, pesticide costs) 5 (1,115) 0.95 RR (0.64, 1.39) No statistically significant effect on pesticide use of FFS neighbours relative to comparison group ++oo Low Moderate risk of bias and serious imprecision FFS may not have any diffusion effect on pesticide use Yields 4 (986) 1.02 RR (0.97, 1.08) No statistically significant effect on the yields of FFS neighbours relative to comparison group ++oo Low Moderate risk of bias, serious inconsistency FFS may not have any diffusion effect on yields Notes: 1/ RR = response ratio. 2/ SMD = standardised mean difference. 3/ The rating guide used for the assessment of the quality of the evidence was adapted from GRADE and is available from the authors. Source: authors based on GRADE. Summary of Findings Table 2: Barriers to and enablers of effects (review question 2) Outcomes No. of studies Statement Barriers to and enablers of knowledge acquisition 17 studies Barriers: FFS facilitators do not receive sufficient training and ongoing support (6 studies). Facilitators do not have enough farming experience and/or appropriate characteristics (2 studies). Lack of adequate and timely resources for FFS schools (3 studies). Farmers excluded due to restrictive targeting criteria or procedures (4 studies). Farmers unable to participate due to gender, cultural norms or poverty (7 studies). High levels of drop‐out due to incorrect expectations or lack of interest, access or time (7 studies). Training delivered in a top‐down manner, using transfer of technology approach (4 studies). Curriculum not appropriate or relevant to the local context (7 studies). Imbalance in relationship between farmers and facilitators (3 studies). Facilitators use national language, in which farmers are not fluent, or too many foreign and scientific terms (2 studies). Enablers: FFS facilitators have experience with farming, are literate and mature, and have a positive personal attitude and leadership skills (3 studies). Gender of facilitator acceptable to participants and their families (2 studies). Farmers motivated to learn and improve livelihoods (5 studies). Training delivered in a participatory, bottom‐up manner (9). Curriculum appropriate and relevant to the local context (7 studies). Facilitators use local language and concepts and metaphors common to farmers (2 studies). Barriers to and enablers of adoption 18 studies Barriers: Training delivered in a top‐down manner, using transfer of technology approach (4 studies). Curriculum is not appropriate and relevant to the local context (7 studies). Farmers do not observe benefits from FFS practices (2 studies). Practices too complex for farmers to implement (3 studies). Farmers lack access to inputs, capital and/or markets (5 studies). Low levels of social capital among participants (1 study). Enablers: Training delivered in a participatory, bottom‐up manner (9 studies). Curriculum is appropriate and relevant to the local context (7 studies). Farmers observe benefits of FFS practices (5 studies). High levels of social capital among participants and tradition of collective action (3 studies). Barriers to and enablers of effectiveness and sustainability 14 studies Barriers: Diverging institutional incentives and objectives (3 studies). Conflicting agricultural policies (2 studies). Institutional legacy from top‐down extension approaches (4 studies). Power of pesticide industry and continued links with the extension service (2 studies). Lack of technical assistance and backstopping from researchers and extensionists (4 studies). Enablers: Active follow‐up and continued support from implementing agency (11 studies). FFS groups with consistent membership, good leadership, collective goals and a supportive group environment (4 studies). Barriers to and enablers of diffusion of knowledge and practices 11 studies Barriers: Complexity and experiential nature of FFS learning (5 studies). Farmers unable to observe FFS practices (2 studies). Farmers are not convinced of the relative advantage of FFS practices (2 studies). Socioeconomic differences between FFS participants and non‐participants (1 study). Low levels of social capital and cohesion limiting communication (2 studies). Enablers: Concrete and relatively easy practices (2 studies). Farmers observe FFS practices (5 studies). Farmers perceive FFS practices to have relative advantage over existing practices (2 studies). High levels of social capital and social networks extending beyond FFS group (3 studies). Active promotion of FFS practices post‐graduation (1 study). Source: authors.