These concluded that FAC’s ‘unique, researcher-led network’ model and its approach to analysing and understanding the political economy of agricultural policy processes provides a ‘quadruple win’ leading to positive impacts and outcomes through synergies between four key areas:
- High-quality research outputs;
- Effective policy engagement;
- Innovative communications and outreach; and
- Significant capacity strengthening (for both Consortium members and Early Career Fellows).
FAC prepared a post hoc (retrospective) ‘Theory of Change’ (TOC) for this evaluation, which was refined in discussions with DFID and the Upper Quartile review team. The Theory of Change was designed to be used in examining examples of where FAC appeared to have made a contribution to stronger ‘influence of evidence’, stronger ‘capacity to use political economy thinking’ and/or the ‘adoption of a policy or practice’, as explained in the TOC.
Impact and the network
Evidence from eight regional and country-level ‘impact case studies’ carried out by the Upper Quartile team using the Theory of Change showed significant use of FAC’s research evidence in African policy making, although it also revealed that the relationship between research-derived evidence and policy making is extremely complex. Nevertheless, some attribution was possible at the ‘influence of evidence’ and ‘capacity to use political economy thinking’ level. In addition, contributions from FAC could be identified at the ‘adoption of policy and practice’ level (the so-called ‘super-impact’ level of the TOC).
The impact case studies also revealed that limited investment in FAC’s distinctive researcher-led model increased value for money significantly through non-financial incentives by fostering a dynamic network built on long-term collaborations, and by paying careful attention to creating an open and responsive organisational culture.
This approach has provided the appropriate springboard for FAC to develop into a predominantly African-based network of researchers, coordinated through Regional Hubs in Kenya, Ghana, South Africa and its Secretariat in the UK.
It has also created good value for money by enabling productive research and capacity building relationships with over 130 individual researchers, Early Career Fellows and communications specialists, but without the significant transaction costs of developing formal partnership agreements with over 50 research and development organisations with which FAC is associated across Africa and elsewhere.
The evaluation concluded that this regionalisation process is still in its early days, but is moving in the right direction, with increasing levels of African ownership and decreasing reliance on DFID core support.
What could we do better?
Of course, the Upper Quartile evaluators did identify several important areas where there was still room for improvement. Chief among these were:
(1) developing better monitoring and evaluation systems, as FAC’s current ones were shown to be very good at the output level, but needed further improvement at outcome and impact levels;
(2) investing more in embedding gender and social inclusion perspectives and approaches in all of FAC’s thematic work; and
(3) narrowing the range of areas in which the Consortium is operating (currently 10 themes) to avoid spreading resources too thinly, while ensuring adequate reflection, adjustment and follow through.
The Upper Quartile team observed that core support that DFID had provided to FAC since 2005 was coming to an end, just as their evaluation was being completed in late 2014, which would create a major challenge for the Consortium. They pointed out that with the move towards increased multi-donor project funding, the management load will increase substantially in winning funds, managing an increasingly complex portfolio and reporting to multiple donors. Furthermore, project funding will have to contribute more towards the core costs, at least in the short term, and there may be a temporary overheads deficit that needs to be managed. The evaluators argued that judicious additional investment in these areas was needed, while remaining lean and networked, in order to maintain FAC’s comparative advantage, value for money and sustainability.
The evaluators also offered some specific recommendations to DFID in light of this review. They noted that, having invested in the creation of ‘a network with future value’, DFID should manage FAC’s exit from core funding in ways that minimise the risk of the loss of that value and maximise the potential future returns from the investment the agency has made in the Consortium. They pointed out that if this exit strategy from core support includes an opportunity for replacement with competitive funding, this should include a realistic assessment of the timescale involved and formal consideration of appropriate contingency plans. Finally, the evaluators pointed out that formal response by DFID to recommendations in external reviews (both past and present) of FAC would increase transparency of decision making and leave a record of emerging thinking for future learning. As of this writing, DFID has not responded formally to the Upper Quartile recommendations, but has stated that Future Agricultures would be welcome to bid on future funding calls as and when they arise.
What next?
Given this extremely encouraging external evaluation by Upper Quartile – and an equally positive internal ‘Programme Completion Report’ produced by DFID in late 2014 which gave the Consortium an ‘A+’ rating – Future Agricultures is now embarking on a new phase of its development. This will aim to secure programmatic funding from a number of donor partners to support an ambitious new strategy and business plan to allow the Consortium to build on the success of its ‘unique, researcher-led network’ model, while making further improvements in key areas identified by the evaluation team.
Among other things, this will include:
- Building on solid, long-term partnerships to grow our researcher-led network, as our institutional embedding efforts are only partially completed at both regional and national levels;
- Reducing the number of core themes, while continuing to produce high quality research and analysis on the politics of agricultural policy in key areas;
- Communicating evidence and facilitating access to knowledge and information by strengthening our capabilities to inform and influence policy processes at different levels;
- Increasing efforts to embed on gender and social inclusion insights and approaches in all of our themes; and
- Investing much more in critical social science capacity of our FAC network partners and young African researchers, as the demand for political economy expertise in the agricultural sector continues to grow across the region.
A final note of thanks
Independent evaluations of research organisations and networks can be challenging and complex affairs, as anyone who has ever been evaluated or served as an evaluator can testify. The most recent evaluation of Future Agricultures was particularly challenging and complex since DFID specified extremely ambitious terms of reference for the assessment. On behalf of the FAC Coordination Team and our network, I would like to express our sincere gratitude to the Upper Quartile team – Martin Whiteside (Team Leader), Kathleen Latimer (Evaluator and Upper Quartile Project Manager), Sally Baden (Senior Evaluator), Carl Jackson (Senior Evaluator) and Chris Boyd (Upper Quartile Project Director) – for rising to the challenge and for providing us with such detailed and rigorous assessment and many helpful recommendations.
Special thanks must also go to the UK Department of International Development and the many colleagues who have supported Future Agricultures over the years. We hope the investments they have made in the Consortium will be a lasting legacy to their vision and their commitment to supporting cutting-edge research to inform and improve decision-making on agricultural policy processes.
John Thompson
Future Agricultures EU Regional Coordinator