By Nigel Simister
In my previous blog I said that adaptive management requires changes to the way in which monitoring and evaluation (M&E) should be conducted within complex programmes. The biggest change may need to be in the skill set of M&E staff, in particular encouraging more strategic thinking.
Adaptive management requires constant learning in order to provide information to decision-makers on how a programme is progressing and what is changing in the wider environment. In theory M&E should contribute to this learning, and there is a widespread belief that a good M&E system is vital for adaptive management.
I am not convinced that many M&E systems are currently capable of doing this, based on many years of implementing and advising on M&E systems at organisational level and within complex programmes. While many M&E systems can cope with ‘tactical’ adaptation, they are not so good at ‘strategic’ adaptation.
Tactical adaptation means making minor alterations to programmes, such as changing the hours of community meetings to better suit beneficiaries or responding to the concerns of key stakeholders. Tactical adaptation is (or should be) a routine part of all projects and programmes, whether complex or not. Indeed, NGO staff often spend a huge amount of time monitoring activities, deliverables, budgets, finances, procurements, contracts, compliance, relationships, logistics, equipment, personnel, risks and actions, as well as the external environment. This monitoring is routinely used for tactical adaptation, but often goes under the radar of academic M&E debates. With all this time spent on monitoring different aspects of a project or programme it is perhaps no wonder that NGO field staff have little time to engage in debates on whether learning or accountability is most important for M&E!
However, it is strategic adaptation that matters in complex programmes. Strategic adaptation involves constantly assessing whether or not a programme is doing the right thing and adjusting accordingly (O’Donnell 2016). Changing the strategic direction of a complex programme has more significant ramifications than making tactical adjustments.
There is no doubt that M&E should contribute to strategic decision-making in complex situations, but unfortunately in practice it is often unable to do so. It is either focused on the wrong thing, or is too unreliable, too confused, too little or too late. Or it focuses primarily on demonstrating accountability for pre-defined results or generating stories for communications or marketing purposes.
So, in practice, many complex programmes and organisations exist quite happily in the absence of an effective M&E system at strategic levels. Learning happens, but it happens through channels other than M&E.
What needs to change?
If M&E systems are to be adjusted to better suit the needs of strategic adaptive management, a number of changes are required, many of which are already happening to some extent. At the most basic level, these include:
- An increased focus on complexity-oriented processes, including methodologies (e.g. the most significant change technique, contribution analysis), different types of evaluation (e.g. developmental or real-time evaluations), and complexity-oriented planning tools (e.g. scenario planning or outcome mapping).
- More focus on sense-making, which involves encouraging different stakeholders from within complex programmes to come together to jointly process existing information generated through both formal and informal processes.
- Ongoing assessments of the balance between the time taken to generate evidence and the reliability of findings. Adaptive management often means taking decisions based on the best possible available evidence – either using existing evidence or engaging in rapid data collection and analysis – rather than waiting longer to produce findings that are more robust or more reliable.
- An emphasis on ongoing planning and design. There is a long history of NGOs developing and using innovative design and planning processes, such as political economy analysis, context analysis, scenario planning, participatory appraisal, power or gender analysis, etc. (see O’Donnell 2016). But regular adaptation means these processes should be part of ongoing M&E and learning, rather than separate planning exercises carried out before a programme starts.
- Addressing practical challenges (see Bowman 2016), including: cultivating the analytical skills of programme staff; enhancing procedures for collecting, storing and using information to bring ongoing M&E more in line with existing ethical standards for evaluation and research; and the pursuit of information and communication technologies (ICTs) that can shorten the time loops between information collection, analysis and use.
More integration, new attitudes and different skills
The most pressing need is arguably to ensure that the different information disciplines of design, planning, monitoring, evaluation, impact assessment, research and learning are all fully integrated. The development community has always recognised that planning and M&E (PME) are mutually inter-dependent, and there is increasing recognition of the importance of integrating M&E and learning (MEL). Within complex programmes, research – traditionally conducted before a programme begins as part of a situational analysis, but often ignored during implementation – should also be part of the ongoing system. If the NGO community needs a new acronym then I would suggest PEARLS – a Planning, Evaluation, Adaptation, Research and Learning System.
NGOs will have to think differently about how to allocate resources to monitoring, evaluation, learning and research, bringing in more flexibility and more fluidity. This does not necessarily mean spending more money overall (although that would be a help), but spending it differently, responding to needs at the time rather than being planned in advance.
In turn, this may require significant changes in attitudes. M&E for adaptive management requires different kinds of personalities and competencies than traditional M&E. Currently, many M&E staff are trained to work in a style that emphasises reporting against pre-defined, quantitative indicators, using linear tools such as the logical framework. But under adaptive management, the abilities to think strategically, identify emerging patterns, build relationships with stakeholders, communicate with different groups, and persuade others are likely to be more important than knowledge and experience of traditional M&E methods. Many years’ experience in traditional M&E may in fact be a hindrance rather than a help.
In the final blog I will discuss some wider implications for M&E within adaptive management.
References
O’Donnell, M. (2016). Adaptive management: What it means for CSOs. BOND, London, UK.
Bowman, K (2016). Evaluation for strategic learning and adaptive management in practice. Oxfam GB, November 18th 2016.