The second in a short series of blogs from M&E consultant Nigel Simister which tackle learning and accountability within organisational monitoring and evaluation systems.
My last blog suggested that pressures exerted on M&E systems by international structures and systems have tended to distort those systems away from learning and improving, and towards accountability upwards to donors and governments. If we are serious about changing this then the first question to ask is “how does a learning-based M&E system differ from one designed (at head offices at least) to promote accountability upwards?” Often people argue that the information required to learn is the same information required to be accountable to donors and supporters; to some extent this is true. However, INTRAC’s experiences in helping design, maintain and run M&E systems over the past two decades, particularly in larger and more complex organisations, suggest there are key features of learning-based M&E systems that are not always present in systems designed primarily for accountability purposes.
- A learning-based M&E system recognises that it needs to interact widely with different parts of an organisation. Learning may be generated or shared through M&E processes, but learning is a much wider discipline, and M&E plays only a small role in the individual and organisational learning that occurs within any organisation. It is important that an M&E system understands its role within wider learning processes, and both contributes to and feeds off wider learning. For example, an M&E system may generate tentative findings which later need to be explored through more in-depth research, or it might play a part in providing information and analyses that contribute to wider learning-based workshops.
- Organisations with learning-based M&E systems try to maximise the involvement of a variety of different stakeholders in the definition, collection, analysis and use of data. This not only helps triangulate information to ensure it is more robust (and therefore useful), but also helps ensure that analyses and lessons are directly relevant to the stakeholders, thereby maximising the potential for learning and improvement at many different levels.
- M&E systems devoted to learning typically attempt to work with realistic plans. They are clear about what they are trying to change, how and why, and how this fits with the work of others. They also set objectives that are grounded in real life, rather than working with unrealistic objectives set far too high in order to leverage funds, or too low in order to ensure simplistic targets are met. This allows organisations to compare sensible plans with what happens in reality in order to reach conclusions about the viability of future work.
- A learning-based M&E system pays as much attention to what did not change as to what did change, whereas an accountability-based system is rarely interested in what didn’t change (e.g. how many advocacy initiatives fizzled out without achieving any real change, or how many pilot projects were not replicated, and why). A learning-based M&E system recognises that as much can be learned from what did not work as from what did.
- Organisations with learning-based M&E systems often develop or adapt their own tools and methodologies in order to serve their own specific needs. They develop expertise in the kinds of tools or methodologies they need to answer their learning questions. Tools or methodologies are not implemented off the peg, nor are they adopted and used in inappropriate situations because of outside pressures, or because they are the flavour of the day.
- A learning-based M&E system pays a lot of attention to ensuring that information does not flow only in one direction, but that feedback and comment are institutionalised throughout the system to ensure that people at different levels of an organisation properly contribute to, reflect on, and use, new learning and analyses. By contrast, in systems designed mostly for accountability, information is more likely to gravitate to, and stick at, the centre.
- A learning-based M&E system actively promotes a culture of openness and honesty. The reporting of mistakes or failures is met with positive responses, and staff feel they are operating within an open and forgiving culture. This culture does not usually develop on its own, but is carefully developed and nurtured – constantly having to be protected against external and internal pressures to show good results.
Perhaps most importantly, organisations operating learning-based M&E systems know what they want to learn, and why. They define key learning questions to provide the information that they believe is necessary in order for them and others to improve. They do not simply develop multiple M&E systems and procedures in the hope that interesting learning will spontaneously arise. (Whilst it is perfectly possible for spontaneous learning to arise, this kind of learning has a tendency to generate repeat learning that has been generated by countless organisations before).
In essence, a genuine learning-based M&E system is clear about what it needs to learn, and how it might apply that learning to make life better for its constituents. An organisation with a learning-based M&E system sees learning and improvement as essential features that enable it to do its job better, rather than a luxury to be indulged depending on sufficient time and resources. Which brings us back to the original point of my previous blog – learning is not enough on its own; it also needs to feed into improved planning and operations, both internally and externally. We will never win the learning vs. accountability argument until we can change it into an improving vs. accountability argument.
My final blog will make some tentative suggestions for how we might begin to bring about a change in focus within M&E systems, or at least a rebalancing, away from simple accountability and towards improvement.