By Jeremy Astill-Brown
This piece was originally published in the August 2021 issue of the INTRAC newsletter. Subscribe today to receive new issues in your inbox.
I am part of an INTRAC team delivering an evaluation of a government donor grant to a faith-based development organisation. We proposed a “developmental evaluation” designed to help the organisation better understand their results and to maximise their effect. A developmental evaluation is primarily designed to support learning and management decision-making; and is particularly appropriate for projects or programmes working in complex or uncertain environments.
Two things about the process have been fascinating. One is how far outside the “normal” experience of evaluations our developmental approach has been for some, and the other is the role of politics. Politics conditions what people do, why they do it and how. Politics both sets up and removes the obstacles to progress that projects and programmes seek to tackle. To the extent that evaluations take politics into account, they usually focus on how programmes and projects have altered the incentives for good or bad behaviour. Both programme design and evaluations alike too often appear to assume that only the political landscape ahead of a programme counts. They rarely consider the view in the political rear-view mirror; the perceptions of the implementor (tending to assume that these are neutral); and almost never consider the political objectives of the donor.
This failure to think and act politically sets up a rather perverse incentive. Even in contexts widely acknowledged to be complex or challenging, programmes often see politics as something which gets in the way, rather than the bread and butter of everyday programmatic life. In pursuit of being apolitical, such programmes actually become sterile. All too often, programmes – and hence evaluations – do not explore how to shift the political dial to support positive change. Politics tends to be seen in the risk matrix rather than as an opportunity. Yet in seeking change in complex environments, decision makers and influencers are primary stakeholders. Strategies to engage them are just as important as strategies to represent the view and aspirations of the people and communities who are traditionally the focus of such programmes.
This raises questions about the nature of a developmental evaluation. Should it be a purely technical process, or should it represent a fusion of political and technical approaches? Is the purpose of such an evaluation to mark the implementing partners’ homework, or to help them better achieve their goals?
Without really intending this to be the case, we have found that our developmental evaluation has been – almost necessarily – “disruptive”. We have asked questions which have made people uncomfortable and questioned sacred cows across the board – often without even meaning to. We have been struck by how easily people can lose sight of why they do what they do. Seemingly straightforward questions about the nature of the problem, for whom it is an issue, why and what value the partnership offers to addressing it have left people feeling that we are criticising them, their choices and their actions. All of this is necessary, and – arrogantly – we think that donor and implementor alike will thank us in the end.
A developmental evaluation is not a licence to wreck and humiliate. It is an opportunity to be a critical friend and to help partners think through, understand and be able clearly to project their ambitions, choices and actions. The privilege of being allowed to conduct a developmental evaluation must be paid for with an approach which puts a human face on otherwise remote processes, and which builds a trusting relationship between evaluator and evaluated.
Jeremy Astill-Brown is part of INTRAC’s network of independent consultants. A former UK diplomat specialising in security and development issues, he is increasingly focussed on helping projects and programmes operating in politically complex and/or insecure environments generate and use evidence of the changes which they support in order to demonstrate their value.