Do training courses really make any difference? How can we know this? Can the process of monitoring and evaluating training courses truly help to improve training effectiveness? Does pressure from donors help or hinder the process?
These are challenging but important questions for training providers, international and national NGOs and donors alike. Taking these questions seriously and discussing them collectively can help us deepen our understanding, share learning and improve our practices. With so much being invested in training courses – can we afford not to?
At the end of January, a group of training providers and INGOs met at an informal workshop led by the Training Providers Forum, of which INTRAC is a founding member. We discussed some of the key factors to consider when deciding whether to monitor and evaluate training; various tools and frameworks and their suitability in different situations, and key recommendations. A paper will be published on this shortly but in the meantime, here are five considerations:
1) The process of learning is profoundly personal. At the end of the day, whether a training course is carried out to strengthen a person, a team, an organisation or a sector, the learning happens at the level of the individual. Learning cannot be done to people – the most we can do is influence, support and encourage. This makes M&E challenging – particularly for ‘soft’ skills which are complex, and may not follow the logic of train X and they will be able to do Y.
2) The process is also relational. There are numerous stakeholders – the learners, their peers, managers, trainers, training managers, budget holders and donors. They affect and influence each other. Learning often happens, is reinforced and is applied through relationships outside of any course itself. In this way, learning is often more influenced by emotional and political factors than through logic. If we want to dig deeper during the evaluation process, then there are numerous stakeholders who will affect the planning, the design, the delivery and the follow up of any training course.
3) Learning is often a longer process of trial and error: d trying things out, making mistakes, and learning incrementally as we go is all part of the process. Feedback from others, encouragement andconfidence are often a large part of our success, as well as opportunities to practice. Thus the timing of any evaluation is critical. The longer the time period, the more we will be able to look at actual changes in behaviour and any effect this is having on the work, the team, the organisation. However, the further removed you are in time from the training course, the harder it may be to attribute any changes in behaviour to the training itself.
4) Another key factor in learning is also the degree to which people feel able to be honest: honest about what they need to learn, why they want to learn it, and whether they have learnt. Whether the evaluation is for accountability to donors or not will affect this, as will whether the future funding relies on learning being demonstrated. People may not feel about to say what further support they need, or where they still have gaps, for fear that this will affect their funding.
5) One of the most significant challenges with M&E is not just getting funding for it or doing it well, but actually making use of the results, particularly for learning and improving capacity building programmes. This will depend on who commissioned the evaluation and why, when the evaluation takes place and how robust the results are. It is critical to consider from the beginning what questions the evaluation is trying to answer and then ensure that data is collected and analysed in such a way that those questions can be answered.
There is one big question which we hope will always be considered in any evaluation which is: was a training programme the right capacity building intervention in the first place? And was it well planned and properly funded and supported? The degree to which we are prepared to ask these fundamental questions, depends on the degree to which they are prepared to know the answer!
Image credit: “A Training Provider’s Forum Poster” by Mango.
“Orange bloomy flowering tree near the Explorers Garden along Beech Path on Bussey Hill”, Arnold Arboretum, 18 May 2010. By Chris Devers. Licensed under Creative Commons Licensing.