Amy and all- So I think we need to differentiate between what is evaluable in terms of the aims of the evaluation (timeframe for exposed typically excludes some projects because we have found that we really need to look 3 to 5 years after closure and if it has been more than seven years, then it’s hard to evaluate, and we’ve also found that project that we implemented longest had a greater likelihood of sustainability of results and emerging impacts), plus we need to look at the quality of evaluative data, (for instance, not just trainings given, but what were participants trained in? Was there any knowledge change or behavior change as a result at the final evaluation so we could evaluate changes to it expost). So an evaluability assessment depends on the aim of that evaluation which is different from how we evaluate. With ex-post evaluations, we want to make sure that partners and participants are still around to be asked (I did one 15 years later post closure in Uganda and the professors training teachers at the pedagogical University were there, but all the ministry staff had changed, and local teachers and students had long moved on).
Yes, I too mandate in our evaluations that we must have boots on the ground to evaluate with local stakeholders, partners at national and regional levels, with the villagers, with the local leaders, and all of that is participatory. Yes, we use focus group interviews, key informant interviews transect walks, lots of participatory tools as outlined in this toolkit we worked on (just out, please comment: 🔗 https://lnkd.in/gyfu_K_3 🌟 Share your comments and suggestions via af-terg-sec@adaptation-fund.org).
To the question of stakeholders creating the evaluation with us: we have asked local people if they would like us to evaluate their project to give them answers they’d want to learn… but repeatedly we have gotten no answers. They look at us rather confused because, I think, they’re just living their lives and they see the project as an input into their lives, one of many, whereas when we try to isolate the project and its results in itself is very strange to folks ;).
Hope this help!
Jindra Čekan/ová PhD Www.ValuingVoices.com and Jindracekan.com
RE: Evaluability Assessments: An invitation to reflect and discuss
Czechia
Jindra Cekan
Founder
Valuing Voices at Cekan Consulting LLC
Posted on 21/08/2024
Amy and all-
So I think we need to differentiate between what is evaluable in terms of the aims of the evaluation (timeframe for exposed typically excludes some projects because we have found that we really need to look 3 to 5 years after closure and if it has been more than seven years, then it’s hard to evaluate, and we’ve also found that project that we implemented longest had a greater likelihood of sustainability of results and emerging impacts), plus we need to look at the quality of evaluative data, (for instance, not just trainings given, but what were participants trained in? Was there any knowledge change or behavior change as a result at the final evaluation so we could evaluate changes to it expost).
So an evaluability assessment depends on the aim of that evaluation which is different from how we evaluate. With ex-post evaluations, we want to make sure that partners and participants are still around to be asked (I did one 15 years later post closure in Uganda and the professors training teachers at the pedagogical University were there, but all the ministry staff had changed, and local teachers and students had long moved on).
Yes, I too mandate in our evaluations that we must have boots on the ground to evaluate with local stakeholders, partners at national and regional levels, with the villagers, with the local leaders, and all of that is participatory. Yes, we use focus group interviews, key informant interviews transect walks, lots of participatory tools as outlined in this toolkit we worked on (just out, please comment: 🔗 https://lnkd.in/gyfu_K_3 🌟 Share your comments and suggestions via af-terg-sec@adaptation-fund.org).
To the question of stakeholders creating the evaluation with us: we have asked local people if they would like us to evaluate their project to give them answers they’d want to learn… but repeatedly we have gotten no answers. They look at us rather confused because, I think, they’re just living their lives and they see the project as an input into their lives, one of many, whereas when we try to isolate the project and its results in itself is very strange to folks ;).
Hope this help!
Jindra Čekan/ová PhD
Www.ValuingVoices.com and Jindracekan.com