How Do Organizations Manage Independent Evaluations? Insights from a Cross-Agency Survey

In 2024, we opened a discussion on this platform titled "Management matters: exploring the link between management models and the use of evaluations". Many of you shared valuable reflections on the role of evaluation managers, the limits of their involvement, and the conditions under which evaluation findings are to be used. This blog returns to that conversation and shares the results of the study it inspired.

While evaluation norms and standards have become increasingly harmonized over recent decades, one critical dimension has escaped standardization: evaluation management. Who drafts the terms of reference? Who designs the methodology? Should evaluation managers join field missions? Who tracks whether recommendations get implemented? Our study examined these practices and explore how they relate to the use of evaluative evidence.

To answer these questions, we surveyed professionals managing independent evaluations across international development and research organizations and received 66 valid responses. Respondents included representatives from UN agencies (36%), government entities (27%), international research organisations (15%), donor and implementing organisations (8%), multilateral funds, and a development bank. We complemented the survey with a literature review on evaluation use in agricultural research for development, a mapping of 100 evaluation reports from nine peer organizations including FAO, IFAD, WFP, UNDP, UN Women, GEF, GCF, AfDB and the World Bank, and insights shared in the EvalforEarth discussion.

There is no single model of evaluation management. Organizations make different choices about design, fieldwork, hiring, AI use, and follow-up and those choices shape how evaluation evidence is ultimately used.

Who Designs the Evaluation? Mostly the Manager, but Not Everywhere

Overall, 77% of respondents reported leading the development of evaluation ToRs, and 70%% said the evaluation manager is primarily responsible for designing the approach, methodology, and key questions. But the picture varies sharply by organization type. Every respondent from UN agencies, donor organizations, and multilateral funds reported leading ToR development. That share drops to 67%% in implementing organizations and to below 40% in government entities, where the responsibility more often sits elsewhere.

Speed differs too. Donors and implementing organizations typically draft a ToR in five days or fewer, while UN agencies, governments, and multilateral funds generally report taking more than six days. Across the sample, 73% finish within 10 days.

Evaluability assessments remain the exception rather than the rule: only 35% of respondents conduct them consistently or most of the time, despite UNEG Standard 4.2 recommending them as a first step. 

Hiring: UN Agencies Prefer Individual Consultants; Others Choose Case by Case

UN agencies showed a clear preference for individual consultants, who are cheaper, faster to contract, and often better suited to project-level evaluations with modest budgets. Other organization types were more pragmatic, choosing between individuals and firms depending on the evaluation, the evaluand and the context, with firms valued for quality assurance, backstopping, and credibility in complex or large-scale evaluations.

Whatever the modality, hiring is a challenge. Although satisfaction with the teams eventually hired was positive across the board, 60% of respondents described finding and contracting the right team as somewhat or very difficult. Time constraints topped the list of challenges (29%), followed by the limited availability of subject matter experts, budget limits and lengthy bureaucratic processes at (21% each). The rarest commodity, respondents told us, is the consultant who combines sector expertise, evaluation knowledge, and strong analytical and writing skills.

Fieldwork: Where Involvement and Participation Become a Dilemma

Virtually everyone contributes to the design of data collection activities, with multilateral funds being the main exception: about half of respondents reported being rarely involved. Field participation splits the sample in two. Around half of respondents from UN agencies and international research organizations travel to the field, compared with about 30% of those from government entities. Respondents from donors, implementing organizations and multilateral funds rarely or never travel to the field. Even so, two thirds of respondents participate in interviews and focus groups, and roughly half take an active role by asking questions rather than simply observing.

This is where the EvalforEarth discussion added the most nuance. Survey respondents who participate in fieldwork highlighted benefits such as improved data quality, faster troubleshooting, credibility with stakeholders and ownership of findings. Yet contributors to the discussion, including Gebril Mahjoub Osman, cautioned against the bias and conflicts of interest that direct involvement can introduce, arguing instead for a facilitative and supportive role. Despite these differing perspectives, both sources converged on one point: roles and responsibilities should be clearly defined during the inception phase to prevent tensions from emerging later in the evaluation.

Artificial Intelligence: Use Is Spreading, Quality Control Is Not

Just over half of respondents reported using artificial intelligence in their evaluation work.. UN agency respondents were the most likely to use AI directly for specific tasks. Donors, governments and multilateral funds reported that AI was used primarily by external consultants, while international research organizations were the least likely to use it at all. When AI is used for note-taking and summarizing, only about half of respondents reported conducting a quality review of the outputs. Such checks were most common among UN agencies.

Publication Is the Norm, Use Is Not

About 80% of respondents said their reports are always or almost always published, with UN agencies and implementing organizations reporting the most consistent publication practices. Nearly 60% publish within three months of validation, with the shortest publication timelines reported by UN agencies and research organizations.

Use is another story. When asked how effectively their organizations use evaluative evidence for decision making, respondents gave an average score of just 3.3 out of 5. Donors were the most positive, with an average score of 3.8, followed by UN agencies (3.5). Research organizations and multilateral funds scored 3.0, while governments ranked lowest at 2.8. Respondents highlighted two recurring patterns. First, mid-term reviews are often seen as more useful than final evaluations because they provide an opportunity to act on findings while implementation is still underway. Second, producing too many evaluations can dilute attention and reduce the uptake of recommendations.

Management Response: Are Recommendations Being Tracked?

According to 76% of respondents, management responses are developed for all evaluations. The practice is standard practice in UN agencies, implementing organizations, and multilateral funds. It is reported by around 60% of governments and research organizations, but by only 25% of donor respondents.

Tracking systems show the same gradient. They exist in 94% of UN agencies, compared with 57% of research organizations, 50% of governments and multilateral funds, and 25% of donors. Publication of the management response follows a similar pattern: 90% of UN agency respondents reported publishing it systematically, compared with half of multilateral funds, around 30% of governments and research organizations, and only a small minority of donors.

One finding stands out: only 30% of respondents said their tracking system is publicly accessible, and every government respondent reported theirs is not. For a sector that promotes transparency and accountability, and in a time of growing emphasis on responsible research and assessment, keeping recommendation tracking behind closed doors is becoming increasingly hard to justify.

Ten Recommendations Across the Evaluation Cycle

Drawing the survey, the literature, the analysis of 100 evaluations and the contributions shared through EvalforEarth, we developed ten recommendations organized across the five phases of an independent evaluation. Check the study for the full list. 

Across all ten recommendations, a single logic holds: the use of evaluative evidence is primarily a management matter. Evaluations are more likely to influence decisions when they are designed with context and resources, supported by evaluability assessments, and guided by clearly defined roles that protect independence without sacrificing quality. Accessible reports, timely mid-term reviews, and open tracking systems further increase the likelihood that findings will be used rather than overlooked.

The full study, Mapping Evaluation Management Practices in International Research and Development Organizations (Toma, Jouini, and Maciel Pinto, 2025), is available on the IAES website. Link: https://cgspace.cgiar.org/server/api/core/bitstreams/513865ad-47fd-495c-8e0d-d5e9b98934d5/content