PhD Researcher in Development Studies | Behavioral Scientist
Erasmus University Rotterdam
Posted on 28/04/2025
In my opinion, we risk missing the real impact if we overlook three elements in evaluating SSTC: the crucial role of grassroots actors, better ways to measure intangible outcomes, and smarter methods to address attribution.
I will share one case study on localization, and suggest two methodological approaches from interdisciplinary perspectives—behavioral science and econometrics—drawing from my field experience and research to illustrate my points.
1. Local actors often make the difference In a UNDP-supported microfinance project I studied, village leaders sought to bring back lessons from Bangladesh’s Grameen Bank model. At first, it didn’t work — the idea of microfinance didn’t translate well, and bad debts piled up. But thanks to the persistence of grassroots organizations and local leaders, they adapted the model to fit their own reality. Over time, it grew into a strong, lasting farmers’ association. If evaluations only look at short-term results, they might label this as a failure and miss the bigger story. Without recognizing the role of local actors and the longer adaptation process, evaluations risk overlooking such "delayed" successes. We need to give more space to local feedback and recognize the slow, sometimes messy, but powerful process of localization.
2. Measuring intangible outcomes through psychology & behavioral science Things like empowerment, ownership, and mutual learning are often seen as “too soft” to measure. But behavioral science and psychology have been studying these for decades. These disciplines offer a range of validated tools and frameworks that could strengthen evaluations in this area. However, adaptation is crucial: many existing measures are designed for WEIRD (Western, Educated, Industrialized, Rich, and Democratic) populations. Tailoring tools to fit local contexts would help ensure that evaluations meaningfully capture the behavioral and psychological dimensions of SSC initiatives.
3. Tackling attribution complexity with stronger causal and people-centered analysis Attribution remains one of the toughest challenges in evaluating SSTC, especially when multiple initiatives overlap or interact. However, evaluations can move beyond simply acknowledging this difficulty. Applying causal inference methods—such as Propensity Score Matching (PSM), natural experiments, and carefully structured comparison groups—can provide clearer evidence of an initiative’s specific contribution. Even when all beneficiaries ultimately receive the program, differences in timing (e.g., earlier versus later adopters) can offer natural comparison opportunities and help evaluators trace causal pathways over time.
Moreover, exploring non-traditional, people-centered statistical methods can further improve attribution analysis. For example, in analyzing data from my recent elderly care survey, I applied Latent Profile Analysis (LPA)—a technique that groups individuals into sub-profiles based on selected indicators such as psychological traits, willingness, and demographic characteristics. This revealed hidden diversity within the population and explained why treatment effects appeared inconsistent at the aggregate level. Applying such approaches in SSC evaluations can uncover latent differences among beneficiaries, helping evaluators better understand nuanced impacts across different sub-groups. Segmenting populations based on both timing and underlying profiles could produce more accurate and meaningful assessments of program effects.
While these methods require additional effort in design and analysis, they offer critical pathways to make SSC evaluations more credible, context-sensitive, and actionable.
In short: Recognizing the contributions of grassroots actors, adopting innovative measurement strategies for intangible outcomes, and applying more diverse causal analysis can make evaluations of SSTC initiatives more responsive, credible, and ultimately more impactful.
I would welcome any thoughts or experiences from others on how you have strengthened evaluations to better reflect local adaptation processes and intangible results in SSTC programs.
Thanks very much to Carlos Tarazona (FAO), Arwa Khalid (FAO), Javier Guarnizo (UNIDO), and Xin Xin Yang (UNICEF) for initiating this important disucssion!
RE: Maximizing the impact of South-South and Triangular Cooperation in a changing aid architecture through evaluation.
Netherlands
Zhiqi Xu
PhD Researcher in Development Studies | Behavioral Scientist
Erasmus University Rotterdam
Posted on 28/04/2025
In my opinion, we risk missing the real impact if we overlook three elements in evaluating SSTC: the crucial role of grassroots actors, better ways to measure intangible outcomes, and smarter methods to address attribution.
I will share one case study on localization, and suggest two methodological approaches from interdisciplinary perspectives—behavioral science and econometrics—drawing from my field experience and research to illustrate my points.
1. Local actors often make the difference
In a UNDP-supported microfinance project I studied, village leaders sought to bring back lessons from Bangladesh’s Grameen Bank model. At first, it didn’t work — the idea of microfinance didn’t translate well, and bad debts piled up. But thanks to the persistence of grassroots organizations and local leaders, they adapted the model to fit their own reality. Over time, it grew into a strong, lasting farmers’ association.
If evaluations only look at short-term results, they might label this as a failure and miss the bigger story. Without recognizing the role of local actors and the longer adaptation process, evaluations risk overlooking such "delayed" successes. We need to give more space to local feedback and recognize the slow, sometimes messy, but powerful process of localization.
2. Measuring intangible outcomes through psychology & behavioral science
Things like empowerment, ownership, and mutual learning are often seen as “too soft” to measure. But behavioral science and psychology have been studying these for decades. These disciplines offer a range of validated tools and frameworks that could strengthen evaluations in this area. However, adaptation is crucial: many existing measures are designed for WEIRD (Western, Educated, Industrialized, Rich, and Democratic) populations. Tailoring tools to fit local contexts would help ensure that evaluations meaningfully capture the behavioral and psychological dimensions of SSC initiatives.
3. Tackling attribution complexity with stronger causal and people-centered analysis
Attribution remains one of the toughest challenges in evaluating SSTC, especially when multiple initiatives overlap or interact. However, evaluations can move beyond simply acknowledging this difficulty. Applying causal inference methods—such as Propensity Score Matching (PSM), natural experiments, and carefully structured comparison groups—can provide clearer evidence of an initiative’s specific contribution. Even when all beneficiaries ultimately receive the program, differences in timing (e.g., earlier versus later adopters) can offer natural comparison opportunities and help evaluators trace causal pathways over time.
Moreover, exploring non-traditional, people-centered statistical methods can further improve attribution analysis. For example, in analyzing data from my recent elderly care survey, I applied Latent Profile Analysis (LPA)—a technique that groups individuals into sub-profiles based on selected indicators such as psychological traits, willingness, and demographic characteristics. This revealed hidden diversity within the population and explained why treatment effects appeared inconsistent at the aggregate level. Applying such approaches in SSC evaluations can uncover latent differences among beneficiaries, helping evaluators better understand nuanced impacts across different sub-groups. Segmenting populations based on both timing and underlying profiles could produce more accurate and meaningful assessments of program effects.
While these methods require additional effort in design and analysis, they offer critical pathways to make SSC evaluations more credible, context-sensitive, and actionable.
In short: Recognizing the contributions of grassroots actors, adopting innovative measurement strategies for intangible outcomes, and applying more diverse causal analysis can make evaluations of SSTC initiatives more responsive, credible, and ultimately more impactful.
I would welcome any thoughts or experiences from others on how you have strengthened evaluations to better reflect local adaptation processes and intangible results in SSTC programs.
Thanks very much to Carlos Tarazona (FAO), Arwa Khalid (FAO), Javier Guarnizo (UNIDO), and Xin Xin Yang (UNICEF) for initiating this important disucssion!
Cheers,
Zhiqi