Skip to main content
Serdar Bayryyev

Italy

Serdar Bayryyev Member since 25/10/2018

FAO

Senior Evaluation Officer

My contributions

  • Discussion

    Maximizing the impact of South-South and Triangular Cooperation in a changing aid architecture through evaluation.

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 01/05/2025

      Thank you for sharing this comprehensive overview of the challenges and opportunities in SSTC evaluation. South-South Cooperation (SSC) embodies a dynamic, ongoing collaborative partnership among diverse stakeholders working towards shared goals, especially within the framework of the Sustainable Development Goals (SDGs). The United Nations Department of Economic and Social Affairs (UNDESA) describes multi-stakeholder partnerships for the SDGs as “An ongoing collaborative relationship among organisations from different stakeholder types aligning their interests around a common vision, combining their complementary resources and competencies and sharing risk, to maximise value creation towards the Sustainable Development Goals and deliver benefit to each of the partners.” 

      My recent evaluations of partnerships between FAO, civil society organizations, and the private sector have also provided valuable insights highlighting the critical need for truly transformative partnerships and collaborative efforts that foster innovation, mutual benefit, and shared ownership among all stakeholders and beneficiaries. Strengthening such partnerships is essential to advancing sustainable development and achieving the SDGs effectively. (These evaluation reports can be accessed here: https://openknowledge.fao.org/handle/20.500.14283/cb1636en; https://openknowledge.fao.org/handle/20.500.14283/ca6678en)

      Building on my experience with these impactful evaluations, I believe that one of the most promising strategies to enhance the evaluation of South-South and Triangular Cooperation lies in embracing multi-stakeholder partnerships and participatory engagement and collaboration that aligns with the core principles of the 2030 agenda, including mutual benefits, solidarity and local ownership.

      My perspective is to view evaluation not just as a measurement tool but as a catalyst for reinforcing partnership dynamics. This means involving all relevant stakeholders actively in co-designing evaluation frameworks and criteria, which can help ensure that assessments reflect locally prioritized outcomes and contextual realities. When national partners lead or co-lead evaluations, it fosters ownership, enhances relevance, and builds local evaluation capacity—addressing some of the capacity gaps and data challenges identified.

      Recognizing SSTC as a form of transformative partnership, evaluations should assess not only immediate results but also the evolution of trust, solidarity, and mutual respect over time. Such an approach underscores the importance of long-term relationship-building as an integral part of sustainable development impact. By embedding participatory, narrative, and contextually grounded evaluation practices, and fostering nationally-led capacities, we can better capture the true value of SSTC initiatives. This, in turn, will strengthen accountability, inform more effective partnership strategies, and ultimately contribute to the SDGs in a manner that is locally owned and globally impactful.

      Best regards,

      Serdar Bayryyev 

      Senior Evaluation Officer at the Food and Agriculture Organization (FAO).

      Read on a separate page
  • Discussion

    Do Big Projects Deliver Effective Solutions in a Complex World?

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 07/04/2025

      The effectiveness of large-scale projects in delivering value for money, particularly concerning food security, agriculture, and rural development, presents a multifaceted challenge. The definition of what constitutes a "large" project can vary significantly based on context and circumstances. For example, in some regions, projects exceeding USD 5 million may be considered large, while in more challenging environments, investments below USD 10 million could be seen as inadequate for driving impactful change. Through my evaluation work, I have encountered a variety of successes and challenges that might be relevant for this discussion.


      1. Outreach Opportunities
      The scale of a project significantly influences its ability to reach beneficiaries. Large projects often mobilize substantial resources from both domestic and international sources, enabling comprehensive interventions at scale that can lead to meaningful transformations within rural economies and food systems. 


      2. Ability to deliver multiple benefits

      Large-scale projects have the capacity to generate multiple advantages, as these interventions often aim to tackle various development challenges simultaneously. One example is the Programme for Improvement of Irrigation Systems in Afghanistan, implemented by the FAO. This initiative aimed to modernize the country's irrigation infrastructure, where over 70 percent was in disrepair, necessitating significant rehabilitation. The evaluation of this programme found that this intervention not only enhanced agricultural productivity and bolstered food security but also fostered local capacity building and promoted community participation in sustainable water management practices. The diverse benefits derived from the programme included:  
      - Increased Agricultural Production and Productivity  
      - Opportunities for Livelihood Diversification  
      - Enhanced Food Security  
      - More Equitable Water Distribution  
      - Capacity Building for Farmers and Local Institutions  
      - Community Engagement and Ownership  
      - Environmental Benefits  

      3. Long-Term Commitments
      Large projects typically necessitate longer-term commitments, both financially and technically. Adequate resources and sustained technical support are crucial in converting initial investments into long-standing impacts. For instance, a partnership between a government agency and an international NGO to develop a sustainable farming initiative might span several years, allowing stakeholders to build trust and refine practices collaboratively. This time investment is essential for fostering deeper collaborations among development actors.


      4. Innovation and Technology Transfer
      Large-scale initiatives can effectively facilitate the introduction and dissemination of innovative agricultural practices and technologies, with adequate time and resources allocated for piloting and scaling up. For example, a large project could introduce precision agriculture techniques through workshops and demonstrations, empowering local farmers with new methods and tools that enhance productivity and reduce environmental impact. This is particularly vital in regions facing resource constraints, where specialized technical assistance can significantly uplift agricultural practices.

      5. Infrastructure Development
      Many successful agricultural and rural development projects have focused on the construction of essential infrastructure, such as roads, irrigation systems, and storage facilities. These infrastructures serve as foundational elements for enhancing market access and bolstering food security. For instance, a project that constructs new rural roads can significantly improve farmers' access to markets, enabling them to sell produce at better prices and reducing post-harvest losses.

      6. Inefficiencies and Bureaucracy
      However, one of the significant challenges that large projects face includes inefficiencies stemming from bureaucratic processes. Stringent preparatory requirements and increased scrutiny for large procurement contracts can result in delays, cost overruns, and a misalignment between project objectives and local needs. For instance, excessive bureaucracy in a large-scale agricultural initiative may hinder the procurement of necessary inputs, causing delays in implementation and increasing frustration among local stakeholders.

      7. Alignment with Local Needs and Ownership
      Another frequent challenge for large projects is the difficulty in fostering genuine local ownership. Decision-making often occurs at higher institutional levels, leading to inadequate engagement of local stakeholders in the planning and execution phases. This disconnect can result in projects implementing solutions that are misaligned with community contexts or priorities. For example, a large-scale irrigation project designed without local consultation might focus on cash crops that do not align with local subsistence needs, jeopardizing the project's sustainability once funding ends.

      8. Measuring Impact
      Evaluating long-term impacts presents its own set of challenges. Immediate outputs, such as the distribution of agricultural tools or construction of various infrastructures and facilities, are relatively easy to measure. However, assessing sustained changes in food security or rural livelihoods often requires extended timelines and a nuanced array of indicators. Implementing robust impact evaluations that include building baseline assessments and monitoring frameworks can enhance the assessment of impact. For instance, a project aimed at improving food security would benefit from multi-year evaluations to capture changes in dietary practices, agricultural resilience, and income stability.

      Although large-scale projects have the capacity to significantly enhance food security, agriculture, and rural development, they frequently face major challenges. Careful assessment of their design, execution strategies, and involvement with the community is crucial to determine if they provide value for money and produce lasting benefits. The effectiveness of these initiatives ultimately relies on a dedication to learning, flexibility, and genuine partnership with local communities. Ongoing monitoring and adaptive project management, bolstered by quality evaluative work, can help optimize the positive effects of large-scale projects and ensure they address the needs of their intended beneficiaries.
       

      Read on a separate page
  • Discussion

    Evaluating Agri-Food Systems Transformation Policies and Regulations

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 05/02/2025

      Dear Mireille, 

      Thank you for raising this important topic and calling for a further discussion in this forum. To facilitate this discussion, I would like to share some reflections on the two questions that you have posted.

      Question 1. What challenges are encountered in evaluating policies and regulations, such as regulatory frameworks like the EUDR, particularly in the context of cocoa and other food products under these new regulations?

      Evaluating policies and regulations, such as the European Deforestation Regulation (EUDR), presents several challenges, particularly in the context of cocoa and other food products. 

      The complexity of supply chains and their globalized nature makes it difficult to trace the origins of products and determine compliance with regulations. This complexity complicates evaluation analysis, particularly for products like cocoa that may be sourced from multiple regions with varying levels of compliance with regulations and governance arrangements.

      Another complexity relates to the engagement of a very diverse groups of stakeholders, (such as smallholder farmers, businesses, consumers, etc.) who have varying interests and capabilities. Engaging all relevant parties effectively in the evaluation process can be challenging and may affect the quality of data collected and the applicability of findings.

      Lack of comprehensive and accessible data can also hinder effective evaluation of policies, as  reliable data may not be uniformly available across regions or sectors. Establishing clear, measurable indicators that appropriately capture both the direct and indirect impacts of regulations is challenging. To measure such impact, evaluation should rely on the long-term data to assess the sustainability and effectiveness of regulatory frameworks, yet such data is often not available.

      Question 2. What indicators and methodologies can effectively capture both short-term results and long-term impacts?

      To effectively capture both short-term results and long-term impacts in policy evaluation, a combination of diverse indicators and methodologies is essential. Mixed-methods approaches, combining quantitative data analysis with qualitative assessments can facilitate review of policy impacts. Quantitative indicators could be obtained through national datasets, focusing on deforestation rates, carbon emissions, agricultural productivity, and economic metrics (e.g., income changes among smallholder farmers). These should be verified and complemented by qualitative methods, including surveys and interviews with relevant stakeholders and data providers. Analyzing specific instances of policy implementation can facilitate a deeper understanding of context, challenges, and successes that inform broader evaluations.

      Looking forward to further ideas and reflections.

      Best regards,

      Serdar Bayryyev

      Senior Evaluation Officer, FAO

       

       

      Read on a separate page
  • Discussion

    Evaluating Scaling Efforts: Measuring What Matters

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 09/12/2024

      Dear Hezekiah,

      Elaborating further on my responses, and answering your questions, the following considerations are quite important to note. The Food and Agriculture Organization (FAO) employs several context-sensitive evaluation approaches to assess the success of its initiatives effectively. Here are some specific strategies that FAO has used:

      1. Stakeholder Engagement: FAO actively involves stakeholders at various levels, including local communities, government bodies, and NGOs, in the evaluation process. This participatory approach helps to gather diverse perspectives, ensuring that evaluations are grounded in the realities and needs of those directly affected.
      2. Contextual Analysis: Prior to evaluations, FAO conducts thorough contextual analyses to understand the socio-economic, environmental, and cultural factors influencing agricultural practices and food security in specific regions. This helps in tailoring evaluations that reflect local conditions and challenges.
      3. Utilizing Mixed Methods: FAO employs a combination of qualitative and quantitative methods in its evaluations. By integrating surveys, case studies, focus group discussions, and interviews, the organization can capture a holistic view of outcomes and impacts, taking into account the complexities of local contexts.

      And ensuring the success of sustainability in scaling agricultural projects is critical for long-term success and impact. Several indicators can help assess the sustainability of scaling efforts in such initiatives. Here are some key indicators to consider:

      1. Economic Viability:
        - Cost-Benefit Ratio: Measure the economic returns from the project relative to the costs incurred.
        - Profit Margin: Assess the profitability of scaled operations.
        - Market Access and Value Chain Integration: Evaluate the ability of farmers to access markets, as well as their participation in value chains.

      2. Environmental Impact:
        - Soil Health: Monitor soil fertility, erosion rates, and organic matter content.
        - Biodiversity: Measure the diversity of crops and wildlife in the area influenced by the project.
        - Water Usage: Assess water efficiency and the sustainability of water resources used in agricultural practices.

      3. Social Equity:
        - Community Involvement: Evaluate the level of local stakeholder engagement and participation in decision-making.
        - Access to Resources: Monitor equitable access to land, credit, and inputs for all farmers, especially marginalized groups.
        - Gender Inclusion: Assess the involvement of women and marginalized groups in farming, decision-making, and benefits from the project.

      4. Resilience and Adaptability:
        - Risk Management Practices: Evaluate the adoption of practices that enhance resilience to climate change, pests, and market fluctuations.
        - Diversity of Income Sources: Analyze the diversification of income streams for farmers involved in the project.

      5. Technological Adoption and Innovation:
        - Use of Sustainable Practices: Measure the extent to which sustainable agricultural practices are adopted, such as organic farming, conservation agriculture, and agroforestry.
        - Innovation Transfer: Assess the effectiveness of knowledge and technology transfer mechanisms.

      6. Capacity Building and Education:
        - Training and Support Programs: Evaluate the availability and effectiveness of training programs for farmers.
        - Access to Information: Monitor the availability and accessibility of agricultural research and extension services.

      7. Impact on Local Communities:
        - Food Security: Measure changes in household food security levels in the target area.
        - Nutrition Improvement: Assess improvements in dietary diversity and nutrition among local populations.

      Hope this is useful and kindest regards,

      Serdar Bayryyev, Senior Evaluation Officer, FAO

      Read on a separate page
    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 05/12/2024

      Dear Hezekiah,

      We have assessed various programs and interventions of FAO and its partners, aimed at enhancing food security and sustainable agriculture. These reports are available from our website: https://www.fao.org/evaluation/en/

      In many instances, our evaluations informed stakeholders on whether efforts were made for scaling up the programs further, and whether scaling-up has worked or not. We have leveraged evaluation findings to provide actionable recommendations for scaling, highlighting key success factors which led to broader adoption strategies at larger scale or in other locations.
       

      Here are a few illustrative cases:
       

      1. Integrated Pest Management (IPM): An evaluation of the FAO's implementation of IPM in various regions showed significant improvements in pest control and crop yields. The evaluation highlighted how local farmer training and the adaptation of IPM strategies led to successful outcomes. This informed stakeholders about the potential benefits of scaling up IPM programs in other agricultural zones, particularly in countries facing similar pest challenges. The key success factors identified included participatory approaches involving local communities and leveraging traditional knowledge.
       

      2. The Improved Global Governance for Food Security initiative: This program aimed to enhance cooperation among stakeholders in food security. Evaluations revealed that enhanced governance structures were instrumental in enabling better stakeholder engagement and resource allocation. The findings recommended scaling up governance training at local levels as a best practice that could be adopted in other regions. The evaluation emphasized the importance of establishing clear communication channels and collaboration frameworks, which were pivotal in achieving positive food security outcomes.
       

      3. Climate-Smart Agriculture (CSA): Evaluations of FAO's CSA programs demonstrated how integrating climate resilience into farming practices benefited farmers in various countries. The assessments pointed out that scaling these interventions led to improved productivity and sustainability. Key success factors included the provision of technical assistance, access to climate information, and the establishment of demonstration plots. As a result, recommendations were made to expand similar CSA initiatives to other vulnerable agricultural areas.
       

      4. Food and Nutrition Security Program in Ethiopia: The evaluation of this multifaceted program indicated significant improvements in dietary diversity and access to food among targeted households. It highlighted the importance of community participation and localized content in training modules, which were critical for the successful implementation of the initiative. The recommendations called for expanding the program further into other regions of Ethiopia and beyond, based on the positive outcomes observed, thereby underscoring the crucial role of context-specific interventions.
       

      5. Support for Smallholder Farmers: An evaluation focused on FAO's initiatives to support smallholder farmers through access to resources and markets found that these programs had a high success rate in improving food security and income levels. The successful models were identified for potential scaling up in different contexts across Africa and Asia. The report emphasized the importance of building local partnerships and ongoing support mechanisms to ensure long-term sustainability and effectiveness.
       

      These evaluations, among others, illustrate how FAO and its partners use assessments to derive actionable insights that facilitate the scaling up of programs aimed at enhancing food security and promoting sustainable agriculture practices globally. The findings provide a roadmap for replication and adaptation in various contexts, ultimately contributing to broader efforts in achieving food security objectives.


      Lessons Learned:
      One key lesson has been the importance of context-sensitive evaluations that consider local conditions and stakeholder dynamics. Furthermore, engaging with local communities throughout the evaluation process enhances ownership and relevance of scaling initiatives. The criteria that could be used for assessing successful scaling could include the following: 
      - Evidence of increased access or adoption of innovations.
      - Positive changes in target outcomes (e.g., improved livelihoods, enhanced food security).
      - Sustainability of scaling efforts over time.


      I look forward to further discussions and shared learning on this important topic!
       

      Read on a separate page
  • Blog

    Responsive Evaluation: A joint reflection from Rome-based agencies on the 2024 National Evaluation Capacities Conference held in Beijing, PRC

    The landscape of governance and development is ever-evolving, necessitating robust mechanisms for evaluation to ensure that programs and policies are responsive to the unprecedented global challenges. The 2024 National Evaluation Capacities (NEC) Conference held in Beijing on October 14-18, served as a pivotal platform for stakeholders to come together and enhance their understanding of evaluation as a tool for accountability and informing timely and effective responses to the global challenges. The NEC Conference brought together over 500 participants from more than 100 countries representing...
    Read more
  • Discussion

    Evaluating sustainable pathways to climate-resilient agriculture: recent experiences from an IFAD evaluation

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 19/08/2024

      Dear Nanthi,

      Thank you for sharing your insights on evaluating sustainable pathways to climate-resilient agriculture. The examples you presented underscore important lessons vital for tackling the intersection of agri-food systems and climate change.

      1. The Role of Evaluation in Sustainable Agriculture  

      I appreciate your emphasis on the significance of evaluation in gauging the overarching context. Instead of concentrating solely on immediate project impacts, evaluations should reflect on the broader objectives of sustainable development. It is essential to analyze the interconnections and synergies among human, socio-economic, and ecological systems, alongside potential trade-offs.

      2. Innovative Methodologies  
      The introduction of a new rubric methodology to examine the relationship between agricultural interventions and ecosystem health is a noteworthy advancement. This innovative approach provides a nuanced perspective on how agricultural practices influence natural resources, facilitating more informed decision-making. Categorizing impacts by typologies will offer greater clarity for stakeholders and enhance their understanding of the associated risks and benefits.

      3. Agile and Responsive Evaluations  
      I completely concur that the unprecedented complexity of today's development challenges necessitates more responsive and proactive evaluation methods. Given the intensifying climate-related challenges, it is crucial to shift away from traditional, lengthy evaluation cycles in favor of more rapid and real-time assessments. Evaluations should deliver timely feedback that supports adaptive management, as such flexibility is vital in the face of rapid changes in both climate and agricultural environments.

      4. Integrating Cross-Cutting Issues 
      Integrating social impact assessments—including considerations for gender equality and women's empowerment—into the evaluation process demonstrates a comprehensive approach to development. By addressing social dimensions alongside environmental factors, a more holistic evaluation can guide decision-makers toward inclusive and sustainable agricultural practices, ensuring that solutions are tailored to respect local ecosystems and meet community needs.

      Best regards,  


      Serdar Bayryyev

      Senior Evaluation Officer

      Food and Agriculture Organization 
       

      Read on a separate page
  • Discussion

    Developing effective, inclusive and gender responsive MEAL systems

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 28/11/2022

      Dear Eriasafu,

      Thank you for raising these important questions and emphasizing the need for integrating the evaluation function with those of monitoring and learning. All these important functions are inter-connected and as such are integral parts of the effective project management cycle. It would, indeed, be the most effective approach if an evaluation could be integrated in the overall results-focused management system from the start, rather than be carried out as a “one-off” exercise occurring only at certain point of the project/programme cycle.

      I would like to highlight a few points that may stimulate further discussion on the questions that you have posted.

      Purpose and utility

      Before any monitoring and evaluation system is established or developed, its main users/stakeholders and developers need to be clear on the purpose and utility of the system. For example, the system that is mainly generating data for higher-level (corporate) reporting is different from the system that focuses on measuring benefits to the local community or beneficiary-level impact monitoring.  Any system developed need also be pragmatic, and take into account available capacities and resources, without creating or adding levels of complexity or functions that may not be used subsequently.

      Human rights and gender-related considerations.

      The monitoring and evaluation systems should adhere to the rights-based approaches, potentially identifying the effect of programmes on people realizing their human rights and identifying potential best practices of ensuring respect for human rights while implementing operations and programmes. The monitoring and evaluation processes and activities should duly integrate gender-related considerations, measuring any effect of programmes on women, girls, men and boys, and assessing benefits and deprivations. These can be achieved by integrating gender-sensitive indicators of performance in projects’ results and resources frameworks, identifying potential sources of gender-disaggregated information, and ensuring that gender-disaggregated data is collected and used for monitoring and evaluation, to the extent possible. 

      Addressing constraints

      Monitoring and evaluation activities should take due consideration of the projects and programmes’ operating environment and in particular potential constraints and risks. In emergencies and crisis-affected settings, there are multiple constraints and limitations, including issues of secure access to project sites and intended informants, availability and reliability of data, possible biases of key informants in crisis-affected settings due to their location or affiliation. These constraints can be addressed by using local partners/experts, using third party monitoring, and/or use of modern technology that allows remote monitoring and data collection (e.g. remote sensing via satellites, geospatial data available, digital data collection, mobile phone-based data platforms, remote sensing with satellites, etc.)

      Do No Harm

      In all contexts, and particularly in crises-affected settings, “Do No Harm” principles should be applied in planning and undertaking the monitoring and evaluation activities, bearing in mind potential sensitivities and tensions. The proposed M&E approaches should be planned in a manner that do not affect providers of monitoring and evaluation data, and do not exacerbate existing tensions, and/or worsen relationships between the informants and the other local actors/communities. 

      Kindest regards,

      Serdar Bayryyev (FAO)

      Read on a separate page
  • Discussion

    Principles for a healthy relationship between evaluators and experts

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 30/07/2022

      Dear Ibtissem,

      Thank you for posting a blog on this important issue. The complexity of development interventions requires that evaluation teams have various types of expertise. Yet, it is often quite difficult to secure right mix of expertise, and it is even more challenging to ensure that this expertise is effectively utilized during a short span of the evaluation process.

      Evaluation managers/leaders (hereinafter referred as Evaluation Team Leaders) need to make their thoughtful decisions on what expertise they need to obtain for the evaluation team, and what combination of expert knowledge and experience would work best for their evaluation.  In making these decisions and in leading the evaluation processes, Evaluation Leaders may indeed be guided by the five competency domains highlighted by Ibtissem  (or the Competencies for Canadian Evaluation Practice), which resonate with the 2016 UNEG Evaluation Competency Framework (http://www.unevaluation.org/document/detail/1915 ).  

      The role of the evaluation leader in managing the teams of experts, is indeed strategic and challenging, and I would like to add just a few considerations in this regard, to stimulate further discussion on this important topic:

      Establishing strong leadership

      Evaluation Team Leaders need to effectively lead the evaluation team, coordinating effective engagement of each expert in the team, validating their individual inputs and undertaking quality controls at key milestones. Continuous communication with each team member throughout the process is essential to ensure that any questions are potential risks of drawbacks are timely and effectively addressed.

      Ensuring compliance with organizational standards

      The Evaluation Team leaders need to require compliance of each team members with applicable rules and regulations, and standards of conduct established by the organization, and establish quality parameters of expected deliverables. In collaboration with each team member, Evaluation Team Leader should ensure that there is a shared understanding of what is to be evaluated and how, clarifying how organizational standards and processes are to be applied, and what is expected from each individual team member.

      Focus, focus, focus

      Biases are unavoidable, yet their influence can be minimized by focusing on evaluation objective and clearly establishing the scope of evaluation analysis. Furthermore, it should be made clear that any expert inputs and viewpoints need to be validated and supported with solid and relevant evidence.

      Building relationships

      Teambuilding skills are essential, to ensure that team members support each other and works towards achieving common objective.

      Engaging external expert panels

      If at all possible, external panels of experts may be engaged to provide feedback on key thematic areas of your evaluation analysis. Recognized experts from counterpart organizations may assist by serving as high-level expert panel members, to review findings produced.

      These are just a few examples, and I anticipate continued discussion on this topic. 

      Kindest regards,


      Serdar Bayryyev

       

      Read on a separate page
  • Discussion

    How to evaluate science, technology and innovation in a development context?

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 29/03/2022

      The need to evaluate science, technology and innovation in a development context

      The significance of science, innovation and research in supporting global efforts towards more sustainable and climate-friendly development is growing. There is an urgent need for relevant science, quality research and innovations that are ground-breaking, as the world is experiencing new and unprecedented challenges and crises. Evaluation of the quality of evaluation and research is essential in determining the usefulness of and effectiveness of science, innovation and research activities.  Evaluation findings should help decision-makers in determining important priority areas for further investigation and facilitate the decision-making on allocating resources for future research activities.

      Key limitations

      The evaluation of science and research is, however, quite complicated, and facing numerous methodological challenges. For example, assessment of the relevance and significance of scientific or research products is mostly based on the use of bibliometric methods. This is a quantitative method that may in fact produce solid evidence-based findings, yet its use is constrained by major limitations.  For example, not all science, innovation and research products are included and properly recorded in the bibliographic databases, or not even published, hence not all products can be assessed.

      The bibliometric methods often are based on calculating average number of citations, which also presents the basis for some biases. For example, there is sometimes an overly exaggerated attention to a specific author, who is known for previous work in a specific field or is affiliated with institutions that have strong political or financial support. In terms of citations, some authors may also deliberately exclude certain reference materials from their publications. Henceforth, whenever the bibliometric data analysis is used, it should be used with caution and should be combined with the use of other methods for validity purposes.

      The other major limitation is that in today’s complex world of science and innovation, there are various standards or criteria of assessing quality of research, science and innovation in various parts of the world, and various parts of science and innovation.

      Assessment of science and research products may also be biased due to the differences in political affiliation, beliefs, culturally or religiously-based perceptions of those who undertake these assessments or evaluations. 

      Key considerations

      As this stream of evaluation function is still evolving, there are few key considerations that need to be taken into account in undertaking relevant evaluation, or in developing appropriate evaluation tools and methods.

      Assessing relevance/significance of science and research.

      Assessment of relevance or significance of science, innovation and research products need to take due consideration to the context in which these products are to be used. What works in one context may not be suited for the other, and what constitutes innovation and ground-breaking science varies substantially depending on the intended use or users.

      Assessing effectiveness (or quality)

      In assessing the effectiveness of the research and scientific analysis, the key is to assess “influence” of these activities, or extent to which the science, innovation and research products have influenced the policies, approaches or processes.   

      Assessing the degree of “networking”, i.e. the degree to which the researchers and scientific institutions have interacted with all relevant stakeholders, including those that may have had a “negative” or opposing stance to the subject research theme/topic.

      Assessing Transformational nature

      In today’s world, perhaps, the most important criteria for assessing the relevance, use and efficiency of science, innovation and research activities, is whether these activities cause truly transformational change, or at least trigger important policy discourse on moving towards such transformational change.

      The above-written are suggestions for consideration, that are aimed at stimulating further feedback into this important discussion.

      Kindest regards,


      Serdar Bayryyev

      Senior Evaluation Officer

       

       

      Read on a separate page
  • Discussion

    Can we use an evidence-based, evolving Theory of Change to achieve "local learning” during project design?

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 19/01/2021

      Dear John,

      In my view, the Theory of Change, or logical framework, or any other method used to guide the design of the development intervention (project) is critical. These methods should be based on comprehensive analysis of the development context and the critical issues to be addressed to meet the needs and desires of local communities.

      Multilateral organizations have some examples of "community-driven development" projects. For example, the International Fund for Agricultural Development (IFAD) has recently published an evaluation synthesis of "Community-driven development in IFAD-supported projects" (IFAD, April 2020), which is based on the review of case studies of community-driven development projects. The theory of change used for this synthesis was based on the assumption that social capital and empowerment are at the center of the community-driven development approach. This theory of change assumes that participatory implementation process "...is expected to achieve a truly sustainable transformation of rural livelihoods by building poor peoples' capacities to make use of a wider range of livelihood options and by transforming community-government relations to better support people-centred development processes". This theory of change is illustrated in Figure 2 on page 5 of the synthesis paper accessible via the following link:

      https://www.ifad.org/documents/38714182/41898849/ESR+CDD+-+final+with+c…

      Kindest regards,

      Serdar Bayryyev

      Food and Agriculture Organization

       

      Read on a separate page
  • Discussion

    Cohérence des interventions dans le secteur agricole

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 30/11/2020

      Dear Nabyouré Jean Stanislas OUEDRAOGO,

      Thank you very much for raising this strategic question, which is at the heart of the discussions held today in the context of best practices and development pathways towards achieving the Agenda 2030. All countries, all developments stakeholders, communities and individuals should act in collaborative and productive partnerships to make progress towards achieving the Sustainable Development Goals. 

      The proliferation of development interventions could indeed become a negative factor, if their universe is composed of projects that do not exploit potential synergies and apparent complementarities, and are not based on partnerships based on solid analysis of mutual benefits generated from joining forces, capacities and resources toward common goals.

      We live in communities and environments that are affected by a multitude of factors that are interconnected, broadly defined as social, economic, environmental, health-related and other development factors. Accordingly, the development interventions should be developed with due attention top and in full consideration of these inter-linkages, inter-connections and trade-offs. There are examples of good practices used by development organizations to coordinate and consolidate the universe of development interventions to take full account of development context and exploit potential complementarities in addressing the inherent interlinked development issues and challenged. Some of these examples are show below: 

      • The United Nations Development Assistance Framework, if planned with due consideration of local context and with robust analysis of development challenges, would guide UN entities and other development actors  in producing coherent and well-coordinated package of development support towards achieving national development goals and objectives.
      • United Nations Global Compact that helped creating multi-stakeholder initiatives, supported by the UN, international financial institutions, private businesses and small and medium-sized enterprises, to address development challenges in a more coordinated manner.
      • The United Nations pooled funding mechanisms, which serve as channels for directing flows of development and humanitarian assistance from diverse groups of external factors through national budgeting and financing schemes, helped to improve effectiveness, reduce duplication and promote alignment among a wide range of actors. 

      These are just a few examples from the recent past. The new generation of the United Nations Sustainable Development Cooperation Frameworks (https://unsdg.un.org/sites/default/files/2019-11/UNSDG-SDG-Primer-Repor…), are being developed to move the development aid paradigm from assistance to cooperation, and from individual contributions by development agencies to a collective and coherent response to countries' opportunities, gaps and challenges. 

      The new Cooperation Frameworks will consider development priorities from multiple perspectives of the diverse groups of stakeholders, taking their views as the basis for developing coherent development support package. In doing so, the Cooperation Frameworks will aim at developing interventions that take full consideration of potential effects among different sectors. If done and implemented right, these Cooperation Frameworks will guide the transformation of development projects into a coherent and well-coordinated package of development assistance, aligned with the national plans towards achievement of Sustainable Development Goals.

      Just a reflection.

      Kind regards,

      Serdar Bayryyev

      Food and Agriculture Organization

       

       

       

       

       

       

      Read on a separate page
  • Discussion

    How to use Knowledge Management to strengthen the impact of Evaluation on smallholder agriculture development?

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 30/09/2020

      Dear Paul,

      Thank you for raising this important issue, frequently faced by development practitioners. While both functions, the M&E on one hand, and the knowledge management, on the other, contribute to organizational learning and effective programming towards generating intended benefits to the people and communities, these functions quite often are managed by different units and follow different organizational practices and cycles. To ensure that these functions complement each other, these have to be planned and synced, preferably at the design stage of the project. For example, the results from periodic monitoring conducted at quarterly intervals, may be used to produce knowledge products, such as newsletters and case study brochures to raise awareness on the results achieved. Annual monitoring exercises could be informing another type of knowledge products - such as the lessons learned. Utility of evaluations can be enhanced by wider dissemination and broadcasting (e.g. via social media, TV, radio), with support from knowledge management professionals.

      In a effort to ensure that both M&E and KM teams work collaboratively together, the following initial steps could be considered and applied: 1) discussing with programme management and beneficiaries their needs in M&E and KM products; 2) Agreeing on a joint plan of M&E and KM activities, focusing on complementarities and sequencing; 3) Developing a plan for effective utilization and dissemination of M&E and KM products.

      Kindest regards,

      Serdar Bayryyev

      Senior Evaluation Officer (FAO)

       

      Read on a separate page
  • Blog

    Evaluation to support the “leaving no one behind” agenda: insights from the UNDP NEC Conference 2019

    Over 500 participants from 100 countries joined this 6th NEC Conference to exchange and learn how to advance national evaluation agendas and to discuss the use of evaluation to accelerate the implementation of the SDGs. Leaving no one behind amid the rise in inequalities in society is a mounting concern in both the developed and developing world. Captured in Sustainable Development Goal 10, addressing inequalities needs to go beyond looking at the increasing gaps between higher and lower levels of income and wealth. Inequalities, in fact, touch on a multidimensional set of social, cultural...
    Read more
  • Discussion

    Challenges of evaluation

    • Serdar Bayryyev

      Italy

      Serdar Bayryyev

      Senior Evaluation Officer

      FAO

      Posted on 28/05/2019

      Dear Hynda,

      You have raised a very important question, which affects the quality of evaluation work. Evaluations of development programmes in the broadly defined areas of rural development, agriculture and food security are inherently complex. The assessments of results in these areas are affected by a multiplicity of biophysical, economic, and social systems and factors. There are different types of constraints and challenges in evaluation work that depend mostly on the context of the programmes or policy work being evaluated.  For example, accurate and timely assessments of potential impact and development change may be affected by the remote location of project sites, social stratification of rural communities, time required to produce productivity gains, adoption capacities of local communities, and many other factors.

      Evaluators often encounter issues with availability of baseline data, or information on the prevailing conditions of the development situation at the start of the projects or programmes addressing food security and agriculture development. This issue could be addressed by reconstructing baselines, for example, using ‘recall’ technique, i.e. requesting key beneficiaries or stakeholders to recollect information about these conditions in the past.

      Security situation in the country may also have a huge impact on the access to data and methods we chose for evaluation. The choice of evaluators could also be highly limited, as not all may have necessary clearance to visit high-risk areas, or experience in working in similar situations. 

      Accessibility of project sites may also be restricted or banned. To address these constraints, local consultants with access to restricted zones may provide support in data collection, and potential alternative evaluation methods could be also considered. In recent FAO’s evaluation of the large irrigation rehabilitation programme in Afghanistan, evaluation team faced a constraint of accessing some of the programme sites. The team opted for alternative method by using the open-source data from Google to assess the potential impact of the programme on the livelihoods in those specific sites. Google Earth maps were utilized to measure the expansion of the irrigated area and the vegetative cover along different sections of the rehabilitated canals. The methodology for measuring these areas was also using preliminary information from enumerators in the field who had access to the restricted zones, and were engaged in supporting collection of necessary data and information for the evaluation (e.g. the GPS coordinates of the irrigated areas in the vicinity of the irrigation canals). Then this information was analyzed based on historic data available from Google Earth on before- and after-project conditions and the changes based on vegetative cover at different periods during a year.

      These are just a few highlights of the constraints and challenges that evaluators may encounter in their work and an example of possible ways to address those. The range of such constraints is quite broad, and we encourage all members of this community to share their experiences in addressing different types of constraints and limitations.

      Kind regards,

      Serdar Bayryyev,

      Evaluation Officer

      Food and Agriculture Organization (FAO)

      Read on a separate page