Allow me a few reflections on communicating evaluation results for maximizing a project impact beyond the end of the project. I think most project evaluations are done to review compliance with initial objectives. While this is important it must be recognized that projects have a limited timeframe and will close shortly after the evaluation is completed. This makes the evaluation mostly a historic statement that can be quickly tossed into the dustbin of completed proclaimed successful projects as the development effort correctly moves on to focus on other active projects. Thus, I would contend that critical contribution of evaluations is the guidance they provide to future projects to better serve the intended beneficiaries. Thus, most critical people to communicate evaluation results are those who are funding and designing future projects. However, these individuals are often too busy working with new projects to critically review evaluation reports of closing projects. Also, to be effective the evaluations must be carefully designed, implemented, and analyzed. Here I have some serious reservations that have resulted in evaluations overlooking critical concerns or over relying on innovations the beneficiaries substantially reject. Please allow me to continue as it applies to projects designed to assist smallholder farmers.
Operational Feasibility – Dietary Energy Deficit
My first concern is evaluations are only as good as the criteria included and questions asked. If critical issues are overlooked than regardless of who receives the evaluation report, it will have limited impact on future projects or contribute to enhanced smallholder production, economic well-being or environmental sustainability. More likely it will just more deeply continue the current project designs that could keep smallholders entrenched in poverty. For smallholder development projects my concern is, are the production innovations operationally feasible. Our small plot research/ demonstration programs to a great job of determining what the physical potential of an area is, but do not address this issue of labor requirements or access to contract mechanization needed to extend the small plot results across a community. It is just assumed not to be a problem. Sorry but it is!!! Who within an agriculture development project is responsible for determining the labor or mechanization requirements to extend small plot results across a community in the timely manner needed to fully take advantage of the innovations? In 50+ years working with smallholder communities I have never seen this fully addressed. I think this issue falls into an administrative void between the agronomists or other bio-scientists who develop innovations and the social scientists who assist smallholders implement them. How easy would it be to include this in an evaluation? Would this explain why there is often very limited acceptance of production innovations, despite extensive effective extension/ education efforts? Don’t we tend to concentrate on informing our less educated colleages than listening to them or carefully observing the field activity, even if adjacent to our demonstration plots.
A big part of this operational feasibility is the dietary energy deficit horrors. That is for a full day of manual agronomic field work you need a diet of 4000+ kcal/day while most smallholder are lucky to get 2500 kcal/day of which 2000 kcal/day must be allocated to basic metabolism leaving only 500 kcal/day for work energy which is enough for only a couple hours of diligent effort, perhaps paced over a couple more. The result is 8+ weeks for basic crop establishment which quickly voids all mid-season crop husbandry activities. This quickly becomes a very strong recommendation of facilitating access to mechanization if we are ever going to lead smallholders out of poverty. However, we seem determined to emphasis manual labor in our promotions without realizing this is poverty entrapment. A community just cannot manually hoe its way out of poverty!!
What would it take to get this concern incorporated into project evaluations? I would think an 8-week spread in crop establishment would be easily visible and should not be dismissed as an educational problem particularly as most smallholder farmers are aware their food security is at stake. While we have recognized smallholder farmers are poor and hungry for decades, why has this not been looked at as a major hinderance to crop husbandry? What questions would get to the crux of the problem? How easy would it be to evaluate the dietary energy available? What is the probability most of our crop management innovations compel smallholder farmers to exert more caloric energy than they have access to? Please note that compelling farmers to exert more caloric energy than they have access to could be referred to the International Criminal Court is The Hague as a “crime against humanity”. Is it possible to obtain widespread community wide acceptance of production innovations without addressing this issue in future projects? Isn’t facilitating access to mechanization the key to improving smallholder agriculture? Who should receive evaluation reports that include this issue so it can be addressed in future projects?
Compliance Evaluations – Cooperative Scandal
Evaluations to document compliance with project objectives are usually internal evaluations rather than independently contracted evaluations. This is basically due to the cost of hiring external evaluators. Thus, the desire for project extensions and future projects makes it almost imperative to skew the evaluations toward a most favorable position rendering them propaganda document from contractor to donor. My best example of this is a coffee cooperative representing some 21,800 growers proudly marketed some 181 metric tons of presumed green bean coffee. This sounds impressive but divide it out and you have 8.3 kg/grower. If a coffee plant typically produces 2 kg of marketable green-bean, the cooperative’s market share is only 8 plants/grower? How many growers are living off only 8 coffee plants? If your average coffee farmer produces I ha of coffee yielding 225 kg of green bean the 8.3 kg would represent only 7% of the production. What happened to the other 93% of the production. Was it marketed through those usually vilified private traders? I would claim the cooperative impact on its members was trivial!! Yet this was promoted and published as a great successful project. Such skewed reporting may help the project contractor with project extensions and future projects, but it is a disservice to the intended beneficiaries and to the extent in encourages future projects to rely on the cooperative business modal a major disservice to the overall agriculture development process allowing billions of US Dollars or Euros to be squandered on programs the beneficiaries are wisely avoiding, as has been the case for nearly 40 years.
While I would like to view this as an exception, in my decade long quiet review publications promoting the cooperative business model imposed by development projects on smallholder communities, I regrettably find it more normal. Here I think the evaluation process has collected good reasonable data, but the analysis emphasizing aggregate analysis is misleading. Aggregate analysis can often provide some impressive data as noted above, good for propaganda promotions, but as shown above, can quickly be reduced to trivia with some simple conversion to percentage. Not good for guiding future projects to better serve the smallholder communities. The need here is for donors to demand more critical evaluation that emphasis targets, preferable percentage targets, that will separate success from failure for such parameters as percentage of eligible farmers actively participating, percent market share within members and community, percent financial benefit for participating, etc. In essence the basic business parameters that determine the success of failure of any business enterprise. Has anyone ever seen these parameters express in project evaluations of imposed producer organizations? Please check the reference below for list of parameters I would like included in evaluation of producer organizations as well as what would be accepted by underwriting donor taxpayers as target compliance. The results I have noted are not even close!! Given how producer organizations – cooperatives are promoted as the most ideal business model available but fail to attract a sizeable participation and trivial market share for the community they serve, one must wonder how much the project is committed to serving the smallholder beneficiaries vs. imposing a socially desirable but non-competitive business model.
Again, if the evaluations are more a propaganda tool to cover up what by normal standards are total failure it really doesn’t manner who receives them. They will do more harm than good.
Summary
As I mentioned at the beginning the real benefit of any evaluation is how well it guides future projects to better serve the intended beneficiaries. The ability to provide such guidance is highly dependent on not overlooking critical issues such as the operational feasibility of innovations and related concern for dietary energy deficit or using evaluations to propagandize innovations such as producer organizations that are wisely rejected by beneficiaries. If we are sincerely interested in assisting smallholder producers out of poverty these issues must be noted in evaluations and address. If not, I am left wondering if there is something more sinister taking place. That is while I accept the implementing personal are devoted, the donor might have a larger more political agenda of subverting host governments for political purposes resulting in going through the motions of assisting people without accomplishing much, as excellently described by John Perkins in his text “Confessions of an Economic Hitman”. A text I would rate at about 80% accurate.
For more details I refer to you an article I prepared for a symposium at Colorado State University reflecting on my 50+ years assisting smallholder communities. The website link is: https://agsci.colostate.edu/smallholderagriculture/wp-content/uploads/sites/77/2023/03/Reflections.pdf . It carefully reviews the issues I have mentioned with links to additional webpages. I hope you can take some time to review the article and it will provide guidance on how to improve evaluations so they better guide future development projects that will truly benefit smallholder communities.
RE: Beyond the final report: What does it take to communicate evaluation well?
United States of America
Richard Tinsley
Professor Emeritus
Colorado State University
Posted on 09/11/2025
Allow me a few reflections on communicating evaluation results for maximizing a project impact beyond the end of the project. I think most project evaluations are done to review compliance with initial objectives. While this is important it must be recognized that projects have a limited timeframe and will close shortly after the evaluation is completed. This makes the evaluation mostly a historic statement that can be quickly tossed into the dustbin of completed proclaimed successful projects as the development effort correctly moves on to focus on other active projects. Thus, I would contend that critical contribution of evaluations is the guidance they provide to future projects to better serve the intended beneficiaries. Thus, most critical people to communicate evaluation results are those who are funding and designing future projects. However, these individuals are often too busy working with new projects to critically review evaluation reports of closing projects. Also, to be effective the evaluations must be carefully designed, implemented, and analyzed. Here I have some serious reservations that have resulted in evaluations overlooking critical concerns or over relying on innovations the beneficiaries substantially reject. Please allow me to continue as it applies to projects designed to assist smallholder farmers.
Operational Feasibility – Dietary Energy Deficit
My first concern is evaluations are only as good as the criteria included and questions asked. If critical issues are overlooked than regardless of who receives the evaluation report, it will have limited impact on future projects or contribute to enhanced smallholder production, economic well-being or environmental sustainability. More likely it will just more deeply continue the current project designs that could keep smallholders entrenched in poverty. For smallholder development projects my concern is, are the production innovations operationally feasible. Our small plot research/ demonstration programs to a great job of determining what the physical potential of an area is, but do not address this issue of labor requirements or access to contract mechanization needed to extend the small plot results across a community. It is just assumed not to be a problem. Sorry but it is!!! Who within an agriculture development project is responsible for determining the labor or mechanization requirements to extend small plot results across a community in the timely manner needed to fully take advantage of the innovations? In 50+ years working with smallholder communities I have never seen this fully addressed. I think this issue falls into an administrative void between the agronomists or other bio-scientists who develop innovations and the social scientists who assist smallholders implement them. How easy would it be to include this in an evaluation? Would this explain why there is often very limited acceptance of production innovations, despite extensive effective extension/ education efforts? Don’t we tend to concentrate on informing our less educated colleages than listening to them or carefully observing the field activity, even if adjacent to our demonstration plots.
A big part of this operational feasibility is the dietary energy deficit horrors. That is for a full day of manual agronomic field work you need a diet of 4000+ kcal/day while most smallholder are lucky to get 2500 kcal/day of which 2000 kcal/day must be allocated to basic metabolism leaving only 500 kcal/day for work energy which is enough for only a couple hours of diligent effort, perhaps paced over a couple more. The result is 8+ weeks for basic crop establishment which quickly voids all mid-season crop husbandry activities. This quickly becomes a very strong recommendation of facilitating access to mechanization if we are ever going to lead smallholders out of poverty. However, we seem determined to emphasis manual labor in our promotions without realizing this is poverty entrapment. A community just cannot manually hoe its way out of poverty!!
What would it take to get this concern incorporated into project evaluations? I would think an 8-week spread in crop establishment would be easily visible and should not be dismissed as an educational problem particularly as most smallholder farmers are aware their food security is at stake. While we have recognized smallholder farmers are poor and hungry for decades, why has this not been looked at as a major hinderance to crop husbandry? What questions would get to the crux of the problem? How easy would it be to evaluate the dietary energy available? What is the probability most of our crop management innovations compel smallholder farmers to exert more caloric energy than they have access to? Please note that compelling farmers to exert more caloric energy than they have access to could be referred to the International Criminal Court is The Hague as a “crime against humanity”. Is it possible to obtain widespread community wide acceptance of production innovations without addressing this issue in future projects? Isn’t facilitating access to mechanization the key to improving smallholder agriculture? Who should receive evaluation reports that include this issue so it can be addressed in future projects?
Compliance Evaluations – Cooperative Scandal
Evaluations to document compliance with project objectives are usually internal evaluations rather than independently contracted evaluations. This is basically due to the cost of hiring external evaluators. Thus, the desire for project extensions and future projects makes it almost imperative to skew the evaluations toward a most favorable position rendering them propaganda document from contractor to donor. My best example of this is a coffee cooperative representing some 21,800 growers proudly marketed some 181 metric tons of presumed green bean coffee. This sounds impressive but divide it out and you have 8.3 kg/grower. If a coffee plant typically produces 2 kg of marketable green-bean, the cooperative’s market share is only 8 plants/grower? How many growers are living off only 8 coffee plants? If your average coffee farmer produces I ha of coffee yielding 225 kg of green bean the 8.3 kg would represent only 7% of the production. What happened to the other 93% of the production. Was it marketed through those usually vilified private traders? I would claim the cooperative impact on its members was trivial!! Yet this was promoted and published as a great successful project. Such skewed reporting may help the project contractor with project extensions and future projects, but it is a disservice to the intended beneficiaries and to the extent in encourages future projects to rely on the cooperative business modal a major disservice to the overall agriculture development process allowing billions of US Dollars or Euros to be squandered on programs the beneficiaries are wisely avoiding, as has been the case for nearly 40 years.
While I would like to view this as an exception, in my decade long quiet review publications promoting the cooperative business model imposed by development projects on smallholder communities, I regrettably find it more normal. Here I think the evaluation process has collected good reasonable data, but the analysis emphasizing aggregate analysis is misleading. Aggregate analysis can often provide some impressive data as noted above, good for propaganda promotions, but as shown above, can quickly be reduced to trivia with some simple conversion to percentage. Not good for guiding future projects to better serve the smallholder communities. The need here is for donors to demand more critical evaluation that emphasis targets, preferable percentage targets, that will separate success from failure for such parameters as percentage of eligible farmers actively participating, percent market share within members and community, percent financial benefit for participating, etc. In essence the basic business parameters that determine the success of failure of any business enterprise. Has anyone ever seen these parameters express in project evaluations of imposed producer organizations? Please check the reference below for list of parameters I would like included in evaluation of producer organizations as well as what would be accepted by underwriting donor taxpayers as target compliance. The results I have noted are not even close!! Given how producer organizations – cooperatives are promoted as the most ideal business model available but fail to attract a sizeable participation and trivial market share for the community they serve, one must wonder how much the project is committed to serving the smallholder beneficiaries vs. imposing a socially desirable but non-competitive business model.
Again, if the evaluations are more a propaganda tool to cover up what by normal standards are total failure it really doesn’t manner who receives them. They will do more harm than good.
Summary
As I mentioned at the beginning the real benefit of any evaluation is how well it guides future projects to better serve the intended beneficiaries. The ability to provide such guidance is highly dependent on not overlooking critical issues such as the operational feasibility of innovations and related concern for dietary energy deficit or using evaluations to propagandize innovations such as producer organizations that are wisely rejected by beneficiaries. If we are sincerely interested in assisting smallholder producers out of poverty these issues must be noted in evaluations and address. If not, I am left wondering if there is something more sinister taking place. That is while I accept the implementing personal are devoted, the donor might have a larger more political agenda of subverting host governments for political purposes resulting in going through the motions of assisting people without accomplishing much, as excellently described by John Perkins in his text “Confessions of an Economic Hitman”. A text I would rate at about 80% accurate.
For more details I refer to you an article I prepared for a symposium at Colorado State University reflecting on my 50+ years assisting smallholder communities. The website link is: https://agsci.colostate.edu/smallholderagriculture/wp-content/uploads/sites/77/2023/03/Reflections.pdf . It carefully reviews the issues I have mentioned with links to additional webpages. I hope you can take some time to review the article and it will provide guidance on how to improve evaluations so they better guide future development projects that will truly benefit smallholder communities.
Thank you
Dick Tinsley
Prof. Emeritus
Soil & Crop Sciences
Colorado State University