Posted on 12/12/2024
Measuring What Matter – Critical Overlooked Evaluation Parameters
Based on the sub-title of this evaluation please allow me to expand on a couple critical but overlooked parameters that once included could substantially change the way we assist smallholder communities and perhaps better was to scale up their acceptance of innovations intended to assist them improve their well-being. Thes parameters may possibly be visibly seen without even interviewing farmers, or at least any interviewed data visibly confirmed.
The critical parameter is “Timing of Field Operations.” Has anyone ever considered this? How long does it take for farmers to complete basic tasks such as crop establishment, weeding, harvesting, and threshing? How does this compare with what development specialists anticipate? Why are there such delays? Typically, the expected time for crop establishment is only 2 weeks, however if carefully evaluated for manual farming, as most smallholder still do, it would be accuratly identified as stretching for over 8 weeks. Well beyond the time anticipated for optimum implementation of intermediate activities such weeding, and with well understood decline in yield potential. A simple physical observation of the fields associated with a project community would easily show this spread in crop establishment. The delay is normally casually noted by the development personnel but not evaluated and attributed to poor motivation, education on the importance for early planting, or spreading risk. I tend to dismiss all of these and add another more likely possibility. That is farmers have limited dietary calories relative to what is needed for a full day of agronomic field work. We tend to recognize farmers are poor and hungry but fail to factor hunger as a hinderance to crop management. This would be another critically overlooked parameter, with surprisingly little solid data. What limited data is available shows smallholder would be lucky to have access to 2500 kcal/day which after allowing 2000 kcal/day for basic metabolism leaves only 500 kcal/day for manual exertion. Against this limited diet the dietary requirement for a full day of agronomic field work is 4000 kcal. With only 500 kcal workday is limited to a couple diligent hours perhaps paced for a couple more hours at lower diligence. It noted to Kenyan introduce to the topic that in Kenya the casual workday is only 5 hours. If we added this to our evaluation parameters, would we get a better understanding of what smallholders are facing and provide guidance to better means for scaling up crop management and food security? However, it has to be recognized as a major hinderance to scaling agronomic acceptance before it will be included as a necessary parameter for evaluations.
Another way to quickly approach to the problem and perhaps confirm the need to address the issue, is to simply make a field visit to where it is possible to observe a variety of farms and see how many people are working in the field in the morning and again after lunch. I expect there will be a substantial decline for the afternoon observation. That was the case in Madibira, Tanzania, the 3000-ha irrigated rice project I worked on, and stimulated my interest in the dietary energy balance concern.
If dietary energy is a critical issue in scaling agriculture production, how critical will it be facilitating smallholder access to contract mechanization to reduce the drudgery for smallholder farmers and enhance the time it takes for crop establishment and allow them to concentrate of other important crop management activities? The importance of mechanization to smallholder farming can be easily seen in paddy producing Asia where some 30+ years ago farmers shifted from water buffalo to power tillers. While there is limited data on it impact, I get the impression it half the crop establishment period, increased the area individual farmer could manage, comfortable allow for double cropping irrigated rice and when small combines were added allowed for 5 rice crops every 2 years. Also, look at Egypt where contract land preparation with tractors has been the common practice for at least 40 years I can account for. Is it possible to scale up smallholder production without facilitating access to mechanization? It might be worth noting that a return visit to Madibira 5 years after the advisor effort ended, the farmers had obtained 50 Asian power tillers for use in the paddy irrigation scheme plus 4 4-wheel tractors for use on the adjacent upland areas. If the development community has limited interest in mechanization the farmers certainly do. The critical need to the financial arrangements that will allow individual owner/operators to obtain the necessary tractors to serve their communities.
Please note the referenced article I prepared reflecting on my 50+ years working with smallholder communities. The article is more interested in factual accuracy than political correctness with a major section on Dietary Energy Balance and critical need for mechanization for smallholder communities. The link is: https://agsci.colostate.edu/smallholderagriculture/wp-content/uploads/sites/77/2023/03/Reflections.pdf
Thank you
United States of America
Richard Tinsley
Professor Emeritus
Colorado State University
Posted on 18/04/2025
Please allow me to make some additional comments on the effectiveness of large vs. small scale projects.
First regardless of the size, how much development projects are more top-down impositions rather than a bottom-up appraisal of what the beneficiary really needs, and how this results in some major oversights that need to be addressed and over reliance on solutions mostly rejected by the beneficiaries.
Project Development Process
Most projects start with the donor who, since they have the money and discretion on how it is to be spent, control the project development process. Donors are also removed from the beneficiaries by 4 administrative layers. That are:
Development projects are usually initiated by the donor, often based on ideas that are socially desirable in the donor country, even if not well received in the donor country. Thus, largely imposed. The project concept is agreed upon by the host government at the ministry level with no input from the intended beneficiaries who are yet to be identified, because the beneficiary communities are yet to be identified.
This starts the million plus dollars and two plus year project development process, with only brief, perhaps one- or two-day, site visit to meet with selected potential beneficiaries. During this process most basic decisions on what project are going to do are made, and expatriate advisory staff identified to comply with such design. With that much up-front investment no one wants to learn the project is off target. This is true for both large and small projects with similar costs being incurred. These similar up-front costs are the primary reason for my original comment on relative overhead costs being similar for both large and small projects.
Finally, the implementation contract is signed and staff fielded, so detailed unbiased interaction with beneficiaries is possible. However, what is left to decide. Perhaps some fine tuning, but too late for major adjustments in project design that would require substantial staffing adjustments. Thus, the participatory interaction has little choice but to lever the results to what has already been planned and make the best of it, with any reporting adjusted to proclaim the success of the project. This appeasement reporting is essential to ensure project extensions and future projects. However, it might have little impact on the intended beneficiaries.
Please review the following webpages for more details on the above concepts:
What has been overlooked
As mentioned at the beginning, the current smallholder development efforts as some major oversights that hinder the large-scale acceptance of innovations across smallholder communities.
The underlying problem is participatory activities either at the beginning of a project or during an evaluation of an ongoing or conclusion of a project are only as good as the questions or criteria included in interviews or simply observed in the field.
My biggest concern is the “Operational Feasibility” of innovations designed and promoted for the benefit of smallholder producers either to enhance production or enhance environmental issues of sustainability or resilience. The issue is that agronomists and environmentalists with their replicated small plot research methodology do a great job of determining what is physically possible but say nothing about the operational resources need to extend the results across an entire community. Who within the development projects is responsible to determine the labor requirements or access to contract mechanism required for an innovation to be implemented across an entire smallholder community in the timely manner to take full advantage of the innovations; how available are the operational resources within the community; and what are the rational compromises smallholders should undertake to adjust the innovations to their limited operational capacity? I think this concern falls into an administrative void between the bio-scientists developing technologies and the social scientists assisting in the implementation. Can someone step in and fill this void?? What would it take to identify the issue? Perhaps a simple field observation of the up to 8-week delay in crop establishment, and a few questions on to determine if it is a limited operational capacity and not an educational problem emphasizing importance of early planting or a risk management effort.
A large component of this could be “Dietary Energy Balance” that is the caloric energy we are compelling smallholder farmers to exert in complying with our promoted innovations vs. the caloric energy they have access to. How many of you acknowledged that most smallholders are poor and hungry, but do not factored diet as a major hinderance to agronomic implementation? If you expect smallholders to undertake a full day of diligent agronomic field work, they need a diet of 4000+ kcal/day. However, they are lucky if they have access to 2500 kcal/day which when you subtract 2000 kcal for basic metabolism leaves only 500 kcal/day for physical exertion to undertake the heavy manual field work. The 500 kcals would be good for only a couple hours of diligent effort, perhaps paced over a couple additional hours with reduced diligence. Thus, how many days will be required to put in the estimated 300-person hours required to manually till a hectare of crop land?? What will the potential yield reduction be by the time you have completed the basic crop establishment for a hectare?? Thus, what is the probability your efforts are compelling smallholder farmers to exert more caloric energy than they have access to?? If you are compelling farmers to exert more energy than they have access and someone asked the International Criminal Court in The Hague to review your work, what could be the dire consequences of such an inquiry??
While we love to show pictures of people hoeing land as being more picturesque, is it really possible to hoe your way out of poverty?? Or do such pictures represent deeply entrenched poverty?? Please note the projects are designed around entire communities and not just individual producers. Thus, while it is possible, and it is often done for demonstration purposes, to hire laborers to expedite crop establishment for a small demonstration area; it must be recognized that often hired laborers are other smallholder farmers opting for a day of casual labor at the expense of their own field operations. Thus, while making for a great demonstration it is not extendable throughout the entire smallholder community. Labor is very finite!!
Why have the operational limits not been identified decades ago?? What would it have taken in an initial diagnostic survey or later evaluation process to identify this? How about a simple field observation of the spread in timing? Or perhaps ask how many hours a workday constitutes? In Kenya the casual laborer day is only 5 hours, with no double shifts. When working in Madibera, Tanzania I noted in the morning the fields were full a people, but very few in the afternoon, although there was plenty of need to do. This observation stimulated my interest in dietary energy balance. Also, how about an inquire into how much maize or rice people retain for family use (200 kg of maize per adult), convert that to dietary calories and estimate on number of hours a person could healthily undertake agronomic field work?? Not complicated inquiries!! Can you separate a person loafing around the village in the afternoon being lazy needing some motivation or hungry needing a hearty meal? Do our extension activities, now emphasizing Farmer Field Schools, address or ignore the issue of operational feasibility? How often do they assume it is not a problem and lack of acceptance related to the limited formal education of many smallholder producers and thus having difficulty in learning the technology? That is badgering smallholders with information they have a basic understanding of, but not the resources to take advantage of!!
If the key to development is to enhance the operational capacity for smallholder producers and reduce the dietary energy deficit, how critical is it to enhance access to contract mechanization for smallholder communities? Are there any other alternatives? I can’t think of any! What are the examples of how mechanization has impacted smallholder production and economic well-being of smallholder producers? The best example would be the shift from water buffalo to power tillers for paddy production throughout the rice producing areas of Asia now some 30 – 40 years ago. However, the impact is poorly documented because it was mostly done independently from the development effort. My best guess is it halved the paddy establishment time, expanded the land area individual farmers could manage, comfortable allowed for annual double cropping rice in irrigated areas, and when small combines were introduced allow for 5 crops every 2 years. All of which greatly enhanced the economic well-being of the smallholder rice producers and provide the rural labor surpluses that fueled the Asian Tiger Economies allowing them to become food security with ample export potential. Thailand being primary example and not the worlds leading exporter of rice.
Didn’t this reduce the external assistance for agriculture development in rice producing countries over the past couple decades?
The other example would be Egypt and much of the Middle East where most of the smallholder farmers have relied on privately owned and operated 65 hp tractors for over 40 years that I have observed starting in 1980, when on my initial assignment in Egypt. In the subtropical climate of Egypt these tractors operations had at least 220 annual workdays just tilling land and excluding extra hauling opportunities. That would be equivalent to the USA or EU work year when factoring in weekends, holidays and annual vacation days.
Given the above commentary I have a difficult time seeing any substantial reduction in poverty across smallholder communities without addressing facilitating access to appropriate mechanization. The real trick to how to do it, and how to develop the financial means for individuals to obtain and operate the equipment. Unfortunately, successful mechanization as discussed above can only be via private individual owner/operators. Any form of joint ownership via government mechanization units or cooperative results in the equipment being surveyed out of service with less than half the designed operating hours. Just look the line-up on out-of-service tractors in any ADP in Nigeria. Also, the owner/operator should be someone willing to drift out of direct farming to become a full-time mechanization service provider, as providing mechanization will be a conflict of interest with any continued farming activity. So how do we provide financing for both the original cost of the equipment and perhaps enough operating funds so the initial land preparation can be done on credit for an in-kind payment at harvest!!
Please review the following webpages for more details on the operational feasibility, dietary energy balance, and critical need for access to mechanization:
What has been over relied upon
The development project imposition on smallholder communities that has been excessively relied upon by donors but rejected by farmers are farmers organizations such as the cooperative business model. These are the darling of development world heavily promoted by academia but having only limited acceptance in the USA and outright rejected in most smallholder communities, unless they are involved with some form of value added usually involving extensive mechanized processing such as milling of coffee beans to produce green coffee.
While in the USA the cooperative movement had a major positive impact 100 years ago, its membership and market share have been trending down for decades so that the USDA, which promotes and monitors the USA cooperative movement, stopped monitoring and reporting membership and market share some 20+ years ago. When it stopped monitoring the market share was less than 20% and that included areas such as the cotton cooperatives in the Mississippi Valley having monopoly control over the marketing of cotton. When operating in a competitive environment with private dealers the market share was even less. Not really the best performance to compelling their use for developing countries.
However, despite the limited use in the USA, they have been mandated for use in development projects serving smallholder communities for nearly 40 years. This is based on the presumption that because they are farmer owned, they will have a competitive advantage providing the farmers with additional income, and the near vilification of private traders for trying to exploit the farmers, which could have some slander liabilities if pursued by the private traders. This is all done by edict without any documented analysis. Has anyone ever seen the cost of doing business comparison between a development project cooperative and competing private dealers? I have looked for this comparison for over 20 years and never seen any. But isn’t the first order of setting up a business analyzing the competition to make certain you have something beneficial to offer??
If someone took a close financial evaluation of the comparison between the development project cooperatives and private traders, it would be very difficult for them to develop a completive advantage simply because the sustainable overhead costs to operate a cooperative, even excluding the facilitating cost covering the donors’ inputs, are greater than the private traders profit margin. Part of this is because profit margins are held down by the limited buying power of the impoverished society they serve. This excludes the inconvenience of the consignment selling through the cooperative and delay between consigning goods to the cooperative and waiting for them to sell the goods before getting paid. This is in a society that prefers to hold their goods in their homes until it is necessary to sell them enroute to the market and need immediate cash to spend in the market for immediate needs before walking home. Sale volume being what the wife can carry on her head to market. I would venture that detailed analysis of relying on cooperatives for marketing goods would force smallholder farmers deeper into poverty!!
The net result is, despite all the promotion, the smallholders wisely avoid the development compelled cooperatives like they have the plague!! Thus, such cooperatives are fortunate to attract 10% of the community members, and even those will side-sell the bulk of their production to the vilified private traders for cash. The cooperative market share in the community being served is a paltry 5% or less, for a net trivial impact on the community’s economy. Most of what is consigned to the cooperative is probably in-kind loan repayments. You would think the project would be embarrassed by this limited impact and do some massive overhaul of the approach, however the reporting claims them the salvation of the smallholders and for 40 years have continued to be the foundation of assistance. Such appeasement reporting will be high on annotations but low on basic business parameters that normally measure the success vs. failure of a business enterprise. This is necessary to encourage continued funding and additional projects, with no concern about enhancing the economic well-being of the smallholder producers. Such cooperatives collapse almost immediately once external support and facilitation ends. Probably before the last advisor clears the departure lounge for the flight home.
Again, please review the following webpages that go more in-depth on my reservation of development mandated cooperatives:
Who is responsible for addressing and overcoming the limited effectiveness of development projects to assistance smallholder producers’ communities they are intended to serve?
From my perspective the failure of agriculture development programs to accurately identify the needs for smallholder communities and evolve programs to better serve their needs rest with the Monitoring & Evaluation effort. While M&E programs can do an excellent job documenting the activities of a project, the most important task is to guide future projects to better serve the intended beneficiaries. Given the degree to which, in both large and small projects, are by necessity more imposed on the communities than collaboratively developed with the community members, the M&E effort becomes the final voice for the beneficiaries and needs to accurately determine the degree the programs are appreciated and relied upon vs. mostly rejected. Unfortunately, too often the M&E effort become more a promotional effort to promote the projects regardless of true success or failure. The M&E analysis often concentrates on aggregate data that can provide some impressive numbers but are truly meaningless. Good for propaganda promotions but not guiding future projects. Instead, an aggregate analysis frequently entrenches projects that are failure and avoided by most intended beneficiaries. Often when the aggregate data converted into a percent of what could be possible the results are trivial as shone for the coffee cooperative in Ethiopia mentioned in one of the referenced articles. M&E is also the voice of the underwriting taxpayers to determine if their taxes are well spent or wasted. Thus, the M&E effort needs to be independent.
Most of the issues mentioned above could have and should have been identified by a well-designed independent M&E program several decades ago and promoted some major adjustment in programs that would have resulted a much greater effective results leading smallholder communities to an enhance food security and lifestyle as enjoyed by their Asian paddy farming compatriots. A key to doing this could be establishing targets expressed as a percentage of potential for the various M&E criteria that would separate success from failure, and these targets might be close to what the interested underwriting taxpayers would find acceptable. This could be the percentage of farming families participating with an expectation of 60+% but a reality of only 10%, thus a total failure. Or percent market share in the community with an expectation again of 60+% but perhaps lucky to have 5% and thus a massive failure. Please note I have left the “L” out of the MEL as currently managed there is only limited learning other than how to deceive the beneficiaries and underwriting taxpayers.
A final couple of webpages for consideration:
With this I will close and thank those who took the time to sort all through my provocative commentary. I hope you appreciate it and provide thoughts for improving the design of future projects to better assist the poverty alleviation and environmental sustainability smallholder communities. I think the ideas in this analysis are more factual accurate than politically correct. I will end with a link to an article I wrote for a recent symposium here at Colorado State University reflecting on my 50+ years assisting smallholder communities. This will again be more factual accurate than politically correct as only an emeritus individual no longer beholden to the system can freely express.
Thank you,
Dick Tinsley
United States of America
Richard Tinsley
Professor Emeritus
Colorado State University
Posted on 05/04/2025
I quick comment is the overhead cost to the donor in supervising projects, as a percent of the total value of the project, is less for large projects than for smaller projects. Thus, donors prefer the larger projects to keep their cost down, and assuming they are equally effective. I agree that larger projects are most likely less effective for the reasons stipulated.