How should a producer organization judge the return on its research?

Across Canada, provincial crop commissions, national commodity councils and grower-funded foundations together direct a large share of the money spent on applied agricultural research. The money comes from producers through levies and check-offs, and producers reasonably want to know what it bought.

That question is harder to answer than it looks. A new variety, an agronomic practice or a pest-management tool may take years to develop, more years to be adopted, and its benefits then run for decades. Several funders usually share the credit. Weather and prices swamp the signal in any single year. Yet boards still have to decide each year which projects to fund, and they still have to report to the people paying the levy.

Here is how we think about the problem.

Two different questions

“What is the return on our research?” bundles two questions that are worth separating.

Did the research work? That is, did the variety, practice or tool improve yields, reduce costs or open a market, compared with what would otherwise have happened?

How much of that improvement belongs to us? Most applied research is co-funded by public agencies, other commissions, universities and private firms, and it builds on earlier work. Credit has to be attributed, and any attribution rule is partly a convention.

Reporting the two separately is more honest than a single headline ratio. It also makes clearer what the organization actually controls.

The standard toolkit, and its traps

The established approach to valuing agricultural research is the economic surplus model, set out in detail by Alston, Norton and Pardey in Science Under Scarcity. Research shifts the supply curve, by lowering costs or raising yields. The model estimates the resulting gains to producers and consumers over time, then compares them with research costs through a benefit–cost ratio or internal rate of return.

The framework is sound. The results depend on a handful of assumptions that deserve more scrutiny than they usually get:

• Adoption. How fast, and how far, will producers take up the innovation? Benefits are very sensitive to the assumed adoption curve.
• Lags. How long between funding and first benefit, and how long do benefits last before the innovation is superseded? Long lags lower present values sharply.
• The counterfactual. Would the innovation have arrived anyway, a few years later, from someone else? If so, the benefit is the time saved, not the whole innovation.
• Attribution. What share of the benefit is credited to this organization’s funding?
• Price effects. For widely grown crops, higher output can lower prices, shifting part of the gain from producers to buyers.

None of these can be pinned down exactly. The useful response is not false precision but transparency. Report a range, state each assumption, and show which assumptions drive the result.

Why governance and evaluation are linked

Producer organizations have an unusual governance feature. Their elected directors, who are producers themselves, are often closely involved in research decisions that in most organizations would be left to management.

This is often viewed as a weakness. There is a good economic argument that it is a sensible response to the long horizon of research. Because the returns from a research portfolio arrive years after the decisions that produced them, it is close to impossible to reward managers on outcomes. Boards therefore judge managers on the quality of their effort and process, and that takes directors’ direct engagement with the decisions themselves.

The practical implication is that good process evidence matters as much as outcome evidence. A commission that records why it chose each project, against what criteria, with what expected benefits, builds the evidence that lets its board, and later its levy payers, judge whether decisions were sound, long before the outcomes are known.

A workable approach

For most producer organizations, we suggest a combination of four things.

1. A portfolio register. For every funded project, record the problem it addresses, the expected pathway to producer benefit, the likely adoption route, co-funders and the main uncertainties. This costs little and makes everything else possible.
2. Simple ex-ante scoring. Use a consistent, transparent framework when choosing projects. It might cover potential benefit, probability of success, time to adoption, fit with grower priorities and what others already fund. The point is not a single score but a comparable record of judgment.
3. Periodic ex-post evaluation of a sample. Rather than trying to value everything, conduct careful economic evaluations of a few completed investments every few years, chosen to cover different kinds of research. Report ranges and assumptions openly.
4. Plain reporting to levy payers. Report what was funded, why, what has been learned, and what selected evaluations found, including investments that did not pay off. Credibility comes from candour, not from a single impressive ratio.

The bottom line

Producer-funded research will always involve uncertain, distant returns. The organizations that handle this well do not claim to know their exact return. They document their decisions, test a sample of results rigorously, report ranges honestly, and give their directors the process evidence they need to govern well.