Testing a policy before it exists: what agent-based models add
When a policy is still a proposal, there is nothing to evaluate. Agent-based simulation lets decision-makers see how individual choices and market prices might respond first.
Program evaluation answers the question “did it work?” But many of the most consequential policy decisions are made before there is anything to evaluate. Should a province introduce an income support? What would a tax on sugary drinks do to diets? How would a subsidy for one crop change what farmers plant?
For questions like these, decision-makers need a way to test a policy before it exists. Agent-based models are one of the most useful tools for doing that, and one of the most misunderstood.
What an agent-based model is
Most economic forecasting summarizes behaviour with averages and aggregate relationships: if incomes rise by one percent, spending on food rises by some fraction of a percent. That is often good enough. But it can miss what policy-makers care about most.
An agent-based model takes a different route. It represents many individual decision-makers, such as households, consumers, farmers or firms. Each agent has its own characteristics, budget and rules for choosing, and the agents act within a shared environment such as a market. Aggregate outcomes are not assumed. They emerge from many individual decisions and their interactions.
Suppose some households receive additional income. In an agent-based model, each household decides what to buy given its new budget, its preferences and current prices. If many households shift spending at once, the model can capture what that does to prices, and what those price changes in turn mean for households that received nothing.
What this adds
Differences between people. The average effect of a policy can hide large differences. An income support may change little for most households and a great deal for the poorest. Agent-based models keep that variation visible, so the results can show who gains, who loses and by how much.
Feedback through markets. When a policy changes many people’s behaviour at once, markets respond. Some of the intended benefit may be absorbed by price changes. Some may spill over to people the policy was not aimed at. Models built on fixed averages usually miss this.
Thresholds and tipping points. Small changes can produce large effects when behaviour depends on what others do, or when people sit near a threshold. Agent-based models can reveal those non-linear responses.
Scenario comparison. Because the model represents the mechanism, not just the historical pattern, it can test options that have never been tried: different payment levels, different eligibility rules, combinations of policies.
What it does not do
Agent-based models are powerful and easy to misuse. Three cautions matter most.
They are only as good as their behavioural rules. If the rules agents follow do not reflect how people really choose, the results will be precise and wrong. Rules should be grounded in data, such as household expenditure surveys, observed price responses and published estimates, and each should be stated explicitly.
They need calibration and validation. A credible model reproduces known patterns before it is trusted to predict new ones. Does it match current spending shares across income groups? Does it reproduce responses to past price changes?
They produce ranges, not answers. Simulation results depend on assumptions that are uncertain. Good practice is to run each scenario many times under varying assumptions and report how far the results move.
Researchers have developed standards to keep these models transparent. The best known is the ODD protocol (Overview, Design concepts and Details), which sets out how to document a model so that others can understand, scrutinize and reproduce it. Any model used to inform public decisions should be documented to that standard.
When to use one
An agent-based model earns its cost when:
- the policy does not yet exist, so there is nothing to evaluate directly;
- differences between people matter to the decision;
- many people’s responses could shift prices or other shared conditions; and
- several options need to be compared on a consistent basis.
When the question is simply the average effect of a modest change, a simpler model is usually better: cheaper, faster and easier to explain.
The bottom line
Simulation does not replace evaluation; it comes before it. A well-built agent-based model lets decision-makers rehearse a policy, see how individual choices and market outcomes might respond, and identify which assumptions the decision hinges on. Once the policy is in place, a good evaluation can test whether the model was right.