Difference-in-differences, explained with one real example

Most decisions worth evaluating were never set up as experiments. A company changes prices in one region. A government raises a wage floor in one state. A charity opens a program in some towns and not others. Afterwards someone asks whether it worked.

Comparing before and after is not enough, because other things changed too. Comparing the changed group with an unchanged one is not enough either, because the two groups were different to begin with. Difference-in-differences combines the two comparisons so that each one fixes the other’s main flaw.

The idea in one sentence

Measure how much the changed group moved, measure how much a similar unchanged group moved over the same period, and treat the gap between those two movements as the effect.

The unchanged group stands in for what would have happened to the changed group without the change. That stand-in is called the counterfactual. Everything rests on how good it is.

The example that made it famous

On April 1, 1992, New Jersey raised its minimum wage from $4.25 to $5.05 an hour. Neighbouring Pennsylvania did not. Economists David Card and Alan Krueger surveyed fast-food restaurants on both sides of the border in February and March 1992, just before the increase, and again in November and December 1992 [1].

Their outcome was full-time-equivalent workers per restaurant. The averages, from their published table:

	Before	After	Change
New Jersey, wage raised	20.44	21.03	+0.59
Pennsylvania, no change	23.33	21.17	-2.16
Difference-in-differences			+2.75

Look at each comparison on its own and you get a muddle. Before and after in New Jersey alone shows a small rise, but it can’t tell you whether the whole region’s fast-food employment was moving. New Jersey against Pennsylvania after the increase shows New Jersey slightly lower, but New Jersey restaurants were smaller to begin with.

Difference-in-differences takes New Jersey’s change, +0.59, and subtracts Pennsylvania’s change, -2.16. The result is +2.75 workers per restaurant. If New Jersey had followed Pennsylvania’s path, it would have fallen to about 18.28. It ended at 21.03 instead.

The finding, that employment did not fall where the wage rose, contradicted the textbook prediction. It started a long debate. Later work using payroll records instead of phone surveys reached different conclusions [2], and recent research asks whether workers and customers moving across the border could have affected both states at once [3]. That debate is part of why the method is now taught with its assumptions spelled out.

The assumption that carries everything

Difference-in-differences assumes parallel trends. Without the change, the two groups would have moved by the same amount. They can start at different levels. Pennsylvania’s restaurants were bigger, and that’s fine. What matters is that their paths would have run in parallel.

You can never observe this directly, because the “without the change” path for the changed group never happened. But you can test whether it is plausible:

• Look at several periods before the change. If the two groups moved together for a year or more beforehand, parallel trends after is easier to believe. If they were already drifting apart, the estimate will pick up that drift.
• Ask what else changed at the same time. A new competitor, a tax change or a bad season that hit only one group breaks the comparison.
• Check for spillovers. If the change in one group pushes customers, staff or money into the other, the comparison group is no longer untouched. That is the point raised about the border restaurants.
• Try another comparison group. If the answer holds with a different reasonable control, it is more likely real.

Where it fits outside economics papers

The same logic applies whenever a change reaches some units and not others, and you have data from before and after for both.

• A retailer changes prices or opening hours in some stores first.
• A health system introduces a new care pathway in some clinics before others.
• A city or province runs a pilot in some districts.
• A company rolls out a new tool to some teams, or a campaign to some regions.
• A funder supports some organizations in one year and others the next.

In each case, the staggered rollout is the opportunity. If you plan it on purpose, by choosing who goes first and recording outcomes before the change, the later evaluation is cheaper and more credible.

Common traps

1. One period before and one after. It’s the minimum, and it leaves parallel trends untested. Collect more periods if you can.
2. Picking the comparison group after seeing the results. Choose it in advance and write down why.
3. Groups that chose to be treated. If the sites that adopted a program were already improving, their trend was not parallel to begin with.
4. Staggered rollouts handled the simple way. When units start at different times, the standard two-way regression can mix up early and late adopters and give misleading answers. Newer estimators handle this, and they should be the default for staggered designs.
5. Ignoring uncertainty. A difference of 2.75 means little without a confidence interval. With few units, and with outcomes that move together within a group, the interval is wider than a naive calculation suggests.

What to take from this

Difference-in-differences is a way to build a believable “what would have happened anyway” from data you may already have. That counterfactual is also the input most return-on-investment calculations skip. Try changing it in our ROI and payback calculator and watch how much the answer moves.

Sources

1. Card, D. and Krueger, A. B. (1994), "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania", American Economic Review 84(4), 772-793. Survey timing and wage change as described in arXiv 2203.16405, https://arxiv.org/abs/2203.16405 (accessed 2026-10-27)
2. Neumark, D. and Wascher, W. (2000), "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Comment", American Economic Review 90(5), 1362-1396.
3. "Difference-in-Differences in the Presence of Unknown Interference", arXiv 2512.21176, which re-tabulates the Card and Krueger means, https://arxiv.org/abs/2512.21176 (accessed 2026-10-27)