Research
Can AI do social science that holds up?
Our research arm tests AI research systems against questions whose answers are already known, then points them at questions that are still open. Every study is published with a scorecard, including what failed.
What every scorecard reports
| Measure | What it tells you |
|---|---|
| Sign match | How often the system finds an effect in the same direction as the published study. |
| Interval coverage | How often its estimate lands inside the published confidence interval. |
| Run-to-run spread | How much the answer moves when the same analysis is run again. |
| Misses, explained | Each case where it disagrees with the published result, and why: our error, the original paper’s, or missing materials. |
| Comparison | The same tasks done by a single model, by our system, and by our system with an economist leading. |
Where the field stands
Reproduction
87%
of 92 instrumental-variable studies reproduced end to end by one AI workflow. Failures came from incomplete materials, not the model. arXiv 2602.16733
Scale
4,452
replication packages from five economics journals run through an open AI workflow in an NBER working paper. NBER w35782
Limits
37% vs 94%
Reproduction rates for AI-led teams and human-only teams in a randomised trial of 288 researchers. RWI
Our first studies
Our first study re-runs published economics and policy evaluations from their public replication data and reports the scorecard above. The second points the tested system at an open question in management or labour economics with newer data. Results will appear on this page as they are checked, with the code for each result.
We publish what we find, including when the system falls short. A result we cannot check does not get published.
Work with us on research
We welcome replication researchers, economists and management scholars who want to test or extend their own findings. Get in touch.