Evidence-based policy needs theory

This imaginary scenario is based on an example from my paper with Baljinder Virk, Stella Mascarenhas-Keyes and Nancy Cartwright: ‘Randomized Controlled Trials: How Can We Know “What Works”?’ 

A research group of practically-minded military engineers are trying to work out how to effectively destroy enemy fortifications with a cannon. They are going to be operating in the field in varied circumstances so they want an approach that has as much general validity as possible. They understand the basic premise of pointing and firing the cannon in the direction of the fortifications. But they find that the cannon ball often fails to hit their targets. They have some idea that varying the vertical angle of the cannon seems to make a difference. So they decide to test fire the cannon in many different cases.

As rigorous empiricists, the research group runs many trial shots with the cannon raised, and also many control shots with the cannon in its ‘treatment as usual’ lower position. They find that raising the cannon often matters. In several of these trials, they find that raising the cannon produces a statistically significant increase in the number of balls that destroy the fortifications. Occasionally, they find the opposite: the control balls perform better than the treatment balls. Sometimes they find that both groups work, or don’t work, about the same. The results are inconsistent, but on average they find that raised cannons hit fortifications a little more often.

A physicist approaches the research group and explains that rather than just trying to vary the height the cannon is pointed in various contexts, she can estimate much more precisely where the cannon should be aimed using the principle of compound motion with some adjustment for wind and air resistance. All the research group need to do is specify the distance to the target and she can produce a trajectory that will hit it. The problem with the physicist’s explanation is that it includes reference to abstract concepts like parabolas, and trigonometric functions like sine and cosine. The research group want to know what works. Her theory does not say whether you should raise or lower the angle of the cannon as a matter of policy. The actual decision depends on the context. They want an answer about what to do, and they would prefer not to get caught up testing physics theories about ultimately unobservable entities while discovering the answer.

Eventually the research group write up their findings, concluding that firing the cannon pointed with a higher angle can be an effective ‘intervention’ but that whether it does or not depends a great deal on particular contexts. So they suggest that artillery officers will have to bear that in mind when trying to knock down fortifications in the field; but that they should definitely consider raising the cannon if they aren’t hitting the target. In the appendix, they mention the controversial theory of compound motion as a possible explanation for the wide variation in the effectiveness of the treatment effect that should, perhaps, be explored in future studies.

This is an uncharitable caricature of contemporary evidence-based policy (for a more aggressive one see ‘Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials’). Metallurgy has well-understood, repeatedly confirmed theories that command consensus among scientists and engineers. The military have no problem learning and applying this theory. Social policy, by contrast, has no theories that come close to that level of consistency. Given the lack of theoretical consensus, it might seem more reasonable to test out practical interventions instead and try to generalize from empirical discoveries. The point of this example is that without theory empirical researchers struggle to make any serious progress even with comparatively simple problems. The fact that theorizing is difficult or controversial in a particular domain does not make it any less essential a part of the research enterprise.


Also relevant: Dylan Wiliam’s quip from this video (around 9:25): ‘a statistician knows that someone with one foot in a bucket of freezing water and the other foot in a bucket of boiling water is not, on average, comfortable.’

Pete Boettke’s discussion of economic theory as an essential lens through which one looks to make the world clearer.

Some Friday links

Actually, Curbing Uber Won’t Relieve Heavy Traffic (Liya Palagashvili, Professor at SUNY, in the New York Times)

CoinFund Is In The Heart Of The Blockchain Revolution (Alisa Cohn’s profile of Jake Brukhman and Alex Felix in Forbes)

The fatal conceit: the hubris of India’s planners (Shruti Rajagopalan, Professor at SUNY, in LiveMint)

Sovereign Entrepreneurship (Alex Salter , Professor at TTU at the Center for Innovative Governance Research)

The promise and peril of blockchain distributed governance


I was very fortunate to learn that my essay ‘Markets for rules‘ has won the Mont Pelerin Society’s 2018 Hayek essay competition for young scholars (one of the perks of academia is being defined as young well until your 30s). I am now looking forward to presenting at MPS’s famous conference, originally organised by F.A. Hayek to build the post-war intellectual case for liberalism.

The essay is an attempt to explain the governance possibilities of blockchain technology through the lens of new institutional economics and more specifically private governance. Blockchains allow people to develop rules that can then enforced autonomously by the participants that use them without further central direction. This could allow communities to rely more on common rules and less on formal coercive authorities to achieve widespread social cooperation. I am cautiously optimistic about the technology (it could also turn into a dystopian nightmare) though not any particular currently existing blockchain.

Here is the abstract: Classical liberals seek the paradoxical: government powerful enough to protect individuals from preying off each other, but limited enough to prevent it becoming a fierce predator itself. The emergence of blockchain technology heralds a potential revolution in our collective capacity to implement limited government. Blockchains offer a more secure and transparent way of implementing rules while permitting individual choice between rulesets that can co-exist at the same time and place. What this could ultimately mean is that a great deal of what we have traditionally conceived to be governance might be disintermediated from the territorially defined monopolistic coercive authorities that classically define states.

Battling Time and Ignorance: Mario Rizzo at 70

Last week my friend and colleague Mario Rizzo, a scholar central to the revival of  contemporary Austrian economics, turned 70. This occasion prompted a spontaneous outpouring of praise for his work, as well as messages of gratitude for his support of students and fellow academics over his decades as an intrepid professor with his home firmly at NYU. They are collected over at ThinkMarkets. Jeffrey Tucker has written an excellent summary of Mario’s intellectual contributions at the American Institute for Economic Research. Below is a segment of my birthday message:

In my home, the United Kingdom, classical liberal thought has until recently been virtually unheard within much of academia. As a student and think-tank researcher ravenous for liberal approaches to public policy, I gorged on Mario’s blog posts from ThinkMarkets. Together with Marginal Revolution and Cafe Hayek, ThinkMarkets was a critical lifeline for me facing an intellectual world dominated by various visions of authoritarianism and only slightly more benign variants of paternalism.

Thanks to Mario’s selfless contributions to the revival of Austrian economics, that intellectual world is changing, even in the UK. His co-founding of the Society for the Development of Austrian Economics and hosting the Program on the Foundations of the Market Economy at NYU has provided support and inspiration for countless young scholars.

I am very fortunate to be among that multitude.



Paul Romer, the World Bank and Angus Deaton’s critique of effective altruism


Last week Paul Romer crashed out of his position as Chief Economist at the World Bank. He had already been isolated from the rest of the World Bank’s researchers for criticizing the reliability of their data. It seems there were several bones of contention, including the accusation that Chile’s current social democratic government falsified data contributing to some of its development indicators. Romer’s allergic reaction to the World Bank’s internal research processes has wider implications for how we think about policy research in international NGOs.

Continue reading

In memory of Christie Davies: defender of the right to joke


As we approach the end of 2017, I remember this year’s sad passing of Christie Davies. Davies was a rare academic beast: a classical liberal sociologist. Despite representing a minority perspective in his discipline, he was able to thrive and leave a mark that will continue to influence scholars for generations.

Continue reading

Make neo-Nazis flop off Broadway: public choice and Tina Fey’s sheetcaking


A week ago a white supremacist rally in Charlottesville protesting the taking down of Confederate Memorial statues turned fatally violent. Other protests were due to take place this weekend in multiple U.S. cities, including New York (now postponed). How should citizens and public authorities deal with this upsurge in violent neo-Nazi protest? I am with Tina Fey on this one: don’t show up, have some cake, and encourage the NYPD to prevent violence.

Some on the left have tried opportunistically and mistakenly to associate Virginian school public choice scholarship with the far-right. This is a sadly missed opportunity because James Buchanan’s theory of club goods helps explain how far-right street protests emerge and suggest how authorities might best subdue them. I draw on John Meadowcroft’s and Elizabeth Morrow’s analysis of the far-right English Defence League (EDL).

Continue reading