The limits of RCTs in international development

This is the clearest thinking I’ve ever encountered on this topic. From an international development perspective, but could apply to RCTs for any social intervention, really. Knowing stuff is hard!

https://​​medium.com/​​@timothyogden/​​experimental-conversations-angus-deaton-b2f768dffd57
There is this sort of belief in magic, that RCTs are attributed with properties that they do not possess.
Causality can change locally too. Even if you’ve uncovered a causal effect that doesn’t mean that causality will work that way somewhere else. It’s not just the size of the effect.
...
For instance if you go back to the 70s and 80s and you read what was written then, people thought quite hard about how you take the result from one experiment and how it would apply somewhere else. I see much too little of that in the development literature today.
MDRC wrestled with that problem of finding mechanisms from the very beginning but they never resolved it. They thought that by going into the details they could find mechanisms that would generalize or transport and they never managed to do that. You can’t do that with RCTs. You’ve got to combine them with theory and observational data so you’re right back where you were.
...
Each study has to be considered on its own. RCTs are fine, but they are just one of the techniques in the armory that one would use to try to discover things. Gold standard thinking is magical thinking.