Overall, my guess is that, in an at least somewhat data-rich area, using data to identify the best interventions can perhaps boost your impact in the area by 3–10 times compared to picking randomly, depending on the quality of your data.
This is still a big boost, and hugely underappreciated by the world at large. However, it’s far less than I’ve heard some people in the effective altruism community claim.
In addition, there are downsides to being data-driven in this way — by insisting on a data-driven approach, you might be ruling out many of the interventions in the tail (which are often hard to measure, and so will be missing). This is why we advocate for first aiming to take a ‘hits-based’ approach, rather than a data-driven one.
“Hits-based rather than data-driven” is a pretty thought-provoking corrective to me, as I’m maybe biased by my background having worked in data-rich environments my whole career.
Tomasik’s claim (emphasis mine)
reminded me of this (again emphasis mine) from Ben Todd’s 80K article How much do solutions to social problems differ in their effectiveness? A collection of all the studies we could find
“Hits-based rather than data-driven” is a pretty thought-provoking corrective to me, as I’m maybe biased by my background having worked in data-rich environments my whole career.