Which papers published in the last decade were most influential for your thinking?
This question is inspired by today’s Future Perfect newsletter (signup link), in which Dylan Matthews wrote:
We’re barely two months from the end of the 2010s, and that has meant a lot of end-of-decade best-of lists on everything from movies to songs to albums to TV shows. And, at least for me, it’s meant a lot of arguments with friends over whether, say, Yeezus actually holds up, or if The Master or Phantom Thread is the better Paul Thomas Anderson movie, or if The Good Place is better than Parks and Recreation.
So I started wondering what a list of the papers — in economics, political science, sociology, psychology, and philosophy — that most influenced me over the 2010s would look like. Unsurprisingly, it looked like a list of ideas that have influenced my writing in Future Perfect profoundly.
I should say that this is a small fraction of the research that’s influenced me greatly this past decade, and if you’re an academic reading this and I’ve left you out, I mean no disrespect at all! But here are five papers that have really changed how I think about the world in the 2010s (and keep an eye out for an expanded list on the site in the coming weeks!).
“Cluelessness” (2016) by Hilary Greaves
The choices we make have unpredictable consequences that ripple out for centuries or millennia, by affecting life and death. This is a very technical paper (this podcast presents a more accessible version), but Greaves does a great job of explaining cases where this kind of cluelessness is fine (where we can just make our best guess as to which action will work out best) and in which cases it’s really, really troubling.
“Free Distribution or Cost-Sharing?” (2010) by Jessica Cohen and Pascaline Dupas
I’m cheating slightly with this one; Cohen and Dupas’s article appeared in working paper form before being officially published in 2010. It uses a randomized experiment to show that giving away anti-malaria bednets for free dramatically increases their usage relative to charging a small, nominal fee.
This implies that charities like Against Malaria Foundation that facilitate the direct distribution of bednets can have huge positive effects. I’ve given thousands of dollars to AMF due in no small part to this paper.
“Using the Results from Rigorous Multisite Evaluations to Inform Local Policy Decisions”(2019) by Larry Orr, Robert Olsen, Stephen Bell, Ian Schmid, Azim Shivji, and Elizabeth Stuart
The Cohen-Dupas paper is in some ways the best possible case for randomized trials being valuable. Here’s the best countercase I’ve seen.
Focusing on education, this team of researchers tries to use average results of education policies, as measured by big randomized trials held in different locations, to predict the results in individual locations. They find that this doesn’t work very well at all: you can’t just take average results and expect that the same effect will hold in your specific case. It’s a challenging result for evidence-based policy and one I’m still grappling with.
“The Coalition Merchants” (2012) by Hans Noel
If public opinion doesn’t determine the future of public policy, what does? Here, Noel tells a compelling story that places “coalition merchants” — party activists, sympathetic journalists, and other ideologues — at the center, deciding “what goes with what” and what it means to be a conservative or a liberal.
He illustrates this using race relations in the 1950s and 1960s; he argues that intellectuals like William F. Buckley and groups like Americans for Democratic Action were crucial in identifying support for government services with support for civil rights, and opposition to one with opposition to the other.
“Does School Spending Matter? The New Literature on an Old Question” (2018) by C. Kirabao Jackson
We probably focus too much on individual studies and not enough on big pooled evidence reviews. In this review (ably summarized here for folks without NBER access), Jackson walks through 13 recent papers, many coauthored by Jackson himself, that use highly rigorous near-random methods to measure the influence of money on school outcomes.
It’s a very basic question — does pouring more money into public schools improve outcomes? — and the answer, Jackson finds in the research base, is yes. It’s a good model for reviewing an evidence base, and it’s a paper that’s genuinely changed my mind on the topic. I previously thought per-student funding didn’t matter much; I now think it matters a great deal.
Matthews restricted himself to papers in “economics, political science, sociology, psychology, and philosophy”, but I’d be interested in papers from any domain!
The following list is highly biased towards EA authors. That’s not to say that non-EA authors haven’t done a lot of important work. It just means that I haven’t read it. I’m only including articles that haven’t been mentioned by others so far.
—
In philosophy:
Several of Nick Bostrom’s papers are insightful. I’m not going to discuss them one-by-one because it would take too long. :-) Even though I don’t agree with all of his arguments, they have nonetheless influenced my thinking.
“On the Overwhelming Importance of Shaping the Long-Term Future” (2013) by Nicholas Beckstead
This PhD thesis presents philosophical arguments in favor of working to improve the far-future. Beckstead discusses different ways of doing so (existential risk reduction, trajectory changes, ripple effects of short-run altruism), and responds to objections. The thesis includes a section on population ethics that I still use as a reference.
“The Importance of Wild-Animal Suffering” (2015) by Brian Tomasik
(Note: The first version of this article was written in July 2009, but only formally published as a paper in 2015.)
This was the first article I read that made me seriously consider reducing wild animal suffering as a cause worthy of significant attention. It was highly influential on me personally. Although I admit it was not the first article to make the general argument that humans should reduce wild-animal suffering, Tomasik goes into much more depth and includes many more details and crucial considerations. Previous work was mainly philosophical in nature. In contrast, Tomasik is largely responsible for the existence of actual organizations like Wild Animal Initiative and Animal Ethics which are actively trying to address wild-animal suffering.
“Dissolving the Fermi Paradox” (2018) by Anders Sandberg, Eric Drexler, Toby Ord
(Note: I’m not sure whether this article was ever actually formally published.)
This article shows that previous work on the Drake equation / Fermi paradox relied on a fundamental mathematical error. Namely, people would take the Drake equation, plug in estimates for each variable, and multiply them together. The authors show that when the calculation is done in a more complicated but also statistically accurate way, the lack of evidence of aliens is no longer such a surprise.
In economics:
“Are Ideas Getting Harder to Find?” (2017) by Bloom et al.
“In many growth models, economic growth arises from people creating ideas, and the long-run growth rate is the product of two terms: the effective number of researchers and their research productivity. We present a wide range of evidence from various industries, products, and firms showing that research effort is rising substantially while research productivity is declining sharply. A good example is Moore’s Law. The number of researchers required today to achieve the famous doubling every two years of the density of computer chips is more than 18 times larger than the number required in the early 1970s. Across a broad range of case studies at various levels of (dis)aggregation, we find that ideas — and in particular the exponential growth they imply — are getting harder and harder to find. Exponential growth results from the large increases in research effort that offset its declining productivity.”
In CS:
“Deep Learning: A Critical Appraisal” (2018) by Gary Marcus
“Although deep learning has historical roots going back decades, neither the term “deep learning” nor the approach was popular just over five years ago, when the field was reignited by papers such as Krizhevsky, Sutskever and Hinton’s now classic (2012) deep network model of Imagenet. What has the field discovered in the five subsequent years? Against a background of considerable progress in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, I present ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence.”
I talked about it before several times, but the biggest one is:
The Possibility of an Ongoing Moral Catastrophe by Evan G. Williams, which I summarized here.
Other than that, in philosophy mostly stuff by Bostrom:
The Unilateralist’s Curse
Information Hazards
(Also flagging Will’s work on moral uncertainty, though it’s unclear to me that his PhD thesis is the best presentation)
In CS:
Adversarial Examples are Not Bugs, They Are Features by Ilyas et.al. (makes clear something I suspected for a while about that topic)
World Models by Ha and Schmidhuber
(Those two papers are far from the most influential ML papers in the last decade! But I usually learn ML from video lectures/blog posts/talking to people rather than papers)
(Probably also various AI Safety stuff, though no specific paper comes to mind).
Designing Data-Intensive Applications quoted a ton of papers (that I did not read).
In Economics:
The academic textbook Compassion by the Pound.
Poor Economics (which won the 2019 Nobel Prize!)
Meta*:
Comment on ‘The aestivation hypothesis for resolving Fermi’s paradox’
Does suffering dominate enjoyment in the animal kingdom?
*(the research/arguments weren’t directly decision-relevant for me, but the fact that they overturned something a lot of EAs believed to be true were a useful meta-update)
The degree that EA thought relies on cutting-edge* research in economics, philosophy, etc, from the last 10 years is kinda surprising if you think about it.
It’s kinda weird that not just Superintelligence but also Poor Economics, Compassion by the Pound, information hazards, unilateralist’s curse, and other things we just kinda assume to be “in the water supply” rely mostly on arguments or research that’s not even a decade old!
*the less polite way to put it is “likely to be overturned” :P
Dylan Matthews’s answer, excerpted from the Future Perfect newsletter
This question is inspired by today’s Future Perfect newsletter (signup link), in which Dylan Matthews wrote:
In 2015, we learned that Google ads are unintentionally sexist: https://www.degruyter.com/downloadpdf/j/popets.2015.1.issue-1/popets-2015-0007/popets-2015-0007.xml
And in 2019, we learned why: https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2018.3093
Can you expand on how this influenced you?
It basically represents my transition from thinking that algorithms were basically fair and fine, to thinking they’re biased because people are biased and so bias is baked in eg through bad data, to realising the are a very wide variety of ways that algorithms can unintentionally discriminate.
They’re not a particularly EA-related pair of papers, but they are very interesting.
I nominate Raj Chetty’s Who Becomes an Inventor in America? The Importance of Exposure to Innovation, which builds on his and his collaborators’ impressive other work using administrative data to estimate intergenerational economic mobility.
Chetty’s recent work is methodologically ahead of the curve, and I hope to see many more economists using large-scale administrative data to address the big questions. But the paper I’ve nominated—the “Lost Einsteins” paper—is exceptionally interesting, and I think that within a few years it will start to be seen as really important.
This is, first, because it very palpably demonstrates that concerns about inequality and economic efficiency and long-run growth are inextricably linked. If you accept endogenous growth theory as a plausible account, then the Lost Einsteins paper suggests (actually, states explicitly) that various kinds of inequality can slow innovation and therefore growth.
Second, I think that this is a fairly EA-relevant paper. It’s clear that individual inventors or small groups of innovators (Haber/Bosch, Borlaug, Tesla, Robert Noyce) can alter the course of history in a meaningful way. It’s impossible to estimate the lost social value of the lost Einsteins, but I think it’s plausible to suggest that it could be significant.