On the face of it, it seems like researching and writing about “mainstream” topics is net positive value for EAs for the reasons you describe, although not obviously an optimal use of time relative to other competing opportunities for EAs. I’ve tried to work out in broad strokes how effective it might be to move money within putatively less-effective causes, and it seems to me like (for instance) the right research, done by the right person or group, really could make a meaningful difference in one of these areas.
Items 2.2 and 2.3 (in your summary) are, to me, simultaneously the riskiest and most compelling propositions to me. Could EAs really do a better job finding the “right answers” than there are to be found in existing work? I take “neglectedness” in the ITN framework to be a heuristic that serves mainly to forestall hubris in this regard: we should think twice before assuming we know better than the experts, as we’re quite likely to be wrong.
But I think there is still reason to suspect that there is value to be captured in mainstream causes. Here are a few reasons I think this might be the case.
“Outcome orientation” and a cost-benefit mindset are surprisingly rare, even in fields that are nominally outcomes-focused. This horse has already been beaten to death, but the mistakes, groupthink, and general confusion in many corners of epidemiology and public health during the pandemic suggests that consequences are less salient in these fields than I would have expected beforehand. Alex Tabarrok, a non-epidemiologist, seems to have gotten most things right well before the relevant domain experts simply by thinking in consequentialist terms. Zeynep Tufekci, Nate Silver, and Emily Oster are in similar positions.
Fields have their own idiosyncratic concerns and debates that eat up a lot of time and energy, IMO to the detriment of overall effectiveness. My (limited) experience in education research and tech in the developed world led me to conclude that the goals of the field are unclear and ill-defined (Are we maximizing graduation rates? College matriculation? Test scores? Are we maximizing anything at all?). Significant amounts of energy are taken up by debates and concerns about data privacy, teacher well-being and satisfaction, and other issues that are extremely important but which, ultimately, are not directly related to the (broadly defined) goals of the field. The drivers behind philanthropic funding seem, to me, to be highly undertheorized.
I think philanthropic money in the education sector should probably go to the developing world, but it’s not obvious to me that developed-world experts are squeezing out all the potential value that they could. Whether the scale of that potential value is large enough to justify improving the sector, or whether such improvements are tractable, are different questions.
There are systematic biases within disciplines, even when those fields or disciplines are full of smart, even outcomes-focused people. Though not really a cause area, David Shor has persuasively argued that Democratic political operatives are ideological at the cost of being effective. My sense is that this is also true to some degree in education.
There are fields where the research quality is just really low. The historical punching bag for this is obviously social psychology, which has been in the process of attempting to improve for a decade now. I think the experience of the replication crisis—which is ongoing—should cause us to update away from thinking that just because lots of people are working on a topic, that means that there is no marginal value to additional research. I think the marginal value can be high, especially for EAs, who are constitutionally hyper-aware of the pitfalls of bad research, have high standards of rigor, and are often quantitatively sophisticated. EAs are also relatively insistent on clarity, the lack of which seems to be a main obstacle to identifying bad research.
Thanks for this comment. I think I essentially agree with all your specific points, though I get the impression that you’re more optimistic about trying to get “better answers to mainstream questions” often being the best use of an EA’s time. That said:
this is just based mainly on something like a “vibe” from your comment (not specific statements)
my own views are fairly tentative anyway
mostly I think people also need to consider specifics of their situation, rather than strongly assuming either that it’s pretty much always a good idea to try to get “better answers” on mainstream questions or that it’s pretty much never a good idea to try that
One minor thing I’d push back on is “especially for EAs, who are constitutionally hyper-aware of the pitfalls of bad research, have high standards of rigor, and are often quantitatively sophisticated.” I think these things are true on average, but “constitutionally” is a bit too strong, and there is also a fair amount of bad research by EAs, low standards of rigour among EAs, and other problems. And I think it’s importnat that we remember that (though not in an over-the-top or self-flagellating way, and not with a sort of false modesty that would guide our behaviour poorly).
To clarify, I’m not sure this is likely to be the best use of any individual EA’s time, but I think it can still be true that it’s potentially a good use of community resources, if intelligently directed.
I agree that perhaps “constitutionally” is too strong—what I mean is that EAs tend (generally) to have an interest in / awareness of these broadly meta-scientific topics.
In general, the argument I would make would be for greater attention to the possibility that mainstream causes deserve attention and more meta-level arguments for this case (like your post).
On the face of it, it seems like researching and writing about “mainstream” topics is net positive value for EAs for the reasons you describe, although not obviously an optimal use of time relative to other competing opportunities for EAs. I’ve tried to work out in broad strokes how effective it might be to move money within putatively less-effective causes, and it seems to me like (for instance) the right research, done by the right person or group, really could make a meaningful difference in one of these areas.
Items 2.2 and 2.3 (in your summary) are, to me, simultaneously the riskiest and most compelling propositions to me. Could EAs really do a better job finding the “right answers” than there are to be found in existing work? I take “neglectedness” in the ITN framework to be a heuristic that serves mainly to forestall hubris in this regard: we should think twice before assuming we know better than the experts, as we’re quite likely to be wrong.
But I think there is still reason to suspect that there is value to be captured in mainstream causes. Here are a few reasons I think this might be the case.
“Outcome orientation” and a cost-benefit mindset are surprisingly rare, even in fields that are nominally outcomes-focused. This horse has already been beaten to death, but the mistakes, groupthink, and general confusion in many corners of epidemiology and public health during the pandemic suggests that consequences are less salient in these fields than I would have expected beforehand. Alex Tabarrok, a non-epidemiologist, seems to have gotten most things right well before the relevant domain experts simply by thinking in consequentialist terms. Zeynep Tufekci, Nate Silver, and Emily Oster are in similar positions.
Fields have their own idiosyncratic concerns and debates that eat up a lot of time and energy, IMO to the detriment of overall effectiveness. My (limited) experience in education research and tech in the developed world led me to conclude that the goals of the field are unclear and ill-defined (Are we maximizing graduation rates? College matriculation? Test scores? Are we maximizing anything at all?). Significant amounts of energy are taken up by debates and concerns about data privacy, teacher well-being and satisfaction, and other issues that are extremely important but which, ultimately, are not directly related to the (broadly defined) goals of the field. The drivers behind philanthropic funding seem, to me, to be highly undertheorized.
I think philanthropic money in the education sector should probably go to the developing world, but it’s not obvious to me that developed-world experts are squeezing out all the potential value that they could. Whether the scale of that potential value is large enough to justify improving the sector, or whether such improvements are tractable, are different questions.
There are systematic biases within disciplines, even when those fields or disciplines are full of smart, even outcomes-focused people. Though not really a cause area, David Shor has persuasively argued that Democratic political operatives are ideological at the cost of being effective. My sense is that this is also true to some degree in education.
There are fields where the research quality is just really low. The historical punching bag for this is obviously social psychology, which has been in the process of attempting to improve for a decade now. I think the experience of the replication crisis—which is ongoing—should cause us to update away from thinking that just because lots of people are working on a topic, that means that there is no marginal value to additional research. I think the marginal value can be high, especially for EAs, who are constitutionally hyper-aware of the pitfalls of bad research, have high standards of rigor, and are often quantitatively sophisticated. EAs are also relatively insistent on clarity, the lack of which seems to be a main obstacle to identifying bad research.
Thanks for this comment. I think I essentially agree with all your specific points, though I get the impression that you’re more optimistic about trying to get “better answers to mainstream questions” often being the best use of an EA’s time. That said:
this is just based mainly on something like a “vibe” from your comment (not specific statements)
my own views are fairly tentative anyway
mostly I think people also need to consider specifics of their situation, rather than strongly assuming either that it’s pretty much always a good idea to try to get “better answers” on mainstream questions or that it’s pretty much never a good idea to try that
One minor thing I’d push back on is “especially for EAs, who are constitutionally hyper-aware of the pitfalls of bad research, have high standards of rigor, and are often quantitatively sophisticated.” I think these things are true on average, but “constitutionally” is a bit too strong, and there is also a fair amount of bad research by EAs, low standards of rigour among EAs, and other problems. And I think it’s importnat that we remember that (though not in an over-the-top or self-flagellating way, and not with a sort of false modesty that would guide our behaviour poorly).
To clarify, I’m not sure this is likely to be the best use of any individual EA’s time, but I think it can still be true that it’s potentially a good use of community resources, if intelligently directed.
I agree that perhaps “constitutionally” is too strong—what I mean is that EAs tend (generally) to have an interest in / awareness of these broadly meta-scientific topics.
In general, the argument I would make would be for greater attention to the possibility that mainstream causes deserve attention and more meta-level arguments for this case (like your post).