If this comment is more about “how could this have been foreseen”, then this comment thread may be relevant. I should note that hindsight bias means that it’s much easier to look back and assess problems as obvious and predictable ex post, when powerful investment firms and individuals who also had skin in the game also missed this.
TL;DR: 1) There were entries that were relevant (this one also touches on it briefly) 2) They were specifically mentioned 3) There were commentsrelevant to this. (notably one of these was apparently deleted because it received a lot of downvotes when initially posted) 4) There has been at least two other posts on the forum prior to the contest that engaged with this specifically
My tentative take is that these issues were in fact identified by various members of the community, but there isn’t a good way of turning identified issues into constructive actions—the status quo is we just have to trust that organisations have good systems in place for this, and that EA leaders are sufficiently careful and willing to make changes or consider them seriously, such that all the community needs to do is “raise the issue”. And I think looking at the systems within the relevant EA orgs or leadership is what investigations or accountability questions going forward should focus on—all individuals are fallible, and we should be looking at how we can build systems in place such that the community doesn’t have to just trust that people who have power and who are steering the EA movement will get it right, and that there are ways for the community to hold them accountable to their ideals or stated goals if it appears to, or risks not playing out in practice.
i.e. if there are good processes and systems in place and documentation of these processes and decisions, it’s more acceptable (because other organisations that probably have a very good due diligence process also missed it). But if there weren’t good processes, or if these decisions weren’t a careful + intentional decision, then that’s comparatively more concerning, especially in context of specific criticisms that have been raised,[1] or previous precedent. For example, I’d be especially curious about the events surrounding Ben Delo,[2] and processes that were implemented in response. I’d be curious about whether there are people in EA orgs involved in steering who keep track of potential risks and early warning signs to the EA movement, in the same way the EA community advocates for in the case of pandemics, AI, or even general ways of finding opportunities for impact. For example, SBF, who is listed as a EtG success story on 80k hours, has publicly stated he’s willing to go 5x over the Kelly bet, and described yield farming in a way that Matt Levine interpreted as a Ponzi. Again, I’m personally less interested in the object level decision (e.g. whether or not we agree with SBF’s Kelly bet comments as serious, or whether Levine’s interpretation as appropriate), but more about what the process was, how this was considered at the time with the information they had etc. I’d also be curious about the documentation of any SBF related concerns that were raised by the community, if any, and how these concerns were managed and considered (as opposed to critiquing the final outcome).
Outside of due diligence and ways to facilitate whistleblowers, decision-making processes around the steering of the EA movement is crucial as well. When decisions are made by orgs that bring clear benefits to one part of the EA community while bringing clear risks that are shared across wider parts of the EA community,[3] it would probably be of value to look at how these decisions were made and what tradeoffs were considered at the time of the decision. Going forward, thinking about how to either diversify those risks, or make decision-making more inclusive of a wider range stakeholders[4], keeping in mind the best interests of the EA movement as a whole.
(this is something I’m considering working on in a personal capacity along with the OP of this post, as well as some others—details to come, but feel free to DM me if you have any thoughts on this. It appears that CEA is also already considering this)
If this comment is about “are these red-teaming contests in fact valuable for the money and time put into it, if it misses problems like this”
I think my view here (speaking only for the red-teaming contest) is that even if this specific contest was framed in a way that it missed these classes of issues, the value of the very top submissions[5] may still have made the efforts worthwhile. The potential value of a different framing was mentioned by another panelist. If it’s the case that red-teaming contests are systematically missing this class of issues regardless of framing, then I agree that would be pretty useful to know, but I don’t have a good sense of how we would try to investigate this.
This tweet seems to have aged particularly well. Despite supportive comments from high-profile EAs on the original forum post, the author seemed disappointed that nothing came of it in that direction. Again, without getting into the object level discussion of the claims of the original paper, it’s still worth asking questions around the processes. If there was were actions planned, what did these look like? If not, was that because of a disagreement over the suggested changes, or the extent that it was an issue at all? How were these decisions made, and what was considered?
Apparently a previous EA-aligned billionaire ?donor who got rich by starting a crypto trading firm, who pleaded guilty to violating the bank secrecy act
Even before this, I had heard from a primary source in a major mainstream global health organisation that there were staff who wanted to distance themselves from EA because of misunderstandings around longtermism.
This doesn’t have to be a lengthy deliberative consensus-building project, but it should at least include internal comms across different EA stakeholders to allow discussions of risks and potential mitigation strategies.
If this comment is more about “how could this have been foreseen”, then this comment thread may be relevant. I should note that hindsight bias means that it’s much easier to look back and assess problems as obvious and predictable ex post, when powerful investment firms and individuals who also had skin in the game also missed this.
TL;DR:
1) There were entries that were relevant (this one also touches on it briefly)
2) They were specifically mentioned
3) There were comments relevant to this. (notably one of these was apparently deleted because it received a lot of downvotes when initially posted)
4) There has been at least two other posts on the forum prior to the contest that engaged with this specifically
My tentative take is that these issues were in fact identified by various members of the community, but there isn’t a good way of turning identified issues into constructive actions—the status quo is we just have to trust that organisations have good systems in place for this, and that EA leaders are sufficiently careful and willing to make changes or consider them seriously, such that all the community needs to do is “raise the issue”. And I think looking at the systems within the relevant EA orgs or leadership is what investigations or accountability questions going forward should focus on—all individuals are fallible, and we should be looking at how we can build systems in place such that the community doesn’t have to just trust that people who have power and who are steering the EA movement will get it right, and that there are ways for the community to hold them accountable to their ideals or stated goals if it appears to, or risks not playing out in practice.
i.e. if there are good processes and systems in place and documentation of these processes and decisions, it’s more acceptable (because other organisations that probably have a very good due diligence process also missed it). But if there weren’t good processes, or if these decisions weren’t a careful + intentional decision, then that’s comparatively more concerning, especially in context of specific criticisms that have been raised,[1] or previous precedent. For example, I’d be especially curious about the events surrounding Ben Delo,[2] and processes that were implemented in response. I’d be curious about whether there are people in EA orgs involved in steering who keep track of potential risks and early warning signs to the EA movement, in the same way the EA community advocates for in the case of pandemics, AI, or even general ways of finding opportunities for impact. For example, SBF, who is listed as a EtG success story on 80k hours, has publicly stated he’s willing to go 5x over the Kelly bet, and described yield farming in a way that Matt Levine interpreted as a Ponzi. Again, I’m personally less interested in the object level decision (e.g. whether or not we agree with SBF’s Kelly bet comments as serious, or whether Levine’s interpretation as appropriate), but more about what the process was, how this was considered at the time with the information they had etc. I’d also be curious about the documentation of any SBF related concerns that were raised by the community, if any, and how these concerns were managed and considered (as opposed to critiquing the final outcome).
Outside of due diligence and ways to facilitate whistleblowers, decision-making processes around the steering of the EA movement is crucial as well. When decisions are made by orgs that bring clear benefits to one part of the EA community while bringing clear risks that are shared across wider parts of the EA community,[3] it would probably be of value to look at how these decisions were made and what tradeoffs were considered at the time of the decision. Going forward, thinking about how to either diversify those risks, or make decision-making more inclusive of a wider range stakeholders[4], keeping in mind the best interests of the EA movement as a whole.
(this is something I’m considering working on in a personal capacity along with the OP of this post, as well as some others—details to come, but feel free to DM me if you have any thoughts on this. It appears that CEA is also already considering this)
If this comment is about “are these red-teaming contests in fact valuable for the money and time put into it, if it misses problems like this”
I think my view here (speaking only for the red-teaming contest) is that even if this specific contest was framed in a way that it missed these classes of issues, the value of the very top submissions[5] may still have made the efforts worthwhile. The potential value of a different framing was mentioned by another panelist. If it’s the case that red-teaming contests are systematically missing this class of issues regardless of framing, then I agree that would be pretty useful to know, but I don’t have a good sense of how we would try to investigate this.
This tweet seems to have aged particularly well. Despite supportive comments from high-profile EAs on the original forum post, the author seemed disappointed that nothing came of it in that direction. Again, without getting into the object level discussion of the claims of the original paper, it’s still worth asking questions around the processes. If there was were actions planned, what did these look like? If not, was that because of a disagreement over the suggested changes, or the extent that it was an issue at all? How were these decisions made, and what was considered?
Apparently a previous EA-aligned billionaire ?donor who got rich by starting a crypto trading firm, who pleaded guilty to violating the bank secrecy act
Even before this, I had heard from a primary source in a major mainstream global health organisation that there were staff who wanted to distance themselves from EA because of misunderstandings around longtermism.
This doesn’t have to be a lengthy deliberative consensus-building project, but it should at least include internal comms across different EA stakeholders to allow discussions of risks and potential mitigation strategies.
e.g. A critical review of GiveWell’s 2022 cost-effectiveness model, Methods for improving uncertainty analysis in EA cost-effectiveness models, and
Biological Anchors external review