Thank you again for your compassionate commitment to helping others, Marie! Thank you for applying your intelligence, genuine kindness, drive, and open-mindedness in the service of inspiring and leading Hi-Med, and thank you for giving your team so much support and autonomy. I’m extremely grateful to have been on your team and learning from and with you all this time. (I know I and others said similar things on Slack, but I think it really can’t be emphasised enough!) Wherever you work, they’ll be lucky you’re on their team!
Clare_Diane
Thanks for your comments, Kestrel, and your response, David (and apologies for not replying at the time). Agree that psychodynamic theory may have some relevance to the concepts we’re talking about, and agree that the SWAP looks interesting!
Describing the SWAP wouldn’t quite have fit with the goals of this piece, since it wasn’t used in the large studies we cited here, and since it focuses on making clinical diagnoses (whereas our focus is on specific traits of concern to the long-term future—we give more details about this distinction in our appendices). But it definitely seems like the SWAP would have relevance in some contexts, and like David mentioned, there’s already a National Security Edition. Your idea to develop more scalable versions of such a tool sounds fascinating, Kestrel. Thank you so much again for your engagement with our piece and for the helpful ideas here!
Agree that this post is confusing in parts and that Altman isn’t EA-aligned. (There were also some other points in the original post that I disagreed with.)
But the issue of “not calling a spade a spade” does seem to apply, at least in SBF’s case. Even now, after his many unethical decisions were discussed at length in court, some people (e.g., both the host and guest in this conversation) are still very hesitant to label SBF’s personality traits.
This doesn’t need to be about soul searching or self-flagellation—I think it can (at times) be very difficult to recognize when someone has low levels of empathy. But sometimes (both in one’s personal life and in organizations) it’s helpful to notice when someone’s personality places them at higher risk of harmful behavior.
(This comment is basically just voicing agreement with points raised in Ryan’s and David’s comments above.)
One of the things that stood out to me about the episode was the argument[1] that working on good governance and working on reducing the influence of dangerous actors are mutually exclusive strategies, and that the former is much more tractable and important than the latter.
Most “good governance” research to date also seems to focus on system-level interventions,[2] while interventions aimed at reducing the impacts of individuals are very neglected, at least according to this review of nonprofit scandals:
It is notable that all the preventive tactics that have been studied and championed—audits, governance practices, internal controls—are aimed at the organizational level. It makes sense to focus on this level, as it is the level that managers have most control over. Prevention can also be implemented at the individual and sectoral levels. Training of staff, job-level checks and balances, and staff evaluations could all help prevent violations with individual-level causes. Sector-level regulation and oversight is becoming common in many countries. We, therefore, encourage future research on preventive measures to take a multilevel perspective, or at least consider the neglected sectoral and individual levels.
Six years before the review quoted above, this article called for psychopathy screening for public leadership positions (which would have represented one potential approach to interventions at the “individual level,” to adopt the terminology of the review quoted above).[3]
This leads me to wonder: what are the most compelling reasons for the lack of research (so far) on interventions to reduce the impact of dangerous actors, and which (if any) of these reasons provide strong arguments against doing at least some research in this neglected area? I think there are lots of possible answers here,[4] but none of them seem strong enough to justify the relative lack of research on this area so far (relative to the scale of the problem).- ^
Here’s a quote from the episode (courtesy of Wei Dai’s transcript) demonstrating this claim:
[Will MacAskill:] There’s really two ways of looking at things: you might ask…is this a bad person—are we focusing on the character? Or you might ask…what oversight, what feedback mechanisms, what incentives does this person face? And yeah, one thing I’ve really taken away from this is to place even more weight than I did before on just the importance of governance, where that means the, you know, importance of people acting with oversight, with the feedback mechanisms and you know, with incentives to incentivize kind of good rather than bad behavior…
I agree that all these aspects of governance are important, but disagree that working on these things would entirely protect an organization from the negative impacts of malevolent actors.
- ^
To be clear, I am glad people are working on system-level solutions to low integrity and otherwise harmful behaviors, but I think it would be helpful if it wasn’t the *only* class of interventions that had substantial amounts of resources directed towards them.
- ^
Interestingly, one of the real-life cases Boddy refers to in support of his argument is the Enron scandal, a case which was also covered in the book Will MacAskill was talking about, Why They Do It.
- ^
Here are some of the reasons I’ve already thought about (listed roughly in order from most to least convincing to me as a reason to be pessimistic about this approach to risk reduction): potential lack of tractability; lower levels of social and political acceptability/feasibility; lack of existing evidence as to what methods work, to what extent, and in which contexts; and perhaps a perception that the problem (of dangerous actors) is small in scale. I’d be interested to know which (if any) of these reasons are the most important, and if there are other considerations I’m overlooking. Overall, despite these reasons against working on it, I still think this area is worth investigating to a greater extent than it has been to date.
- ^
Hi, happy to speak to the methodological points here.
Thanks for sharing the link and suggestion. We agree that understanding how much we can trust the data is crucial for interpreting our results, so thank you for engaging with this critically.
We didn’t measure individual reaction times for questions so using RT modelling isn’t an option. Modelling carelessness in other ways (e.g., modelling it as a latent tendency) would be fascinating, but I don’t endorse the assumptions we’d need to model carelessness as a latent variable. (I like how Rohrer & Paulewicz 2025 point out that latent variable modelling requires making some strong assumptions.) So even though we could try to model carelessness, I currently don’t think we should do so.
The headline results were designed such that it would be very unlikely for participants selecting at random to meet our definition of “consistent and concerning” endorsers. In the main piece, the focus is on who agreed with the hell question, AND selected “Forever” for the duration question (the last of 11 options), AND selected 1% or more for the proportion question. The supplementary materials additionally report those who endorse BOTH the “endorses system” question and “would create” question, AND selected “Forever” for the duration question (the last of 11 options), AND selected 1% or more for the proportion question. Partly due to the design of these “headline” results variables, the proportion of the sample meeting our definition of “consistent and concerning” turned out to be robust to the removal of all attention checks (i.e., the inclusion of everyone who didn’t drop out before the questions of interest, with no filtering).
Regarding the other things in the list you linked to from that LLM chat, though, we did do most of those things. For example, we included unobtrusive checks and multiple different quality measures, not just attention checks—I’d be interested in your thoughts on the checks outlined in our supplementary folder. And importantly, for our headline results, we did sensitivity analyses and shared the results (including confidence intervals) in our supplementary materials folder.
(Also, just to address the point about the N varying between questions—that’s because different numbers of participants completed some questions; slightly fewer completed the duration and hell questions because we had tested different wordings for both questions early in the study, before they were replaced with new versions that were used for the rest of the study.)
Would be happy to answer follow-up questions too. Thanks!