I appreciate the effort you’ve put into this, and your analysis makes sense based on publicly available data and your worldview. However, many policy organizations are working on initiatives that haven’t been/can’t be publicly discussed, which might lead you to make some incorrect conclusions. For example, I’m glad Malo clarified MIRI does indeed work with policymakers in this comment thread.
Tone is difficult to convey online, so I want to clarify I’m saying the next statement gently: I think if you do this kind of report—that a ton of people are reading and taking seriously—you have some responsibility to send your notes to the mentioned organizations for fact checking before you post.
I also want to note: the EA community does not have good intuitions around how politics works or what kind of information is net productive for policy organizations to share. The solution is not to blindly defer to people who say they understand politics, but I am worried that our community norms actively work against us in this space. Consider checking some of your criticisms of policy orgs with a person who has worked for the US Government; getting an insider’s perspective on what makes sense/seems suspicious could be useful.
I think it’s reasonable for a donor to decide where to donate based on publicly available data and to share their conclusions with others. Michael disclosed the scope and limitations of his analysis, and referred to other funders having made different decisions. The implied reader of the post is pretty sophisticated and would be expected to know that these funders may have access to information on initiatives that haven’t been/can’t be publicly discussed.
While I appreciate why orgs may not want to release public information on all initiatives, the unavoidable consequence of that decision is that small/medium donors are not in a position to consider those initiatives when deciding whether to donate. Moreover, I think Open Phil et al. are capable of adjusting their own donation patterns in consideration of the fact that some orgs’ ability to fundraise from the broader EA & AIS communities is impaired by their need for unusually-low-for-EA levels of public transparency.
“Run posts by orgs” is ordinarily a good practice, at least where you are conducting a deep dive into some issue on which one might expect significant information to be disclosed. Here, it seems reasonable to assume that orgs will have made a conscious decision about what general information they want to share with would-be small/medium donors. So there isn’t much reason to expect that an inquiry (along with notice that the author is planning to publish on-Forum) would yield material additional information.[1] Against that, the costs of reaching out to ~28 orgs is not insignificant and would be a significant barrier to people authoring this kind of post. The post doesn’t seem to rely on significant non-public information, accuse anyone of misconduct, or have other characteristics that would make advance notice and comment particularly valuable.
Balancing all of that, I think the opportunity for orgs to respond to the post in comments was and is adequate here.
In contrast, when one is writing a deep dive on a narrower issue, the odds seem considerably higher that the organization has material information that isn’t published because of opportunity costs, lack of any reason to think there would be public interest, etc. But I’d expect most orgs’ basic fundraising ask to have been at least moderately deliberate.
Here, it seems reasonable to assume that orgs will have made a conscious decision about what general information they want to share with would-be small/medium donors. So there isn’t much reason to expect that an inquiry (along with notice that the author is planning to publish on-Forum) would yield material additional information.[1]
This seems quite false to me. Far from “isn’t much reason”, we already know that such an inquiry would have yielded additional information, because Malo almost definitely would have corrected Michael’s material misunderstanding about MIRI’s work.
Additionally, my experience of writing similar posts is that there are often many material small facts that small orgs haven’t disclosed but would happily explain in an email. Even basic facts like “what publications have you produced this year” would be impossible to determine otherwise. Small orgs just aren’t that strategic about what they disclose!
you have some responsibility to send your notes to the mentioned organizations for fact checking before you post
I spent a good amount of time thinking about whether I should do this and I read various arguments for and against it, and I concluded that I don’t have that responsibility. There are clear advantages to running posts by orgs, and clear disadvantages, and I decided that the disadvantages outweighted the advantages in this case.
I did it in my head and I haven’t tried to put it into words so take this with a grain of salt.
Pros:
Orgs get time to correct misconceptions.
(Actually I think that’s pretty much the only pro but it’s a big pro.)
Cons:
It takes a lot longer. I reviewed 28 orgs; it would take me a long time to send 28 emails and communicate with potentially 28 people. (There’s a good chance I would have procrastinated on this and not gotten my post out until next year, which means I would have had to make my 2024 donations without publishing this writeup first.)
Communicating beforehand would make me overly concerned about being nice to the people I talked to, and might prevent me from saying harsh but true things because I don’t want to feel mean.
Orgs can still respond to the post after it’s published, it’s not as if it’s impossible for them to respond at all.
Here are some relevant EA Forum/LW posts (the comments are relevant too):
It takes a lot longer. I reviewed 28 orgs; it would take me a long time to send 28 emails and communicate with potentially 28 people.
This is quite a scalable activity. When I used to do this, I had a spreadsheet to keep track, generated emails from a template, and had very little back and forth—orgs just saw a draft of their section, had a few days to comment, and then I might or might not take their feedback into account.
IIRC didn’t you somewhat frequently remove sections if the org objected because you didn’t have enough time to engage with them? (which I think was reasonably costly)
I remember removing an org entirely because they complained, though in that case they claimed they didn’t have enough time to engage with me (rather than the opposite). It’s also possible there are other cases I have forgotten. To your point, I have no objections to Michael’s “make me overly concerned about being nice” argument which I do think is true.
Thanks for clarifying! Really appreciate you engaging with this.
Re: It takes a lot longer. It seems like it takes a lot of time for you to monitor the comments on this post and update your top level post in response. The cost of doing that after you post publicly, instead of before, is that people who read your initial post are a lot less likely to read the updated one. So I don’t think you save a massive amount of time here, and you increase the chance other people become misinformed about orgs.
Re: Orgs can still respond to the post after it’s published. Some orgs aren’t posting some information publicly on purpose, but they will tell you things in confidence if you ask privately. If you publicly blast them on one of these topics, they will not publicly respond. I know EAs can be allergic to these kind of dynamics, but politics is qualitatively different than ML research; managing relationships with multiple stakeholders with opposing views is delicate, and there are a bunch of bad actors working against AI safety in DC. You might be surprised by what kind of information is very dangerous for orgs to discuss publicly.
I’m just curious, have you discussed any of your concerns with somebody who has worked in policy for the US Government?
I appreciate the effort you’ve put into this, and your analysis makes sense based on publicly available data and your worldview. However, many policy organizations are working on initiatives that haven’t been/can’t be publicly discussed, which might lead you to make some incorrect conclusions. For example, I’m glad Malo clarified MIRI does indeed work with policymakers in this comment thread.
Tone is difficult to convey online, so I want to clarify I’m saying the next statement gently: I think if you do this kind of report—that a ton of people are reading and taking seriously—you have some responsibility to send your notes to the mentioned organizations for fact checking before you post.
I also want to note: the EA community does not have good intuitions around how politics works or what kind of information is net productive for policy organizations to share. The solution is not to blindly defer to people who say they understand politics, but I am worried that our community norms actively work against us in this space. Consider checking some of your criticisms of policy orgs with a person who has worked for the US Government; getting an insider’s perspective on what makes sense/seems suspicious could be useful.
I think it’s reasonable for a donor to decide where to donate based on publicly available data and to share their conclusions with others. Michael disclosed the scope and limitations of his analysis, and referred to other funders having made different decisions. The implied reader of the post is pretty sophisticated and would be expected to know that these funders may have access to information on initiatives that haven’t been/can’t be publicly discussed.
While I appreciate why orgs may not want to release public information on all initiatives, the unavoidable consequence of that decision is that small/medium donors are not in a position to consider those initiatives when deciding whether to donate. Moreover, I think Open Phil et al. are capable of adjusting their own donation patterns in consideration of the fact that some orgs’ ability to fundraise from the broader EA & AIS communities is impaired by their need for unusually-low-for-EA levels of public transparency.
“Run posts by orgs” is ordinarily a good practice, at least where you are conducting a deep dive into some issue on which one might expect significant information to be disclosed. Here, it seems reasonable to assume that orgs will have made a conscious decision about what general information they want to share with would-be small/medium donors. So there isn’t much reason to expect that an inquiry (along with notice that the author is planning to publish on-Forum) would yield material additional information.[1] Against that, the costs of reaching out to ~28 orgs is not insignificant and would be a significant barrier to people authoring this kind of post. The post doesn’t seem to rely on significant non-public information, accuse anyone of misconduct, or have other characteristics that would make advance notice and comment particularly valuable.
Balancing all of that, I think the opportunity for orgs to respond to the post in comments was and is adequate here.
In contrast, when one is writing a deep dive on a narrower issue, the odds seem considerably higher that the organization has material information that isn’t published because of opportunity costs, lack of any reason to think there would be public interest, etc. But I’d expect most orgs’ basic fundraising ask to have been at least moderately deliberate.
This seems quite false to me. Far from “isn’t much reason”, we already know that such an inquiry would have yielded additional information, because Malo almost definitely would have corrected Michael’s material misunderstanding about MIRI’s work.
Additionally, my experience of writing similar posts is that there are often many material small facts that small orgs haven’t disclosed but would happily explain in an email. Even basic facts like “what publications have you produced this year” would be impossible to determine otherwise. Small orgs just aren’t that strategic about what they disclose!
I spent a good amount of time thinking about whether I should do this and I read various arguments for and against it, and I concluded that I don’t have that responsibility. There are clear advantages to running posts by orgs, and clear disadvantages, and I decided that the disadvantages outweighted the advantages in this case.
Thanks for being thoughtful about this! Could you clarify what your cost benefit analysis was here? I’m quite curious!
I did it in my head and I haven’t tried to put it into words so take this with a grain of salt.
Pros:
Orgs get time to correct misconceptions.
(Actually I think that’s pretty much the only pro but it’s a big pro.)
Cons:
It takes a lot longer. I reviewed 28 orgs; it would take me a long time to send 28 emails and communicate with potentially 28 people. (There’s a good chance I would have procrastinated on this and not gotten my post out until next year, which means I would have had to make my 2024 donations without publishing this writeup first.)
Communicating beforehand would make me overly concerned about being nice to the people I talked to, and might prevent me from saying harsh but true things because I don’t want to feel mean.
Orgs can still respond to the post after it’s published, it’s not as if it’s impossible for them to respond at all.
Here are some relevant EA Forum/LW posts (the comments are relevant too):
https://forum.effectivealtruism.org/posts/f77iuXmgiiFgurnBu/run-posts-by-orgs
https://www.lesswrong.com/posts/Hsix7D2rHyumLAAys/run-posts-by-orgs#comments
https://forum.effectivealtruism.org/posts/hM4atR2MawJK7jmwe/building-cooperative-epistemology-response-to-ea-has-a-lying
https://forum.effectivealtruism.org/posts/tuSQBGgnoxvsXwXJ3/criticism-is-sanctified-in-ea-but-like-any-intervention
https://forum.effectivealtruism.org/posts/QH9BGAoh2xnCdn2yS/omega-s-shortform?commentId=manyAkGcg6mgZio8t
This is quite a scalable activity. When I used to do this, I had a spreadsheet to keep track, generated emails from a template, and had very little back and forth—orgs just saw a draft of their section, had a few days to comment, and then I might or might not take their feedback into account.
IIRC didn’t you somewhat frequently remove sections if the org objected because you didn’t have enough time to engage with them? (which I think was reasonably costly)
I remember removing an org entirely because they complained, though in that case they claimed they didn’t have enough time to engage with me (rather than the opposite). It’s also possible there are other cases I have forgotten. To your point, I have no objections to Michael’s “make me overly concerned about being nice” argument which I do think is true.
Cool, I might just be remembering that one instance.
Thanks for clarifying! Really appreciate you engaging with this.
Re: It takes a lot longer. It seems like it takes a lot of time for you to monitor the comments on this post and update your top level post in response. The cost of doing that after you post publicly, instead of before, is that people who read your initial post are a lot less likely to read the updated one. So I don’t think you save a massive amount of time here, and you increase the chance other people become misinformed about orgs.
Re: Orgs can still respond to the post after it’s published. Some orgs aren’t posting some information publicly on purpose, but they will tell you things in confidence if you ask privately. If you publicly blast them on one of these topics, they will not publicly respond. I know EAs can be allergic to these kind of dynamics, but politics is qualitatively different than ML research; managing relationships with multiple stakeholders with opposing views is delicate, and there are a bunch of bad actors working against AI safety in DC. You might be surprised by what kind of information is very dangerous for orgs to discuss publicly.
I’m just curious, have you discussed any of your concerns with somebody who has worked in policy for the US Government?