Can you assure me that Rethink’s researchers are independent?
Yes. Rethink Priorities is devoted to editorial independence. None of our contracts with our clients include editorial veto power (except that we obviously cannot publish confidential information) and we wouldn’t accept these sorts of contracts.
I think our reliance on individual large clients is important, but overstated. Our single largest client made up ~47% of our income in 2023 and we’re on track for this to land somewhere between 40% and 60% in 2024. This means in the unlikely event that we were to lose this large client completely, it would certainly suck, but some version of RP would still survive. Our leadership is committed to taking this path if necessary.
This is also why we are proactively working hard to diversify our client base and why we are soliciting individual funding. Funding from you and others would help us further safeguard our editorial independence. We’re very lucky and grateful for our existing individual donors that collectively donate a significant portion of our expenses and preserve our independence.
Luckily, I am not aware of any instance where we wanted to publish work that any of our institutional clients did not want to publish. We try to work with clients that value our independence and aim to promote truth-seeking even if it may go against the initial views of our client. We do not work with clients where they only want to hear from us a confirmation of their pre-existing beliefs—we have been asked to do something like this (though not in these exact words) many times actually and have always turned down this work.
There have sometimes been disagreements between individual staff and RP management about what to publish that did not involve anything about upsetting our clients, and in these cases we’ve always allowed the staff to still publish their work in their personal capacity. Rethink Priorities does not censor staff or prevent them from publishing.
If you have reason to believe that Rethink Priorities researchers are not independent, please let me know so I can correct any issues and reassure relevant staff.
When I was asked to resign from RP, one of the reasons given was that I wrote the sentence “I don’t think that EAs should fund many WAW researchers since I don’t think that WAW is a very promising cause area” in an email to OpenPhil, after OpenPhil asked for my opinion on a WAW (Wild Animal Welfare) grant. I was told that this is not okay because OpenPhil is one of the main funders of RP’s WAW work. That did not make me feel very independent. Though perhaps that was the only instance in the four years I worked at RP.
Because of this instance, I was also concerned when I saw that RP is doing cause prioritization work because I was afraid that you would hesitate to publish stuff that threatens RP funding, and would more willingly publish stuff that would increase RP funding. I haven’t read any of your cause prio research though, so I can’t comment on whether I saw any of that.
Uh, didn’t expect people to notice this old thread. Just to make the situation sound less dramatic, I should’ve said that I had informed RP that I wanted to quit RP in three months. Then a situation arose after which the person I was supposed to work closely with for those three months didn’t want to work with me. I think that was the main reason why I was asked to resign right away instead of after those three months. It makes sense.
I’m very sorry that you felt that way – that wasn’t our intention. We aren’t going to get into the details of your resignation in public, but as you mention in your follow up comment, neither this incident, nor our disagreement over WAW views were the reason for your resignation.
As you recall, you did publish your views on wild animal welfare publicly. Because RP leadership was not convinced by the reasoning in your piece, we rejected your request to publish it under the RP byline as an RP article representative of an RP position. This decision was based on the work itself; OP was not at all a factor involved in this decision. Moreover, we made no attempt to censor your views or prevent them from being shared (indeed I personally encouraged you to publish the piece if you wanted).
To add some additional context without getting into the details of this specific scenario, we can share some general principles about how we approach donor engagement.
We have ~40 researchers working across a variety of areas. Many of them have views about what we should do and what research should be done. By no means do we expect our staff to publicly or privately agree with the views of leadership, let alone with our donors. Still, we have a donor engagement policy outlining how we like to handle communication with donors.
One relevant dimension is that we think that if one of our researchers, especially while representing RP, is sending something to a funder that has a plausible implication that one of the main funders of a department should seriously reduce or stop funding that department, we should know they are planning to do so before they do so, and roughly what is being said so that we can be prepared. While we don’t want to be seen as censoring our researchers, we do think it’s important to approach these sorts of things with clarity and tact.
There are also times when we think it is important for RP to speak with a unified voice to our most important donors and represent a broader, coordinated consensus on what we think. Or, if minority views of one of our researchers that RP leadership disagrees with are to be considered, this needs to be properly contextualized and coordinated so that we can interact with our donors with full knowledge of what is being shared with them (for example, we don’t want to accidentally convey that the view of a single member of staff represents RP’s overall position).
With regard to cause prioritization, funders don’t filter or factor into our views in any way. They haven’t been involved in any way with setting what we do or don’t say in our cause prioritization work. Further, as far as I’m aware, OP hasn’t adopted the kind of approach we’ve suggested on any of our major cause prioritization on moral weight or as seen in the CURVE sequence.
What bothers me is that if I said that I was excited about funding WAW research, no one would have said anything. I was free to say that. But to say that I’m not excited, I have to go through all these hurdles. This introduces a bias because a lot of the time researchers won’t want to go through hurdles and opinions that would indirectly threaten RP’s funding won’t be shared. Hence, funders would have a distorted view of researchers’ opinions.
Put yourself into my shoes. OpenPhil sends an email to multiple people asking for opinions on a WAW grant. What I did was that I wrote a list of pros and cons about funding that grant, recommended funding it, and pressed “send”. It took like 30 minutes. Later OpenPhil said that it helped them to make the decision. Score! I felt energized. I probably had more impact in those 30 minutes than I had in three months of writing about aquatic noise.
Now imagine I knew that I had to inform the management about saying that I’m not excited about WAW. My manager was new to RP, he would’ve needed to escalate to directors. Writing my manager’s manager’s manager a message like “Can I write this thing that threatens the funding of our organisation?” is awkward. Also, these sorts of complex conversations can sometimes take a lot of time for both me and the upper management who are very busy. So I might not have done it. And I’m a very disagreeable employee who knows directors personally, so other researchers are probably less likely to do stuff like that. I also didn’t want to write OpenPhil an email that speaks frankly about the pros of funding the grant but not the cons. So it might have been easier to just ignore OpenPhil’s email and focus on finishing the aquatic noise report.
Maybe all of this is very particular to this situation. Situations like this didn’t arise when I worked at RP very often. I spent most of my time researching stuff where my conclusions had no impact on RP’s funding. But if RP grows, conflicts of interest are likely to arise more often. And such concerns might apply more often to cause-prioritisation research. If a cause prioritisation researcher concluded that say AI safety research is more important than all other causes, would they be able to talk about that freely to funders, even though it would threaten RP’s funding? If not, that’s a problem.
Great points, Saulius! I think it would be pretty valuable to have a question post asking about how organisations are handling situations such as yours (you may be interested in doing it; if you or other person do not, I may do it myself). I would say employees/contractors should be free, and ideally encouraged, to share their takes with funders and publicly, more of less regardless of what are the consequences for their organisation, as long as they are not sharing confidential information.
Thanks Vasco. Personally, I won’t do it. Actually, I pushed this issue here way further than I intended to and I don’t want to talk about it more. I’m afraid that this might be one of those things that might seem like a big deal in theory but is rarely relevant in practice because reality is usually more complicated. And it’s the sort of topic that can be discussed for too long, distracting busy EA executives from their actual work. Many people seem to have read this discussion, so arguments on both sides will likely be considered at RP and some other EA orgs. That is enough for me. I particularly hope it will be discussed by RP’s cause prioritization team.
If you do decide to pose a question, I suggest focusing on whether employees at EA organizations feel any direct or indirect pressure to conform to specific opinions due to organizational policies and dynamics, or even due to direct pressure from management. Or just don’t feel free to speak their minds on certain issues related to their job. People should have an option to answer anonymously. Basically to find out if my case was an isolated incident or not.
One relevant dimension is that we think that if one of our researchers, especially while representing RP, is sending something to a funder that has a plausible implication that one of the main funders of a department should seriously reduce or stop funding that department, we should know they are planning to do so before they do so, and roughly what is being said so that we can be prepared. While we don’t want to be seen as censoring our researchers, we do think it’s important to approach these sorts of things with clarity and tact.
There are also times when we think it is important for RP to speak with a unified voice to our most important donors and represent a broader, coordinated consensus on what we think. Or, if minority views of one of our researchers that RP leadership disagrees with are to be considered, this needs to be properly contextualized and coordinated so that we can interact with our donors with full knowledge of what is being shared with them (for example, we don’t want to accidentally convey that the view of a single member of staff represents RP’s overall position).
FWIW, this sounds to me like a very substantial tax on sharing information with funders, and I am surprised by this policy.
As someone who is sometimes acting in a funder capacity I wasn’t expecting my information to be filtered this way (and also, I feel very capable of not interpreting the statements of a single researcher as being non-representative of a whole organization, I expect other funders are similarly capable of doing that).
Indeed, I think I was expecting the opposite, which is that I can ask researchers openly and freely about their takes on RP programs, and they have no obligation to tell you that they spoke to me at all. Even just an obligation to inform you after the fact seems like it would frequently introduce substantial chilling effects. I certainly don’t think my employees are required to tell me if they give a reference or give an assessment of any Lightcone project, and would consider an obligation to inform me if they do as quite harmful for healthy information flow (and if someone considers funding Lightcone, or wants to work with us in some capacity, I encourage you to get references from my employees or other people we’ve worked with, including offering privacy or confidentiality however you see fit about the information you gain this way).
Hey, thanks for the feedback. I do think reasonable people can disagree about this policy and it entails an important trade-off.
To understand a bit about the other side of the trade-off, I would ask that you consider that we at RP are roughly an order of magnitude bigger than Lightcone Infrastructure and we need policies to be able to work well with ~70 people that I agree make no sense with ~7 (relatively independent, relatively senior) people.
Could you say more about the other side of the tradeoff? As in, what’s the affirmative case for having this policy? So far in this thread the main reason has been “we don’t want people to get the impression that X statement by a junior researcher represents RP’s views”. I see a very simple alternative as “if individuals make statements that don’t represent RP’s views they should always make that clear up front”. So is there more reason to have this policy?
Fair, but the flip side of that is that it’s considerably less likely that a sophisticated donor would somehow misunderstand a junior researcher’s clearly-expressed-as-personal views as expressing the institutional view of a 70-person org.
Hmm, my sense is that the tradeoffs here mostly go in the opposite direction. It seems like the costs scale with the number of people, and it’s clear that very large organizations (200+) don’t really have a chance of maintaining such a policy anyways, as the cost of enforcement grows with the number of members (as well as the costs on information flow), while the benefits don’t obviously scale in the same way.
It seems to me more reasonable for smaller organizations to have a policy like this, and worse the larger the organization is (in terms of how unhappy to be with the leadership of the organization for instituting such a policy).
I think most larger orgs attempt to have even less information leave than this. So you’re statement seems wrong. Many large organisations have good boundaries in terms of information—apple is very good at keeping upcoming product releases quiet.
I think larger organizations are obviously worse than this, though I agree that some succeed nevertheless. I was mostly just making an argument about relative cost (and think that unless you put a lot of effort into it, at 200+ it usually becomes prohibitively expensive, though it of course depends on the exact policy and). See Google and OpenAI for organizations that I think are more representative here (and are more what I was thinking about).
Naah I think I still disagree. I guess the median large consultancy or legal firm is much more likely to go after you for sharing stuff than than the median small business. Because they have the resources and organisational capital to do so, and because their hiring allows them to find people who probably won’t mind and because they capture more of the downside and lose less to upside.
I’m not endorsing this but it’s what I would expect from Rethink, OpenPhil, FTX, Manifold, 80k, Charity Entreprenurship, Longview, CEA, Lightcone, Miri. And looking at those orgs, It’s what, Lightcone and Manifold that aren’t normal to secretive in terms of internal information. Maybe I could be convinced to give MIRI/Longivew a pass because their secrecy might be for non-institutional reasons but “organisations become less willing for random individuals to speak their true views about internal processes as they get larger/more powerful” seems a reasonable rule of thumb, inside and outside EA.
Peter, I’m not sure if it is worth your time to share, but I wonder if there are some additional policies RP has which are obvious to you but are not obvious to outsiders, and this is what is causing outsiders to be surprised.
E.g. perhaps you have a formal policy about how funders can get feedback from your employees without going through RP leadership which replaces the informal conversations that funders have with employees at other organizations. And this policy would alleviate some of the concerns people like Habryka have; we just aren’t aware of it because we aren’t familiar enough with RP.
Would you consider whipping this up into a publicly posted policy? I’m sure Lightcone employees don’t need a formal policy, but I think it would be desirable for orgs to have legible published policies on this as a way to flag which ones are / are not insisting their staff filter their opinions through management. Donors/funders will be able to weigh opinions from organizational employees accordingly.
I feel weird making a top-level post about it (and feel like I would need to polish it a bunch before I feel comfortable making it an official policy). Do feel free to link to my comments here.
If many people would find this helpful, I would be happy to do it, but it does seem like a bunch of work so wouldn’t want to do it based on just the support expressed so far.
[Edit: as per Saulius’ reply below I was perhaps to critical here, especially regarding the WAW post, and it sounds like Saulius thinks that was manged relatively well by RP senior staff]
I found this reply made me less confident in Rethink’s ability to address publication bias. Some things that triggered my ‘hmmm not so sure about this’ sense were:
The reply did not directly address the claims in Saulius’s comment. E.g. “I’m sorry you feel that way” not “I’m sorry”. No acknowledgement that if, as Saulius claimed, a senior staff told him that it was wrong to have expressed his views to OpenPhil (when asked by OpenPhil for his views), that this might have been a mistake by that staff. I guess I was expecting more of a ‘huh that sounds bad, let us look into it’ style response than a ‘nothing to see here, all is great in Rethink land’ style response.
The story about Saulius’ WAW post. Judging from Marcus’ comment It sounds like Rethink stopped Saulius from posting a WAW post (within a work context) and it also looks like there was a potential conflict of interest here for senior staff as posting could affect funding. Marcus says that RP senior staff were only deciding based on the quality of the post. Now I notice the post itself (published a few months after Saulius left Rethink) seems to have significantly more upvotes than almost any of Rethink’s posts (and lots of positive feedback). I recognise upvotes are not a great measure of research quality but it does make me worry that about the possibility that this post was actually of sufficient quality and the conflict of interest may have biased senior Rethink staff to stop the post being published as a Rethink post.
The stated donor engagement principles seem problematic. Eg: “there are also times when we think it is important for RP to speak with a unified voice to our most important donors” is exactly the kind of reason I would use if I was censoring staff’s interaction with donors. It is not that “unified organisational voice” policies are wrong, just that without additional safeguards they have a risk of being abused, of facilitating the kinds of conflict of interest driven actions under discussion here. Also as Saulius mentions such policies can also be one-sided where staff are welcome to say anything that aligns with a certain worldview but need to get sign-off to disagree with that worldview, another source of bias.
I really really do like Rethink. And I do not think that this is a huge problem, or enough of a problem to stop donors giving to Rethink in most cases. But I would still be interested in seeing additional evidence of Rethink addressing this risk of bias.
It sounds like Rethink stopped Saulius from posting a WAW post (within a work context) and it also looks like there was a potential conflict of interest here for senior staff as posting could affect funding
It is true that I wasn’t allowed to publish some of my WAW work on behalf of RP. Note that this WAW work includes not only the short summary Why I No Longer Prioritize Wild Animal Welfare (which got a lot of upvotes) but also three longer articles that it summarises (this, this, and this). Some of these do not threaten RP’s funding in any way. That said, I was allowed to work on these articles in my work time which should count for something.
Let me give a bit of context because it might not make sense without it. My project was finding the best WAW intervention. I struggled with it a lot. Instead of doing what I was supposed to, I started writing about why I was struggling, which eventually turned into all those posts. I asked my manager (who was quite new to RP) if I could continue working on those posts and publish them on behalf of RP. My manager allowed it. Then Marcus read some of my drafts and gave detailed useful feedback. He said that while he was happy that I looked into this stuff, these articles couldn’t be published on behalf of RP. As I remember it, the reason was that they were basically opinion pieces. He wanted RP to only post stuff that is closer to an academic publication. I asked why but I don’t want to share his answer publicly in case what he told me wasn’t public. I disagreed with his position but it’s the sort of thing that reasonable people can disagree on. We both thought that it wasn’t worth trying to bring my articles to the degree of polish (and perhaps rigour) that would meet RP’s publishing standards but that they were still worth publishing. Marcus said that I’m free to finish my posts in my work time, which was very kind of him. Note that this was financially disadvantageous for RP. I felt uncomfortable spending more time on non-RP stuff in my work time so I took an unpaid leave to finish my posts, but it was my own idea and Marcus explicitly told me that I don’t need to do that.
Thank you Saulius. Very helpful to hear. This sounds like a really positive story of good management of a difficult situation. Well done to Marcus.
If I read between the lines a bit I get the impression that maybe more junior (be that less competent or just newer to the org) managers at Rethink with less confidence in their actions not rocking the Rethink<->funder relationship were perhaps more likely to put unwelcome pressure on researchers about what to publish. Just a hypothesis, so might be wrong. But also the kind of thing good internal policies, good onboarding, good senior example setting, or just discussions of this topic, can all help with.
No, sorry, I wasn’t saying that. My manager was Jacob Peacock, he was a great manager. He didn’t put any unwelcome pressure and wasn’t the one who talked to me about the email to OpenPhil. He said that I can publish my WAW articles on behalf of RP but then Marcus disagreed.
I get RP’s concerns that an individual researcher’s opinions not come across as RP’s organizational position. However, equal care needs to be given to the flipside—that the donor does not get the impression that a response fully reflects the researcher’s opinion when it has been materially affected by the donor-communication policy.
I’m not suggesting that management interference is inappropriate . . . but the donor has the right to know when it is occurring. Otherwise, if I were a donor/funder, I would have to assume that all communications from RP researchers reflected management influence, and would adjust each response in an attempt to filter out any management influence and “recover” the researcher’s original intent. For example, I would not update much on the omission of any statement that would have potential negative institutional effect on RP.
As an analogy, imagine that every statement by a professor in an academic department had to be filtered through a process to make sure there was no material adverse effect on the university’s interests. I would view any professorial statement coming out of that university with a jaundiced eye, and I think most people would do. In contrast, as a practicing lawyer, everyone assumes that my statements at work are not really truth-seeking and are heavily influenced by my client’s institutional interests. I submit that it is important for RP’s effectiveness that donors/funders see RP researchers as closer to academics than the functional equivalent of RP’s lawyers.
All of this is fairly process-heavy, but that may be OK given the representation that these sorts of situations are infrequent.
(1) The donor-engagement policy should be publicly posted, so that donors/funders are aware when communications they receive may have been impacted by it. As @Habryka explains, donors/funders may not expect “information to be filtered this way” and could affirmatively expect an absence of such filtering.
(2) An employee should be able to respond to a request from a major donor simply with: “Based on RP’s donor-communication policy, I am forwarding your communication to management for their response” without any negative repercussions from RP. If this happens, the employee should not be expected to sign any response unless they agree that it 100% fairly reflects their thinking on the topic. This practice would mitigate against employees feeling pressured to present representations of the RP party as their own views. It should also signal to the donor that the response they receive may have been influenced by management consideration of RP’s institutional interests.
(3) Ordinarily, an employee should be allowed to include an appendix setting forth their own views to the donor, without any negative consequences from RP. While management should have a few days’ advance notice of the contents of this appendix, they should not have the power to change it. In rare cases, it may be appropriate for management to disapprove the appendix entirely—but in those cases, the policy should commit to a prominent note that the employee’s request to file an appendix was denied.
Probably not the right place to discuss it, but at some point I’d be interested in both the object level question of whether marginal wild animal welfare research should be funded and the more meta question of what RP WAW employees and ex-employees believe on this issue.
I obviously have no inside knowledge of the situation @saulius describes here, but I do think it touches on a potential concern of mine that RP could become (or could already be) too influential in the cause prioritization domain. This is not based on anything about RP other than its size: I think it’s almost inevitable that any organization that exists (or could plausibly exist) in an RP-like space will have its own private institutional interests that could reasonably be expected to influence its outputs. Allowing those interests to affect outputs seems rather undesirable.
Three ways to manage this problem occur to me. I tentatively think that decentralization is best; no one organization should control too much of the “cause prioritization thinking/research market.”[1] In a sufficiently decentralized system, there is another similarly-capable organization that can correct errors influenced by private institutional interests. Next in line, I’d suggest externally enforceable guarantees, such as a freedom-of-expression policy with third-party enforcement. I’d place promises that are not externally enforceable in a distinctly lower category; I wouldn’t put much stock in them to the extent that the promisor was the main/only game in town.
I’m ignoring Open Phil here, because I don’t think the Open Phil hegemony is a fundamentally tractable problem. In contrast, RP is seeking funds from Forum users, and so the question of keeping its private institutional interests from potentially affecting community views on cause prioritization is fairly on the table.
Yes. Rethink Priorities is devoted to editorial independence. None of our contracts with our clients include editorial veto power (except that we obviously cannot publish confidential information) and we wouldn’t accept these sorts of contracts.
I think our reliance on individual large clients is important, but overstated. Our single largest client made up ~47% of our income in 2023 and we’re on track for this to land somewhere between 40% and 60% in 2024. This means in the unlikely event that we were to lose this large client completely, it would certainly suck, but some version of RP would still survive. Our leadership is committed to taking this path if necessary.
This is also why we are proactively working hard to diversify our client base and why we are soliciting individual funding. Funding from you and others would help us further safeguard our editorial independence. We’re very lucky and grateful for our existing individual donors that collectively donate a significant portion of our expenses and preserve our independence.
Luckily, I am not aware of any instance where we wanted to publish work that any of our institutional clients did not want to publish. We try to work with clients that value our independence and aim to promote truth-seeking even if it may go against the initial views of our client. We do not work with clients where they only want to hear from us a confirmation of their pre-existing beliefs—we have been asked to do something like this (though not in these exact words) many times actually and have always turned down this work.
There have sometimes been disagreements between individual staff and RP management about what to publish that did not involve anything about upsetting our clients, and in these cases we’ve always allowed the staff to still publish their work in their personal capacity. Rethink Priorities does not censor staff or prevent them from publishing.
If you have reason to believe that Rethink Priorities researchers are not independent, please let me know so I can correct any issues and reassure relevant staff.
When I was asked to resign from RP, one of the reasons given was that I wrote the sentence “I don’t think that EAs should fund many WAW researchers since I don’t think that WAW is a very promising cause area” in an email to OpenPhil, after OpenPhil asked for my opinion on a WAW (Wild Animal Welfare) grant. I was told that this is not okay because OpenPhil is one of the main funders of RP’s WAW work. That did not make me feel very independent. Though perhaps that was the only instance in the four years I worked at RP.
Because of this instance, I was also concerned when I saw that RP is doing cause prioritization work because I was afraid that you would hesitate to publish stuff that threatens RP funding, and would more willingly publish stuff that would increase RP funding. I haven’t read any of your cause prio research though, so I can’t comment on whether I saw any of that.
EDIT: I should’ve said that this was not the main reason I was asked to resign and that I had said that I would quit in three months before this happened.
Uh, didn’t expect people to notice this old thread. Just to make the situation sound less dramatic, I should’ve said that I had informed RP that I wanted to quit RP in three months. Then a situation arose after which the person I was supposed to work closely with for those three months didn’t want to work with me. I think that was the main reason why I was asked to resign right away instead of after those three months. It makes sense.
Hey Saulius,
I’m very sorry that you felt that way – that wasn’t our intention. We aren’t going to get into the details of your resignation in public, but as you mention in your follow up comment, neither this incident, nor our disagreement over WAW views were the reason for your resignation.
As you recall, you did publish your views on wild animal welfare publicly. Because RP leadership was not convinced by the reasoning in your piece, we rejected your request to publish it under the RP byline as an RP article representative of an RP position. This decision was based on the work itself; OP was not at all a factor involved in this decision. Moreover, we made no attempt to censor your views or prevent them from being shared (indeed I personally encouraged you to publish the piece if you wanted).
To add some additional context without getting into the details of this specific scenario, we can share some general principles about how we approach donor engagement.
We have ~40 researchers working across a variety of areas. Many of them have views about what we should do and what research should be done. By no means do we expect our staff to publicly or privately agree with the views of leadership, let alone with our donors. Still, we have a donor engagement policy outlining how we like to handle communication with donors.
One relevant dimension is that we think that if one of our researchers, especially while representing RP, is sending something to a funder that has a plausible implication that one of the main funders of a department should seriously reduce or stop funding that department, we should know they are planning to do so before they do so, and roughly what is being said so that we can be prepared. While we don’t want to be seen as censoring our researchers, we do think it’s important to approach these sorts of things with clarity and tact.
There are also times when we think it is important for RP to speak with a unified voice to our most important donors and represent a broader, coordinated consensus on what we think. Or, if minority views of one of our researchers that RP leadership disagrees with are to be considered, this needs to be properly contextualized and coordinated so that we can interact with our donors with full knowledge of what is being shared with them (for example, we don’t want to accidentally convey that the view of a single member of staff represents RP’s overall position).
With regard to cause prioritization, funders don’t filter or factor into our views in any way. They haven’t been involved in any way with setting what we do or don’t say in our cause prioritization work. Further, as far as I’m aware, OP hasn’t adopted the kind of approach we’ve suggested on any of our major cause prioritization on moral weight or as seen in the CURVE sequence.
Thank you for your answer Marcus.
What bothers me is that if I said that I was excited about funding WAW research, no one would have said anything. I was free to say that. But to say that I’m not excited, I have to go through all these hurdles. This introduces a bias because a lot of the time researchers won’t want to go through hurdles and opinions that would indirectly threaten RP’s funding won’t be shared. Hence, funders would have a distorted view of researchers’ opinions.
Put yourself into my shoes. OpenPhil sends an email to multiple people asking for opinions on a WAW grant. What I did was that I wrote a list of pros and cons about funding that grant, recommended funding it, and pressed “send”. It took like 30 minutes. Later OpenPhil said that it helped them to make the decision. Score! I felt energized. I probably had more impact in those 30 minutes than I had in three months of writing about aquatic noise.
Now imagine I knew that I had to inform the management about saying that I’m not excited about WAW. My manager was new to RP, he would’ve needed to escalate to directors. Writing my manager’s manager’s manager a message like “Can I write this thing that threatens the funding of our organisation?” is awkward. Also, these sorts of complex conversations can sometimes take a lot of time for both me and the upper management who are very busy. So I might not have done it. And I’m a very disagreeable employee who knows directors personally, so other researchers are probably less likely to do stuff like that. I also didn’t want to write OpenPhil an email that speaks frankly about the pros of funding the grant but not the cons. So it might have been easier to just ignore OpenPhil’s email and focus on finishing the aquatic noise report.
Maybe all of this is very particular to this situation. Situations like this didn’t arise when I worked at RP very often. I spent most of my time researching stuff where my conclusions had no impact on RP’s funding. But if RP grows, conflicts of interest are likely to arise more often. And such concerns might apply more often to cause-prioritisation research. If a cause prioritisation researcher concluded that say AI safety research is more important than all other causes, would they be able to talk about that freely to funders, even though it would threaten RP’s funding? If not, that’s a problem.
Great points, Saulius! I think it would be pretty valuable to have a question post asking about how organisations are handling situations such as yours (you may be interested in doing it; if you or other person do not, I may do it myself). I would say employees/contractors should be free, and ideally encouraged, to share their takes with funders and publicly, more of less regardless of what are the consequences for their organisation, as long as they are not sharing confidential information.
Thanks Vasco. Personally, I won’t do it. Actually, I pushed this issue here way further than I intended to and I don’t want to talk about it more. I’m afraid that this might be one of those things that might seem like a big deal in theory but is rarely relevant in practice because reality is usually more complicated. And it’s the sort of topic that can be discussed for too long, distracting busy EA executives from their actual work. Many people seem to have read this discussion, so arguments on both sides will likely be considered at RP and some other EA orgs. That is enough for me. I particularly hope it will be discussed by RP’s cause prioritization team.
If you do decide to pose a question, I suggest focusing on whether employees at EA organizations feel any direct or indirect pressure to conform to specific opinions due to organizational policies and dynamics, or even due to direct pressure from management. Or just don’t feel free to speak their minds on certain issues related to their job. People should have an option to answer anonymously. Basically to find out if my case was an isolated incident or not.
FWIW, this sounds to me like a very substantial tax on sharing information with funders, and I am surprised by this policy.
As someone who is sometimes acting in a funder capacity I wasn’t expecting my information to be filtered this way (and also, I feel very capable of not interpreting the statements of a single researcher as being non-representative of a whole organization, I expect other funders are similarly capable of doing that).
Indeed, I think I was expecting the opposite, which is that I can ask researchers openly and freely about their takes on RP programs, and they have no obligation to tell you that they spoke to me at all. Even just an obligation to inform you after the fact seems like it would frequently introduce substantial chilling effects. I certainly don’t think my employees are required to tell me if they give a reference or give an assessment of any Lightcone project, and would consider an obligation to inform me if they do as quite harmful for healthy information flow (and if someone considers funding Lightcone, or wants to work with us in some capacity, I encourage you to get references from my employees or other people we’ve worked with, including offering privacy or confidentiality however you see fit about the information you gain this way).
Hey, thanks for the feedback. I do think reasonable people can disagree about this policy and it entails an important trade-off.
To understand a bit about the other side of the trade-off, I would ask that you consider that we at RP are roughly an order of magnitude bigger than Lightcone Infrastructure and we need policies to be able to work well with ~70 people that I agree make no sense with ~7 (relatively independent, relatively senior) people.
Could you say more about the other side of the tradeoff? As in, what’s the affirmative case for having this policy? So far in this thread the main reason has been “we don’t want people to get the impression that X statement by a junior researcher represents RP’s views”. I see a very simple alternative as “if individuals make statements that don’t represent RP’s views they should always make that clear up front”. So is there more reason to have this policy?
Fair, but the flip side of that is that it’s considerably less likely that a sophisticated donor would somehow misunderstand a junior researcher’s clearly-expressed-as-personal views as expressing the institutional view of a 70-person org.
Hmm, my sense is that the tradeoffs here mostly go in the opposite direction. It seems like the costs scale with the number of people, and it’s clear that very large organizations (200+) don’t really have a chance of maintaining such a policy anyways, as the cost of enforcement grows with the number of members (as well as the costs on information flow), while the benefits don’t obviously scale in the same way.
It seems to me more reasonable for smaller organizations to have a policy like this, and worse the larger the organization is (in terms of how unhappy to be with the leadership of the organization for instituting such a policy).
I think most larger orgs attempt to have even less information leave than this. So you’re statement seems wrong. Many large organisations have good boundaries in terms of information—apple is very good at keeping upcoming product releases quiet.
I think Apple is very exceptional here, and it does come at great cost as many Apple employees have complained about over the past years:
I think larger organizations are obviously worse than this, though I agree that some succeed nevertheless. I was mostly just making an argument about relative cost (and think that unless you put a lot of effort into it, at 200+ it usually becomes prohibitively expensive, though it of course depends on the exact policy and). See Google and OpenAI for organizations that I think are more representative here (and are more what I was thinking about).
Naah I think I still disagree. I guess the median large consultancy or legal firm is much more likely to go after you for sharing stuff than than the median small business. Because they have the resources and organisational capital to do so, and because their hiring allows them to find people who probably won’t mind and because they capture more of the downside and lose less to upside.
I’m not endorsing this but it’s what I would expect from Rethink, OpenPhil, FTX, Manifold, 80k, Charity Entreprenurship, Longview, CEA, Lightcone, Miri. And looking at those orgs, It’s what, Lightcone and Manifold that aren’t normal to secretive in terms of internal information. Maybe I could be convinced to give MIRI/Longivew a pass because their secrecy might be for non-institutional reasons but “organisations become less willing for random individuals to speak their true views about internal processes as they get larger/more powerful” seems a reasonable rule of thumb, inside and outside EA.
Peter, I’m not sure if it is worth your time to share, but I wonder if there are some additional policies RP has which are obvious to you but are not obvious to outsiders, and this is what is causing outsiders to be surprised.
E.g. perhaps you have a formal policy about how funders can get feedback from your employees without going through RP leadership which replaces the informal conversations that funders have with employees at other organizations. And this policy would alleviate some of the concerns people like Habryka have; we just aren’t aware of it because we aren’t familiar enough with RP.
Would you consider whipping this up into a publicly posted policy? I’m sure Lightcone employees don’t need a formal policy, but I think it would be desirable for orgs to have legible published policies on this as a way to flag which ones are / are not insisting their staff filter their opinions through management. Donors/funders will be able to weigh opinions from organizational employees accordingly.
I feel weird making a top-level post about it (and feel like I would need to polish it a bunch before I feel comfortable making it an official policy). Do feel free to link to my comments here.
If many people would find this helpful, I would be happy to do it, but it does seem like a bunch of work so wouldn’t want to do it based on just the support expressed so far.
Hi Oliver,
I think you meant “interpreting” instead of “not interpreting”, given you write “non-representive” afterwards.
[Edit: as per Saulius’ reply below I was perhaps to critical here, especially regarding the WAW post, and it sounds like Saulius thinks that was manged relatively well by RP senior staff]
I found this reply made me less confident in Rethink’s ability to address publication bias. Some things that triggered my ‘hmmm not so sure about this’ sense were:
The reply did not directly address the claims in Saulius’s comment. E.g. “I’m sorry you feel that way” not “I’m sorry”. No acknowledgement that if, as Saulius claimed, a senior staff told him that it was wrong to have expressed his views to OpenPhil (when asked by OpenPhil for his views), that this might have been a mistake by that staff. I guess I was expecting more of a ‘huh that sounds bad, let us look into it’ style response than a ‘nothing to see here, all is great in Rethink land’ style response.
The story about Saulius’ WAW post. Judging from Marcus’ comment It sounds like Rethink stopped Saulius from posting a WAW post (within a work context) and it also looks like there was a potential conflict of interest here for senior staff as posting could affect funding. Marcus says that RP senior staff were only deciding based on the quality of the post. Now I notice the post itself (published a few months after Saulius left Rethink) seems to have significantly more upvotes than almost any of Rethink’s posts (and lots of positive feedback). I recognise upvotes are not a great measure of research quality but it does make me worry that about the possibility that this post was actually of sufficient quality and the conflict of interest may have biased senior Rethink staff to stop the post being published as a Rethink post.
The stated donor engagement principles seem problematic. Eg: “there are also times when we think it is important for RP to speak with a unified voice to our most important donors” is exactly the kind of reason I would use if I was censoring staff’s interaction with donors. It is not that “unified organisational voice” policies are wrong, just that without additional safeguards they have a risk of being abused, of facilitating the kinds of conflict of interest driven actions under discussion here. Also as Saulius mentions such policies can also be one-sided where staff are welcome to say anything that aligns with a certain worldview but need to get sign-off to disagree with that worldview, another source of bias.
I really really do like Rethink. And I do not think that this is a huge problem, or enough of a problem to stop donors giving to Rethink in most cases. But I would still be interested in seeing additional evidence of Rethink addressing this risk of bias.
It is true that I wasn’t allowed to publish some of my WAW work on behalf of RP. Note that this WAW work includes not only the short summary Why I No Longer Prioritize Wild Animal Welfare (which got a lot of upvotes) but also three longer articles that it summarises (this, this, and this). Some of these do not threaten RP’s funding in any way. That said, I was allowed to work on these articles in my work time which should count for something.
Let me give a bit of context because it might not make sense without it. My project was finding the best WAW intervention. I struggled with it a lot. Instead of doing what I was supposed to, I started writing about why I was struggling, which eventually turned into all those posts. I asked my manager (who was quite new to RP) if I could continue working on those posts and publish them on behalf of RP. My manager allowed it. Then Marcus read some of my drafts and gave detailed useful feedback. He said that while he was happy that I looked into this stuff, these articles couldn’t be published on behalf of RP. As I remember it, the reason was that they were basically opinion pieces. He wanted RP to only post stuff that is closer to an academic publication. I asked why but I don’t want to share his answer publicly in case what he told me wasn’t public. I disagreed with his position but it’s the sort of thing that reasonable people can disagree on. We both thought that it wasn’t worth trying to bring my articles to the degree of polish (and perhaps rigour) that would meet RP’s publishing standards but that they were still worth publishing. Marcus said that I’m free to finish my posts in my work time, which was very kind of him. Note that this was financially disadvantageous for RP. I felt uncomfortable spending more time on non-RP stuff in my work time so I took an unpaid leave to finish my posts, but it was my own idea and Marcus explicitly told me that I don’t need to do that.
Thank you Saulius. Very helpful to hear. This sounds like a really positive story of good management of a difficult situation. Well done to Marcus.
If I read between the lines a bit I get the impression that maybe more junior (be that less competent or just newer to the org) managers at Rethink with less confidence in their actions not rocking the Rethink<->funder relationship were perhaps more likely to put unwelcome pressure on researchers about what to publish. Just a hypothesis, so might be wrong. But also the kind of thing good internal policies, good onboarding, good senior example setting, or just discussions of this topic, can all help with.
No, sorry, I wasn’t saying that. My manager was Jacob Peacock, he was a great manager. He didn’t put any unwelcome pressure and wasn’t the one who talked to me about the email to OpenPhil. He said that I can publish my WAW articles on behalf of RP but then Marcus disagreed.
Thank you for the correction
I get RP’s concerns that an individual researcher’s opinions not come across as RP’s organizational position. However, equal care needs to be given to the flipside—that the donor does not get the impression that a response fully reflects the researcher’s opinion when it has been materially affected by the donor-communication policy.
I’m not suggesting that management interference is inappropriate . . . but the donor has the right to know when it is occurring. Otherwise, if I were a donor/funder, I would have to assume that all communications from RP researchers reflected management influence, and would adjust each response in an attempt to filter out any management influence and “recover” the researcher’s original intent. For example, I would not update much on the omission of any statement that would have potential negative institutional effect on RP.
As an analogy, imagine that every statement by a professor in an academic department had to be filtered through a process to make sure there was no material adverse effect on the university’s interests. I would view any professorial statement coming out of that university with a jaundiced eye, and I think most people would do. In contrast, as a practicing lawyer, everyone assumes that my statements at work are not really truth-seeking and are heavily influenced by my client’s institutional interests. I submit that it is important for RP’s effectiveness that donors/funders see RP researchers as closer to academics than the functional equivalent of RP’s lawyers.
All of this is fairly process-heavy, but that may be OK given the representation that these sorts of situations are infrequent.
(1) The donor-engagement policy should be publicly posted, so that donors/funders are aware when communications they receive may have been impacted by it. As @Habryka explains, donors/funders may not expect “information to be filtered this way” and could affirmatively expect an absence of such filtering.
(2) An employee should be able to respond to a request from a major donor simply with: “Based on RP’s donor-communication policy, I am forwarding your communication to management for their response” without any negative repercussions from RP. If this happens, the employee should not be expected to sign any response unless they agree that it 100% fairly reflects their thinking on the topic. This practice would mitigate against employees feeling pressured to present representations of the RP party as their own views. It should also signal to the donor that the response they receive may have been influenced by management consideration of RP’s institutional interests.
(3) Ordinarily, an employee should be allowed to include an appendix setting forth their own views to the donor, without any negative consequences from RP. While management should have a few days’ advance notice of the contents of this appendix, they should not have the power to change it. In rare cases, it may be appropriate for management to disapprove the appendix entirely—but in those cases, the policy should commit to a prominent note that the employee’s request to file an appendix was denied.
Probably not the right place to discuss it, but at some point I’d be interested in both the object level question of whether marginal wild animal welfare research should be funded and the more meta question of what RP WAW employees and ex-employees believe on this issue.
Was Saulius representing RP at the time? It sounds like they were asked out of band, and surely OP already had access to the RP consensus view.
OpenPhil email was sent to my RP email address, so I think that means that I was representing RP
I obviously have no inside knowledge of the situation @saulius describes here, but I do think it touches on a potential concern of mine that RP could become (or could already be) too influential in the cause prioritization domain. This is not based on anything about RP other than its size: I think it’s almost inevitable that any organization that exists (or could plausibly exist) in an RP-like space will have its own private institutional interests that could reasonably be expected to influence its outputs. Allowing those interests to affect outputs seems rather undesirable.
Three ways to manage this problem occur to me. I tentatively think that decentralization is best; no one organization should control too much of the “cause prioritization thinking/research market.”[1] In a sufficiently decentralized system, there is another similarly-capable organization that can correct errors influenced by private institutional interests. Next in line, I’d suggest externally enforceable guarantees, such as a freedom-of-expression policy with third-party enforcement. I’d place promises that are not externally enforceable in a distinctly lower category; I wouldn’t put much stock in them to the extent that the promisor was the main/only game in town.
I’m ignoring Open Phil here, because I don’t think the Open Phil hegemony is a fundamentally tractable problem. In contrast, RP is seeking funds from Forum users, and so the question of keeping its private institutional interests from potentially affecting community views on cause prioritization is fairly on the table.