[Edit: as per Saulius’ reply below I was perhaps to critical here, especially regarding the WAW post, and it sounds like Saulius thinks that was manged relatively well by RP senior staff]
I found this reply made me less confident in Rethink’s ability to address publication bias. Some things that triggered my ‘hmmm not so sure about this’ sense were:
The reply did not directly address the claims in Saulius’s comment. E.g. “I’m sorry you feel that way” not “I’m sorry”. No acknowledgement that if, as Saulius claimed, a senior staff told him that it was wrong to have expressed his views to OpenPhil (when asked by OpenPhil for his views), that this might have been a mistake by that staff. I guess I was expecting more of a ‘huh that sounds bad, let us look into it’ style response than a ‘nothing to see here, all is great in Rethink land’ style response.
The story about Saulius’ WAW post. Judging from Marcus’ comment It sounds like Rethink stopped Saulius from posting a WAW post (within a work context) and it also looks like there was a potential conflict of interest here for senior staff as posting could affect funding. Marcus says that RP senior staff were only deciding based on the quality of the post. Now I notice the post itself (published a few months after Saulius left Rethink) seems to have significantly more upvotes than almost any of Rethink’s posts (and lots of positive feedback). I recognise upvotes are not a great measure of research quality but it does make me worry that about the possibility that this post was actually of sufficient quality and the conflict of interest may have biased senior Rethink staff to stop the post being published as a Rethink post.
The stated donor engagement principles seem problematic. Eg: “there are also times when we think it is important for RP to speak with a unified voice to our most important donors” is exactly the kind of reason I would use if I was censoring staff’s interaction with donors. It is not that “unified organisational voice” policies are wrong, just that without additional safeguards they have a risk of being abused, of facilitating the kinds of conflict of interest driven actions under discussion here. Also as Saulius mentions such policies can also be one-sided where staff are welcome to say anything that aligns with a certain worldview but need to get sign-off to disagree with that worldview, another source of bias.
I really really do like Rethink. And I do not think that this is a huge problem, or enough of a problem to stop donors giving to Rethink in most cases. But I would still be interested in seeing additional evidence of Rethink addressing this risk of bias.
It sounds like Rethink stopped Saulius from posting a WAW post (within a work context) and it also looks like there was a potential conflict of interest here for senior staff as posting could affect funding
It is true that I wasn’t allowed to publish some of my WAW work on behalf of RP. Note that this WAW work includes not only the short summary Why I No Longer Prioritize Wild Animal Welfare (which got a lot of upvotes) but also three longer articles that it summarises (this, this, and this). Some of these do not threaten RP’s funding in any way. That said, I was allowed to work on these articles in my work time which should count for something.
Let me give a bit of context because it might not make sense without it. My project was finding the best WAW intervention. I struggled with it a lot. Instead of doing what I was supposed to, I started writing about why I was struggling, which eventually turned into all those posts. I asked my manager (who was quite new to RP) if I could continue working on those posts and publish them on behalf of RP. My manager allowed it. Then Marcus read some of my drafts and gave detailed useful feedback. He said that while he was happy that I looked into this stuff, these articles couldn’t be published on behalf of RP. As I remember it, the reason was that they were basically opinion pieces. He wanted RP to only post stuff that is closer to an academic publication. I asked why but I don’t want to share his answer publicly in case what he told me wasn’t public. I disagreed with his position but it’s the sort of thing that reasonable people can disagree on. We both thought that it wasn’t worth trying to bring my articles to the degree of polish (and perhaps rigour) that would meet RP’s publishing standards but that they were still worth publishing. Marcus said that I’m free to finish my posts in my work time, which was very kind of him. Note that this was financially disadvantageous for RP. I felt uncomfortable spending more time on non-RP stuff in my work time so I took an unpaid leave to finish my posts, but it was my own idea and Marcus explicitly told me that I don’t need to do that.
Thank you Saulius. Very helpful to hear. This sounds like a really positive story of good management of a difficult situation. Well done to Marcus.
If I read between the lines a bit I get the impression that maybe more junior (be that less competent or just newer to the org) managers at Rethink with less confidence in their actions not rocking the Rethink<->funder relationship were perhaps more likely to put unwelcome pressure on researchers about what to publish. Just a hypothesis, so might be wrong. But also the kind of thing good internal policies, good onboarding, good senior example setting, or just discussions of this topic, can all help with.
No, sorry, I wasn’t saying that. My manager was Jacob Peacock, he was a great manager. He didn’t put any unwelcome pressure and wasn’t the one who talked to me about the email to OpenPhil. He said that I can publish my WAW articles on behalf of RP but then Marcus disagreed.
[Edit: as per Saulius’ reply below I was perhaps to critical here, especially regarding the WAW post, and it sounds like Saulius thinks that was manged relatively well by RP senior staff]
I found this reply made me less confident in Rethink’s ability to address publication bias. Some things that triggered my ‘hmmm not so sure about this’ sense were:
The reply did not directly address the claims in Saulius’s comment. E.g. “I’m sorry you feel that way” not “I’m sorry”. No acknowledgement that if, as Saulius claimed, a senior staff told him that it was wrong to have expressed his views to OpenPhil (when asked by OpenPhil for his views), that this might have been a mistake by that staff. I guess I was expecting more of a ‘huh that sounds bad, let us look into it’ style response than a ‘nothing to see here, all is great in Rethink land’ style response.
The story about Saulius’ WAW post. Judging from Marcus’ comment It sounds like Rethink stopped Saulius from posting a WAW post (within a work context) and it also looks like there was a potential conflict of interest here for senior staff as posting could affect funding. Marcus says that RP senior staff were only deciding based on the quality of the post. Now I notice the post itself (published a few months after Saulius left Rethink) seems to have significantly more upvotes than almost any of Rethink’s posts (and lots of positive feedback). I recognise upvotes are not a great measure of research quality but it does make me worry that about the possibility that this post was actually of sufficient quality and the conflict of interest may have biased senior Rethink staff to stop the post being published as a Rethink post.
The stated donor engagement principles seem problematic. Eg: “there are also times when we think it is important for RP to speak with a unified voice to our most important donors” is exactly the kind of reason I would use if I was censoring staff’s interaction with donors. It is not that “unified organisational voice” policies are wrong, just that without additional safeguards they have a risk of being abused, of facilitating the kinds of conflict of interest driven actions under discussion here. Also as Saulius mentions such policies can also be one-sided where staff are welcome to say anything that aligns with a certain worldview but need to get sign-off to disagree with that worldview, another source of bias.
I really really do like Rethink. And I do not think that this is a huge problem, or enough of a problem to stop donors giving to Rethink in most cases. But I would still be interested in seeing additional evidence of Rethink addressing this risk of bias.
It is true that I wasn’t allowed to publish some of my WAW work on behalf of RP. Note that this WAW work includes not only the short summary Why I No Longer Prioritize Wild Animal Welfare (which got a lot of upvotes) but also three longer articles that it summarises (this, this, and this). Some of these do not threaten RP’s funding in any way. That said, I was allowed to work on these articles in my work time which should count for something.
Let me give a bit of context because it might not make sense without it. My project was finding the best WAW intervention. I struggled with it a lot. Instead of doing what I was supposed to, I started writing about why I was struggling, which eventually turned into all those posts. I asked my manager (who was quite new to RP) if I could continue working on those posts and publish them on behalf of RP. My manager allowed it. Then Marcus read some of my drafts and gave detailed useful feedback. He said that while he was happy that I looked into this stuff, these articles couldn’t be published on behalf of RP. As I remember it, the reason was that they were basically opinion pieces. He wanted RP to only post stuff that is closer to an academic publication. I asked why but I don’t want to share his answer publicly in case what he told me wasn’t public. I disagreed with his position but it’s the sort of thing that reasonable people can disagree on. We both thought that it wasn’t worth trying to bring my articles to the degree of polish (and perhaps rigour) that would meet RP’s publishing standards but that they were still worth publishing. Marcus said that I’m free to finish my posts in my work time, which was very kind of him. Note that this was financially disadvantageous for RP. I felt uncomfortable spending more time on non-RP stuff in my work time so I took an unpaid leave to finish my posts, but it was my own idea and Marcus explicitly told me that I don’t need to do that.
Thank you Saulius. Very helpful to hear. This sounds like a really positive story of good management of a difficult situation. Well done to Marcus.
If I read between the lines a bit I get the impression that maybe more junior (be that less competent or just newer to the org) managers at Rethink with less confidence in their actions not rocking the Rethink<->funder relationship were perhaps more likely to put unwelcome pressure on researchers about what to publish. Just a hypothesis, so might be wrong. But also the kind of thing good internal policies, good onboarding, good senior example setting, or just discussions of this topic, can all help with.
No, sorry, I wasn’t saying that. My manager was Jacob Peacock, he was a great manager. He didn’t put any unwelcome pressure and wasn’t the one who talked to me about the email to OpenPhil. He said that I can publish my WAW articles on behalf of RP but then Marcus disagreed.
Thank you for the correction