Thanks for writing this, this is also something that’s been on my mind with some degree of uncertainty.
My confidence for a lot of these reports would increase if I could see the peer review comments and responses or if otherwise the research (in parts or in full) was published in a peer reviewed academic journal. I know a lot of Open Phil reports commission external peer review and if I recall correct the climate change report was also peer reviewed. At the same time some of the comments in that thread implied reviewers had a lot of disagreements and it’s hard to say how much of the feedback was responded to or not. To be clear, you don’t have to agree with every peer review comment but seeing the responses would increase my confidence.
I’m still left with the impression that most work within EA isn’t externally reviewed.
I wonder if some of the recent public award prizes, like Open Phil’s cause prioritization one and GiveWell’s change our minds somewhat fit here. More for Open Phil but I never found or got a sense of how entries were assessed or ranked. What are the crteria? This me doubt how we assess and valued expertise and research rigor.
Thanks for writing this, this is also something that’s been on my mind with some degree of uncertainty.
My confidence for a lot of these reports would increase if I could see the peer review comments and responses or if otherwise the research (in parts or in full) was published in a peer reviewed academic journal. I know a lot of Open Phil reports commission external peer review and if I recall correct the climate change report was also peer reviewed. At the same time some of the comments in that thread implied reviewers had a lot of disagreements and it’s hard to say how much of the feedback was responded to or not. To be clear, you don’t have to agree with every peer review comment but seeing the responses would increase my confidence.
I’m still left with the impression that most work within EA isn’t externally reviewed.
I wonder if some of the recent public award prizes, like Open Phil’s cause prioritization one and GiveWell’s change our minds somewhat fit here. More for Open Phil but I never found or got a sense of how entries were assessed or ranked. What are the crteria? This me doubt how we assess and valued expertise and research rigor.