Thanks for your comment. I am also concerned about groupthink within homogenous communities. I hope this contest is one small push against groupthink at Open Phil. By default, I do, unfortunately, expect most of the submissions to come from people who share the same basic worldview as Open Phil staff. And for submissions that come from people with radically different worldviews, there is the danger that we fail to recognize an excellent point because we are less familiar with the stylistic and epistemic conventions within which it is embedded.
For these sorts of reasons, we did explicitly consider including non-Open Phil judges for the contest. Ultimately, we decided that didn’t make sense for this use case. We are, after all, hoping that submissions update our thinking, and it’s harder for an outside judge to represent our point of view.
But this contest is not the only way we are stress-testing our thinking. For example, I’m involved in another project in which we are engaging directly with smart people who disagree with us about AI risk. We hope that as a result of that adversarial collaboration, we can generate a consensus of cruxes so that we have a better handle on how new developments ought to change our credences. I hope to be able to share more details on that project over the summer.
If you want to chat more about groupthink concerns, shoot me a DM. I believe it’s a somewhat underappreciated worry within EA.
You could of course commit to acting on some kind of judgment of some diverse group you think worth differing too, rather than acting on your own opinion. One way to understand what David Thorstad is asking (which he might or might not endorse) is why you don’t do that given it would (allegedly) mean acting on a more-like-to-be-correct opinion, rather than one that is less-likely to be correct. From that point of view, it’s just missing the point to say ‘we’re trying to get our opinion updated’, because you shouldn’t be using your opinions, rather than some properly diverse groups opinions to be setting policy in general.
Hi David,
Thanks for your comment. I am also concerned about groupthink within homogenous communities. I hope this contest is one small push against groupthink at Open Phil. By default, I do, unfortunately, expect most of the submissions to come from people who share the same basic worldview as Open Phil staff. And for submissions that come from people with radically different worldviews, there is the danger that we fail to recognize an excellent point because we are less familiar with the stylistic and epistemic conventions within which it is embedded.
For these sorts of reasons, we did explicitly consider including non-Open Phil judges for the contest. Ultimately, we decided that didn’t make sense for this use case. We are, after all, hoping that submissions update our thinking, and it’s harder for an outside judge to represent our point of view.
But this contest is not the only way we are stress-testing our thinking. For example, I’m involved in another project in which we are engaging directly with smart people who disagree with us about AI risk. We hope that as a result of that adversarial collaboration, we can generate a consensus of cruxes so that we have a better handle on how new developments ought to change our credences. I hope to be able to share more details on that project over the summer.
If you want to chat more about groupthink concerns, shoot me a DM. I believe it’s a somewhat underappreciated worry within EA.
You could of course commit to acting on some kind of judgment of some diverse group you think worth differing too, rather than acting on your own opinion. One way to understand what David Thorstad is asking (which he might or might not endorse) is why you don’t do that given it would (allegedly) mean acting on a more-like-to-be-correct opinion, rather than one that is less-likely to be correct. From that point of view, it’s just missing the point to say ‘we’re trying to get our opinion updated’, because you shouldn’t be using your opinions, rather than some properly diverse groups opinions to be setting policy in general.