Very curious to hear how Open Philanthropy has updated as a result of this competition.
Looking at the winning entries would seem to suggest that Open Philanthropy is likely to now be less worried about these risks, but it would be interesting to know by how much.
(1) I wouldn’t model Open Phil as having a single view on these sorts of questions. There’s a healthy diversity of opinions, and as stated in the “caveats” section, I think different Open Phil employees might have chosen different winners.
(2) Even for the subset of Open Phil employees who served as judges, I wouldn’t interpret these entries as collectively moving our views a ton. We were looking for the best challenges to our AI worldview in this contest, and as such I don’t think it should be too surprising that the winning entries are more skeptical of AI risks than we are.
Hi Jason, thank you for giving a quick response. Both points are very reasonable.
The contest announcement post outlined “several ways an essay could substantively inform the thinking of a panelist”, namely, changing the central estimate or shape of the probability distribution of AGI / AGI catastrophe, or clarifying a concept or identifying a crux.
It would be very interesting to hear if any of the submissions did change any of the panelists’ (or other Open Phil employees’) mind in these ways, and how so. If not, whether because you learned an unanticipated kind of a thing, or because the contest turned out to be less useful than you initially hoped, I think that might also be very valuable for the community to know.
(2) Even for the subset of Open Phil employees who served as judges, I wouldn’t interpret these entries as collectively moving our views a ton. We were looking for the best challenges to our AI worldview in this contest, and as such I don’t think it should be too surprising that the winning entries are more skeptical of AI risks than we are.
It would be great if you could put words to this effect—or state your actual current views on AI x-risk—right up front in your winners announcement, because to me (and no doubt many others) it basically looks like OpenPhil are updating away from the problem being urgent right at the point where we’ve hitcrunch time and it couldn’t be more urgent! I’m really quite upset about this.
I also got that feeling. I do assume this is just unfortunate optics and they mostly wanted to reward the winners for making good and original arguments, but it would be good at least to state how their views have been influenced, and what the particular arguments of each winning essay were the most relevant for their decision.
Panelist credences on the probability of AGI by 2043 range from ~10% to ~45%. Conditional on AGI being developed by 2070, panelist credences on the probability of existential catastrophe range from ~5% to ~50%.
Very curious to hear how Open Philanthropy has updated as a result of this competition.
Looking at the winning entries would seem to suggest that Open Philanthropy is likely to now be less worried about these risks, but it would be interesting to know by how much.
Hi Chris,
Thanks for your question. Two quick points:
(1) I wouldn’t model Open Phil as having a single view on these sorts of questions. There’s a healthy diversity of opinions, and as stated in the “caveats” section, I think different Open Phil employees might have chosen different winners.
(2) Even for the subset of Open Phil employees who served as judges, I wouldn’t interpret these entries as collectively moving our views a ton. We were looking for the best challenges to our AI worldview in this contest, and as such I don’t think it should be too surprising that the winning entries are more skeptical of AI risks than we are.
Hi Jason, thank you for giving a quick response. Both points are very reasonable.
The contest announcement post outlined “several ways an essay could substantively inform the thinking of a panelist”, namely, changing the central estimate or shape of the probability distribution of AGI / AGI catastrophe, or clarifying a concept or identifying a crux.
It would be very interesting to hear if any of the submissions did change any of the panelists’ (or other Open Phil employees’) mind in these ways, and how so. If not, whether because you learned an unanticipated kind of a thing, or because the contest turned out to be less useful than you initially hoped, I think that might also be very valuable for the community to know.
Thanks!
It would be great if you could put words to this effect—or state your actual current views on AI x-risk—right up front in your winners announcement, because to me (and no doubt many others) it basically looks like OpenPhil are updating away from the problem being urgent right at the point where we’ve hit crunch time and it couldn’t be more urgent! I’m really quite upset about this.
I also got that feeling. I do assume this is just unfortunate optics and they mostly wanted to reward the winners for making good and original arguments, but it would be good at least to state how their views have been influenced, and what the particular arguments of each winning essay were the most relevant for their decision.
Unfortunate optics confirmed. But would still be good to get an update on:
[from the contest announcement]