This was a cool contest, thanks for running it! In my view there’s a lot of value in doing this. Doing a deep dive into polygenic selection for IQ was something I had wanted to do for quite a while and your contest motivated me to finally sit down and actually do it and to write it up in a way that would be potentially useful to others.
I think your initial criteria of how much a writeup changed your minds may have played a role in fewer than expected entries as well. Your forecasts on the set of questions seemed very reasonable and my own forecasts were pretty similar on the ones I had forecasted, so I didn’t feel that I had much to contribute in terms of the contest criteria for most of them.
Hopefully that’s helpful feedback to you or anyone else looking to run a contest like this in the future!
At first I thought the scenarios were separate so they would be combined with an OR to get an overall probability, which then made me confused when you looked at only scenario 1 for determining your probability for technological feasibility.
I was also confused about why you assigned 30% to polygenic scores reaching 80% predictive power in Scenario 2 while assigning 80% to reaching saturation at 40% predictive power in the Scenario 1, because when I read 80% to reach saturation at 40% predictive power I read this as “capping out at around 40%” which would only leave a maximum of 20% for scenarios with much greater than 40%?
Finally, I was a little confused about where the likelihood of iterated embryo selection fit into your scenarios; this seems highly relevant/important and is maybe implicitly accounted for in e.g. “Must be able to generate 100 embryos to select from”? But could be good to make more explicit.
There are good points and helpful, thanks! I agree I wasn’t clear about viewing the scenarios exclusively in the initial comment, I think I made that a little clearer in the follow up.
when I read 80% to reach saturation at 40% predictive power I read this as “capping out at around 40%” which would only leave a maximum of 20% for scenarios with much greater than 40%?
Ah I think I see how that’s confusing. My use of the term saturation probably confuses things too much. My understanding is saturation is the likely maximum that could be explained with current approaches, so my forecast was an 80% chance we get to the 40% “saturation” level, but I think there’s a decent chance our technology/understanding advances so that more than the saturation can be explained, and I gave a 30% chance that we reach 80% predictive power.
That’s a good point about iterated embryo selection, I totally neglected that. My initial thought is it would probably overlap a lot with the scenarios I used, but I should have given that more thought and discussed it in my comment.
This was a cool contest, thanks for running it! In my view there’s a lot of value in doing this. Doing a deep dive into polygenic selection for IQ was something I had wanted to do for quite a while and your contest motivated me to finally sit down and actually do it and to write it up in a way that would be potentially useful to others.
I think your initial criteria of how much a writeup changed your minds may have played a role in fewer than expected entries as well. Your forecasts on the set of questions seemed very reasonable and my own forecasts were pretty similar on the ones I had forecasted, so I didn’t feel that I had much to contribute in terms of the contest criteria for most of them.
Hopefully that’s helpful feedback to you or anyone else looking to run a contest like this in the future!
Yes, thank you; that makes sense and is very helpful!
Thanks for sharing Ryan, and that makes sense in terms of another unintended consequence of our judging criteria; good to know for future contests.
No problem!
Also if you’re interested in elaborating about why my scenarios were unintuitive I’d appreciate the feedback, but if not no worries!
At first I thought the scenarios were separate so they would be combined with an OR to get an overall probability, which then made me confused when you looked at only scenario 1 for determining your probability for technological feasibility.
I was also confused about why you assigned 30% to polygenic scores reaching 80% predictive power in Scenario 2 while assigning 80% to reaching saturation at 40% predictive power in the Scenario 1, because when I read 80% to reach saturation at 40% predictive power I read this as “capping out at around 40%” which would only leave a maximum of 20% for scenarios with much greater than 40%?
Finally, I was a little confused about where the likelihood of iterated embryo selection fit into your scenarios; this seems highly relevant/important and is maybe implicitly accounted for in e.g. “Must be able to generate 100 embryos to select from”? But could be good to make more explicit.
There are good points and helpful, thanks! I agree I wasn’t clear about viewing the scenarios exclusively in the initial comment, I think I made that a little clearer in the follow up.
Ah I think I see how that’s confusing. My use of the term saturation probably confuses things too much. My understanding is saturation is the likely maximum that could be explained with current approaches, so my forecast was an 80% chance we get to the 40% “saturation” level, but I think there’s a decent chance our technology/understanding advances so that more than the saturation can be explained, and I gave a 30% chance that we reach 80% predictive power.
That’s a good point about iterated embryo selection, I totally neglected that. My initial thought is it would probably overlap a lot with the scenarios I used, but I should have given that more thought and discussed it in my comment.