[I’ll put some thoughts on the ALLFED section here to keep discussion organised, but this is responding to Nuno’s section rather than David’s comment.]
I feel that that 50% is still pretty good, but the contrast between it and the model’s initial 95% is pretty noticeable to me, and makes me feel that the 95% is uncalibrated/untrustworthy. On the other hand, my probabilities above can also be seen as a sort of sensitivity analysis, which shows that the case for an organization working on ALLFED’s cause area is somewhat more robust than one might have thought.
[...]
In conclusion, I disagree strongly with ALLFED’s estimates (probability of cost overruns, impact of ALLFED’s work if deployed, etc.), however, I feel that the case for an organization working in this area is relatively solid. My remaining uncertainty is about ALLFED’s ability to execute competently and cost-effectively; independent expert evaluation might resolve most of it.
I think this mostly sounds similar to my independent impression, as expressed here, though I didn’t specifically worry particularly about their ability to execute competently and cost-effectively. (I’m not saying I felt highly confident about that; it just didn’t necessarily stand out much to me as a key uncertainty, for whatever reason.)
E.g., I wrote in the linked comment:
Their cost-effectiveness estimates seem remarkably promising (see here and here).
But it does seem quite hard to believe that the cost-effectiveness is really that good. And many of the quantities are based on a survey of GCR researchers, with somewhat unclear methodology (e.g., how were the researchers chosen?)
I also haven’t analysed the models very closely
But, other than perhaps the reliance on that survey, I can’t obviously see major flaws, and haven’t seen comments that seem to convincingly point out major flaws. So maybe the estimates are in the right ballpark?
One thing I’d add is that most of your (Nuno’s) section on ALLFED sounds like it’s seeing ALLFED’s impact as mostly being about their research & advocacy itself. But I think it’s worth also giving a fair amount of emphasis to this question of yours: “Given that ALLFED has a large team, is it a positive influence on its team members? How would we expect employees and volunteers to rate their experience with the organization?”
I’d see a substantial fraction of the value of ALLFED as coming from how it might work as a useful talent pipeline. And I think that this could also be a source of nontrivial downside risk from ALLFED, e.g. if their training is low-quality for some reason, or if people implicitly learn bad habits of thinking/research/modelling, or if their focuses aren’t good focuses and they make their volunteers more likely to stay focused on that long-term.
(I’m not saying that these things are the case. I’d currently guess that ALLFED produces notable impact as a talent pipeline. But I haven’t looked closely and think it’d be worth doing so if one wanted to do a “thorough” evaluation of ALLFED.)
[I’ll put some thoughts on the ALLFED section here to keep discussion organised, but this is responding to Nuno’s section rather than David’s comment.]
I think this mostly sounds similar to my independent impression, as expressed here, though I didn’t specifically worry particularly about their ability to execute competently and cost-effectively. (I’m not saying I felt highly confident about that; it just didn’t necessarily stand out much to me as a key uncertainty, for whatever reason.)
E.g., I wrote in the linked comment:
One thing I’d add is that most of your (Nuno’s) section on ALLFED sounds like it’s seeing ALLFED’s impact as mostly being about their research & advocacy itself. But I think it’s worth also giving a fair amount of emphasis to this question of yours: “Given that ALLFED has a large team, is it a positive influence on its team members? How would we expect employees and volunteers to rate their experience with the organization?”
I’d see a substantial fraction of the value of ALLFED as coming from how it might work as a useful talent pipeline. And I think that this could also be a source of nontrivial downside risk from ALLFED, e.g. if their training is low-quality for some reason, or if people implicitly learn bad habits of thinking/research/modelling, or if their focuses aren’t good focuses and they make their volunteers more likely to stay focused on that long-term.
(I’m not saying that these things are the case. I’d currently guess that ALLFED produces notable impact as a talent pipeline. But I haven’t looked closely and think it’d be worth doing so if one wanted to do a “thorough” evaluation of ALLFED.)