I would say “comparing the crowd’s accuracy to reality” would be best, but “future GiveWell evaluations” is another reasonable option.
Consider Metaculus’s record vs any other paid experts.
Metaculus produces world class answers off a user base of 12,000.
I don’t know what Metaculus’s record is against “other paid experts,” and I expect it would depend on which experts and which topic was up for prediction. I think the average researcher at GiveWell is probably much, much better at probabilistic reasoning than the average pundit or academic, because GiveWell’s application process tests this skill and working at GiveWell requires that the skill be used frequently.
I also don’t know where your claim that “Metaculus produces world-class answers” comes from. Could you link to some evidence? (In general, a lot of your comments make substantial claims without links or citations, which can make it hard to engage with them.)
Open Philanthropy has contracted with Good Judgment Inc. for COVID forecasting, so this idea is definitely on the organization’s radar (and by extension, GiveWell’s). Have you tried asking them why they don’t ask questions on Metaculus or make more use of crowdsourcing in general? I’m sure they’d have a better explanation for you than anything I could hypothesize :-)
I don’t feel like open philanthropy would answer my speculative emails. Now that you point it out they might, but in general I don’t feel worthy of their time.
(Originally I wrote this beaty of a sentence ” previously I don’t think I’d have thought they thought me worthy of their time.”)
If you really think GiveWell or Open Philanthropy is missing out on a lot of value by failing to pursue a certain strategy, it seems like you should aim to make the most convincing case you can for their sake!
(Perhaps it would be safer to write a post specifically about this topic, then send it to them; that way, even if there’s no reply, you at least have the post and can get feedback from other people.)
Also, possibly room for a “request citation” button. When you talk in different online communities it’s not clear how much citing you should do. An easy way to request and add citations would not require additional comments.
I would say “comparing the crowd’s accuracy to reality” would be best, but “future GiveWell evaluations” is another reasonable option.
I don’t know what Metaculus’s record is against “other paid experts,” and I expect it would depend on which experts and which topic was up for prediction. I think the average researcher at GiveWell is probably much, much better at probabilistic reasoning than the average pundit or academic, because GiveWell’s application process tests this skill and working at GiveWell requires that the skill be used frequently.
I also don’t know where your claim that “Metaculus produces world-class answers” comes from. Could you link to some evidence? (In general, a lot of your comments make substantial claims without links or citations, which can make it hard to engage with them.)
Open Philanthropy has contracted with Good Judgment Inc. for COVID forecasting, so this idea is definitely on the organization’s radar (and by extension, GiveWell’s). Have you tried asking them why they don’t ask questions on Metaculus or make more use of crowdsourcing in general? I’m sure they’d have a better explanation for you than anything I could hypothesize :-)
Noted on the lack of citations.
I don’t feel like open philanthropy would answer my speculative emails. Now that you point it out they might, but in general I don’t feel worthy of their time.
(Originally I wrote this beaty of a sentence ” previously I don’t think I’d have thought they thought me worthy of their time.”)
If you really think GiveWell or Open Philanthropy is missing out on a lot of value by failing to pursue a certain strategy, it seems like you should aim to make the most convincing case you can for their sake!
(Perhaps it would be safer to write a post specifically about this topic, then send it to them; that way, even if there’s no reply, you at least have the post and can get feedback from other people.)
Also, possibly room for a “request citation” button. When you talk in different online communities it’s not clear how much citing you should do. An easy way to request and add citations would not require additional comments.