I would say âcomparing the crowdâs accuracy to realityâ would be best, but âfuture GiveWell evaluationsâ is another reasonable option.
Consider Metaculusâs record vs any other paid experts.
Metaculus produces world class answers off a user base of 12,000.
I donât know what Metaculusâs record is against âother paid experts,â and I expect it would depend on which experts and which topic was up for prediction. I think the average researcher at GiveWell is probably much, much better at probabilistic reasoning than the average pundit or academic, because GiveWellâs application process tests this skill and working at GiveWell requires that the skill be used frequently.
I also donât know where your claim that âMetaculus produces world-class answersâ comes from. Could you link to some evidence? (In general, a lot of your comments make substantial claims without links or citations, which can make it hard to engage with them.)
Open Philanthropy has contracted with Good Judgment Inc. for COVID forecasting, so this idea is definitely on the organizationâs radar (and by extension, GiveWellâs). Have you tried asking them why they donât ask questions on Metaculus or make more use of crowdsourcing in general? Iâm sure theyâd have a better explanation for you than anything I could hypothesize :-)
I donât feel like open philanthropy would answer my speculative emails. Now that you point it out they might, but in general I donât feel worthy of their time.
(Originally I wrote this beaty of a sentence â previously I donât think Iâd have thought they thought me worthy of their time.â)
If you really think GiveWell or Open Philanthropy is missing out on a lot of value by failing to pursue a certain strategy, it seems like you should aim to make the most convincing case you can for their sake!
(Perhaps it would be safer to write a post specifically about this topic, then send it to them; that way, even if thereâs no reply, you at least have the post and can get feedback from other people.)
Also, possibly room for a ârequest citationâ button. When you talk in different online communities itâs not clear how much citing you should do. An easy way to request and add citations would not require additional comments.
I would say âcomparing the crowdâs accuracy to realityâ would be best, but âfuture GiveWell evaluationsâ is another reasonable option.
I donât know what Metaculusâs record is against âother paid experts,â and I expect it would depend on which experts and which topic was up for prediction. I think the average researcher at GiveWell is probably much, much better at probabilistic reasoning than the average pundit or academic, because GiveWellâs application process tests this skill and working at GiveWell requires that the skill be used frequently.
I also donât know where your claim that âMetaculus produces world-class answersâ comes from. Could you link to some evidence? (In general, a lot of your comments make substantial claims without links or citations, which can make it hard to engage with them.)
Open Philanthropy has contracted with Good Judgment Inc. for COVID forecasting, so this idea is definitely on the organizationâs radar (and by extension, GiveWellâs). Have you tried asking them why they donât ask questions on Metaculus or make more use of crowdsourcing in general? Iâm sure theyâd have a better explanation for you than anything I could hypothesize :-)
Noted on the lack of citations.
I donât feel like open philanthropy would answer my speculative emails. Now that you point it out they might, but in general I donât feel worthy of their time.
(Originally I wrote this beaty of a sentence â previously I donât think Iâd have thought they thought me worthy of their time.â)
If you really think GiveWell or Open Philanthropy is missing out on a lot of value by failing to pursue a certain strategy, it seems like you should aim to make the most convincing case you can for their sake!
(Perhaps it would be safer to write a post specifically about this topic, then send it to them; that way, even if thereâs no reply, you at least have the post and can get feedback from other people.)
Also, possibly room for a ârequest citationâ button. When you talk in different online communities itâs not clear how much citing you should do. An easy way to request and add citations would not require additional comments.