Glad you’re thinking about improving research and forecasting! My very quick take after only skimming your post is that it just happens too rarely that an EA forum post significantly informs specific Metaculus questions. Consequently investing in such a feature seems not sufficiently useful. (Though that might change of course if there were many more Metaculus questions that relate to topics in EA.) Maybe you could list some concrete examples for such EA forum—Metaculus connections? (Sorry if I missed you listing examples!)
Some other random ideas about forecasting on the forum:
Display relative Brier Scores from Metaculus in EAF profiles → incentivize forecasting, enable forum readers to roughly weigh comments by epistemic track records
Hook up the forum to the GPT-4 API and allow forum writers to generate forecasting questions at the end of their posts.
Scott Alexander does this regularly and I find it useful to get a concrete bottom line of his uncertainties and concrete things he’s forecasting based on the post, e.g. see here.
This post comes to mind, which I cited in my nuclear GCR forecasts here, along with many other posts from that series. In general I expect posts from Rethink Priorities to be relevant. I’ve seen similar quality posts for AI risks and pandemics here. Most of my familiarity is with GCR’s but I expected there to be strong overlap between popular forecasting topics and popular EA forum topics more generally. There are lots of GCR related questions on Metaculus, and you can find many cited in that link with my forecasts.
Still, I think you’re right that this wouldn’t be applicable to the majority of EA forum posts. Maybe it’s only even displayed once a post is cited in a forecast, or only a particular tag is eligible for this in order to simplify the implementation.
I do think making people’s forecasting performance more obvious in different contexts would be very useful for the community (re: your brier scores in EAF profiles idea), and would love a central site that’s sort of like a minimum viable linkedin that consolidates relevant metrics for an individual across the top forecasting platforms and has an API that makes it easy to connect to other accounts, or use with discord bots etc. I may write about this soon.
Generating forecasts associated with a post is interesting and I’m sure there are UX opportunities to make this easier / more common, but I need to think more about it.
Glad you’re thinking about improving research and forecasting! My very quick take after only skimming your post is that it just happens too rarely that an EA forum post significantly informs specific Metaculus questions. Consequently investing in such a feature seems not sufficiently useful. (Though that might change of course if there were many more Metaculus questions that relate to topics in EA.) Maybe you could list some concrete examples for such EA forum—Metaculus connections? (Sorry if I missed you listing examples!)
Some other random ideas about forecasting on the forum:
Display relative Brier Scores from Metaculus in EAF profiles → incentivize forecasting, enable forum readers to roughly weigh comments by epistemic track records
Hook up the forum to the GPT-4 API and allow forum writers to generate forecasting questions at the end of their posts.
Scott Alexander does this regularly and I find it useful to get a concrete bottom line of his uncertainties and concrete things he’s forecasting based on the post, e.g. see here.
Good points.
This post comes to mind, which I cited in my nuclear GCR forecasts here, along with many other posts from that series. In general I expect posts from Rethink Priorities to be relevant. I’ve seen similar quality posts for AI risks and pandemics here. Most of my familiarity is with GCR’s but I expected there to be strong overlap between popular forecasting topics and popular EA forum topics more generally. There are lots of GCR related questions on Metaculus, and you can find many cited in that link with my forecasts.
Still, I think you’re right that this wouldn’t be applicable to the majority of EA forum posts. Maybe it’s only even displayed once a post is cited in a forecast, or only a particular tag is eligible for this in order to simplify the implementation.
I do think making people’s forecasting performance more obvious in different contexts would be very useful for the community (re: your brier scores in EAF profiles idea), and would love a central site that’s sort of like a minimum viable linkedin that consolidates relevant metrics for an individual across the top forecasting platforms and has an API that makes it easy to connect to other accounts, or use with discord bots etc. I may write about this soon.
Generating forecasts associated with a post is interesting and I’m sure there are UX opportunities to make this easier / more common, but I need to think more about it.
Thanks for the thoughtful response!