RSS

Metaculus

TagLast edit: 1 Dec 2022 10:41 UTC by brook

Metaculus is a reputation-based site for soliciting and aggregating predictions. It was founded in November 2015 by astrophysicist Anthony Aguirre, cosmologist Greg Laughlin and data scientist Max Wainwright.[1][2]

Tutorials

The official Metaculus tutorials[3] are suitable for beginners as a way to introduce forecasting, probability distributions, and a basic understanding of how to behave under uncertainty.

Tutorial #1 is over-explaining for those with a basic understanding of the forecasting, but still a good introduction to using Metaculus specifically.

For those familiar with forecasting, tutorials #2 and #3 are more useful as a (small) amount of calibration practice and building familiarity with the Metaculus interface.

A video introduction to Metaculus is also available[4].

Funding

As of August 2022, Metaculus has received $5.6 million in grants from Open Philanthropy,[5] and over $300,000 from Effective Altruism Funds.[6]

Further reading

Aguirre, Anthony (2021) A primer on the Metaculus scoring rule, Medium, February 27.

dan (2020) A preliminary look at Metaculus and expert forecasts, Metaculus, June 2.

Hobbhahn, Marius (2022) How good were our AI timeline predictions so far?, Metaculus, April 10.

External links

Metaculus official website

Metaculus official tutorials

Apply for a job

Related entries

AI forecasting | Forecasting | Prediction markets | Manifold markets

  1. ^

    Mann, Adam (2016) The power of prediction markets, Nature, October 18.

  2. ^
  3. ^

    Metaculus, Tutorials

  4. ^

    Brook, Victoria (2022), Metaculus Tutorial, Loom

  5. ^

    Open Philanthropy (2022) Grants database: Metaculus, Open Philanthropy.

  6. ^

    Effective Altruism Infrastructure Fund (2021) May-August 2021: EA Infrastructure Fund grants, Effective Altruism Funds, August.

Me­tac­u­lus’ pre­dic­tions are much bet­ter than low-in­for­ma­tion priors

Vasco Grilo🔸11 Apr 2023 8:36 UTC
53 points
0 comments6 min readEA link

Get­ting GPT-3 to pre­dict Me­tac­u­lus questions

MathiasKB🔸6 May 2022 12:12 UTC
59 points
6 comments2 min readEA link

Ex­plor­ing Me­tac­u­lus’ com­mu­nity predictions

Vasco Grilo🔸24 Mar 2023 7:59 UTC
95 points
17 comments10 min readEA link

Data on fore­cast­ing ac­cu­racy across differ­ent time hori­zons and lev­els of fore­caster experience

Charles Dillon 🔸27 May 2021 18:51 UTC
125 points
7 comments23 min readEA link

AI Risk & Policy Fore­casts from Me­tac­u­lus & FLI’s AI Path­ways Workshop

Will Aldred16 May 2023 8:53 UTC
41 points
0 comments8 min readEA link

Launch­ing the AI Fore­cast­ing Bench­mark Series Q3 | $30k in Prizes

christian8 Jul 2024 17:20 UTC
17 points
0 comments1 min readEA link
(www.metaculus.com)

Analysing In­di­vi­d­ual Con­tri­bu­tions to the Me­tac­u­lus Com­mu­nity Prediction

nikos8 May 2023 22:58 UTC
28 points
1 comment12 min readEA link

Me­tac­u­lus is build­ing a team ded­i­cated to AI forecasting

christian18 Oct 2022 16:08 UTC
35 points
0 comments1 min readEA link
(apply.workable.com)

Bot­tle­necks to more im­pact­ful crowd forecasting

elifland26 Dec 2021 22:48 UTC
47 points
2 comments10 min readEA link
(www.foxy-scout.com)

An es­ti­mate of the value of Me­tac­u­lus questions

NunoSempere22 Oct 2021 17:45 UTC
45 points
19 comments8 min readEA link

Types of speci­fi­ca­tion prob­lems in forecasting

Juan Gil20 Jul 2021 4:17 UTC
35 points
1 comment12 min readEA link

Me­tac­u­lus Launches Q4 Quar­terly Cup!

christian9 Oct 2023 21:36 UTC
8 points
0 comments1 min readEA link
(www.metaculus.com)

[Me­tac­u­lus Event] April 7 Fore­cast Fri­day: A Pro Fore­caster Pre­sents on Longevity Trends in G7 Coun­tries

christian7 Apr 2023 2:01 UTC
11 points
0 comments1 min readEA link
(www.metaculus.com)

Ex­plor­ing Me­tac­u­lus’s AI Track Record

Peter Scoblic1 May 2023 21:02 UTC
52 points
5 comments5 min readEA link

Me­tac­u­lus In­tro­duces New ‘Con­di­tional Pair’ Fore­cast Ques­tions for Mak­ing Con­di­tional Predictions

christian20 Feb 2023 13:36 UTC
60 points
2 comments2 min readEA link
(www.metaculus.com)

Me­tac­u­lus In­tro­duces New Fore­cast Scores, New Leader­board & Medals

christian20 Nov 2023 20:33 UTC
13 points
0 comments3 min readEA link
(www.metaculus.com)

Me­tac­u­lus An­nounces Fore­cast­ing Tour­na­ment to Eval­u­ate Fo­cused Re­search Or­ga­ni­za­tions, in Part­ner­ship With the Fed­er­a­tion of Amer­i­can Scien­tists

christian3 Oct 2023 16:44 UTC
21 points
0 comments2 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Space Tech­nol­ogy & Cli­mate Fore­cast­ing Ini­ti­a­tive

christian11 Oct 2023 1:29 UTC
11 points
1 comment1 min readEA link
(www.metaculus.com)

Fore­cast­ing Newslet­ter: Fe­bru­ary 2022

NunoSempere5 Mar 2022 19:16 UTC
25 points
2 comments9 min readEA link

Pre­dict­ing Poly­genic Selec­tion for IQ

Metaculus28 Mar 2022 18:00 UTC
41 points
4 comments20 min readEA link

Long-Term Fu­ture Fund: April 2019 grant recommendations

Habryka23 Apr 2019 7:00 UTC
142 points
242 comments46 min readEA link

[Question] Me­tac­u­lus Ques­tions Suggest Money Will Do More Good in the Future

MichaelDickens22 Jul 2021 1:56 UTC
20 points
17 comments2 min readEA link

Me­tac­u­lus Launches Chi­nese AI Chips Tour­na­ment, Sup­port­ing In­sti­tute for AI Policy and Strat­egy Research

christian6 Dec 2023 11:26 UTC
27 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Fu­ture of AI Series, Based on Re­search Ques­tions by Arb

christian13 Mar 2024 21:14 UTC
34 points
0 comments1 min readEA link
(www.metaculus.com)

EA In­fras­truc­ture Fund: May–Au­gust 2021 grant recommendations

Max_Daniel24 Dec 2021 10:42 UTC
85 points
19 comments21 min readEA link
(funds.effectivealtruism.org)

Im­pact­ful Fore­cast­ing Prize Re­sults and Reflections

elifland29 Mar 2022 16:16 UTC
40 points
10 comments3 min readEA link

An ex­am­i­na­tion of Me­tac­u­lus’ re­solved AI pre­dic­tions and their im­pli­ca­tions for AI timelines

Charles Dillon 🔸20 Jul 2021 9:07 UTC
81 points
2 comments7 min readEA link

Me­tac­u­lus Pre­dicts Weak AGI in 2 Years and AGI in 10

Chris Leong24 Mar 2023 19:43 UTC
27 points
12 comments1 min readEA link

6 Year De­crease of Me­tac­u­lus AGI Prediction

Chris Leong12 Apr 2022 5:36 UTC
40 points
6 comments1 min readEA link

Me­tac­u­lus Begin­ner Tour­na­ment for New Forecasters

Anastasia6 Jan 2023 2:35 UTC
33 points
5 comments1 min readEA link

Long-Term Fu­ture Fund: April 2020 grants and recommendations

Habryka18 Sep 2020 10:28 UTC
40 points
7 comments9 min readEA link
(app.effectivealtruism.org)

Com­pe­ti­tion for “For­tified Es­says” on nu­clear risk

MichaelA🔸17 Nov 2021 20:55 UTC
35 points
0 comments3 min readEA link
(www.metaculus.com)

900+ Fore­cast­ers on Whether Rus­sia Will In­vade Ukraine

Metaculus19 Feb 2022 13:29 UTC
51 points
0 comments4 min readEA link
(metaculus.medium.com)

Event on Oct 9: Fore­cast­ing Nu­clear Risk with Re­think Pri­ori­ties’ Michael Aird

MichaelA🔸29 Sep 2021 17:45 UTC
24 points
3 comments2 min readEA link
(www.eventbrite.com)

[Me­tac­u­lus Event] April 14 Fore­cast Fri­day: A Pro Fore­caster on Shift­ing Ter­ri­to­rial Con­trol in Ukraine

christian14 Apr 2023 0:40 UTC
5 points
0 comments1 min readEA link

Rus­sia-Ukraine Con­flict: Fore­cast­ing Nu­clear Risk in 2022

Metaculus24 Mar 2022 21:03 UTC
23 points
1 comment12 min readEA link

Im­pact­ful Fore­cast­ing Prize for fore­cast write­ups on cu­rated Me­tac­u­lus questions

elifland4 Feb 2022 20:06 UTC
91 points
13 comments4 min readEA link

“Nu­clear risk re­search, fore­cast­ing, & im­pact” [pre­sen­ta­tion]

MichaelA🔸21 Oct 2021 10:54 UTC
20 points
0 comments1 min readEA link
(www.youtube.com)

[Question] What im­por­tant ques­tions are miss­ing from Me­tac­u­lus?

Charles Dillon 🔸26 May 2021 14:03 UTC
38 points
12 comments1 min readEA link

An anal­y­sis of Me­tac­u­lus pre­dic­tions of fu­ture EA re­sources, 2025 and 2030

Charles Dillon 🔸22 Sep 2021 10:24 UTC
50 points
2 comments8 min readEA link

Me­tac­u­lus Pre­sents: Does Gen­er­a­tive AI In­fringe Copy­right?

christian6 Nov 2023 23:41 UTC
5 points
0 comments1 min readEA link

Me­tac­u­lus Pre­sents: Trans­for­ma­tive Science at Startup Speed

christian31 Oct 2023 21:12 UTC
5 points
0 comments1 min readEA link

Me­tac­u­lus Launches the ‘Fore­cast­ing Our World In Data’ Pro­ject to Probe the Long-Term Future

christian14 Oct 2022 17:00 UTC
65 points
6 comments1 min readEA link
(www.metaculus.com)

An­nounc­ing the AI Fore­cast­ing Bench­mark Series | July 8, $120k in Prizes

christian19 Jun 2024 21:37 UTC
50 points
4 comments5 min readEA link
(www.metaculus.com)

Open Sourc­ing Metaculus

christian25 Jun 2024 18:40 UTC
75 points
4 comments2 min readEA link
(www.metaculus.com)

An­nounc­ing the Ben­tham Prize

Pablo21 Jan 2020 22:23 UTC
33 points
1 comment1 min readEA link

An­nounc­ing the Nu­clear Risk Fore­cast­ing Tournament

MichaelA🔸16 Jun 2021 16:12 UTC
38 points
0 comments2 min readEA link

Me­tac­u­lus Is Open Source

christian7 Oct 2024 19:55 UTC
33 points
1 comment1 min readEA link
(www.metaculus.com)

Com­par­ing Fore­cast­ing Track Records for AI Bench­mark­ing and Beyond

Tom Liptay25 Sep 2024 21:01 UTC
14 points
1 comment19 min readEA link
(www.metaculus.com)

De­sign­ing Ar­tifi­cial Wis­dom: De­ci­sion Fore­cast­ing AI & Futarchy

Jordan Arel14 Jul 2024 5:10 UTC
5 points
1 comment6 min readEA link

Me­tac­u­lus World Map Experiment

Ryan Beck16 Jul 2024 18:19 UTC
20 points
1 comment1 min readEA link
(www.metaculus.com)

Can AI Out­pre­dict Hu­mans? Re­sults From Me­tac­u­lus’s Q3 AI Fore­cast­ing Benchmark

Tom Liptay10 Oct 2024 18:58 UTC
32 points
1 comment6 min readEA link
(www.metaculus.com)

Me­tac­u­lus’s ‘Mini­tac­u­lus’ Ex­per­i­ments — Col­lab­o­rate With Us

christian26 Aug 2024 20:44 UTC
10 points
0 comments1 min readEA link
(www.metaculus.com)

OpenAI and An­thropic Donate Cred­its for AI Fore­cast­ing Bench­mark Tournament

christian17 Jul 2024 21:50 UTC
2 points
0 comments1 min readEA link

Launch­ing the Re­s­pi­ra­tory Out­look 2024/​25 Fore­cast­ing Series

christian17 Jul 2024 19:51 UTC
8 points
0 comments1 min readEA link
(www.metaculus.com)

Pre­dic­tions as Public Works Pro­ject — What Me­tac­u­lus Is Build­ing Next

Sylvain22 Oct 2024 16:35 UTC
24 points
4 comments5 min readEA link
(www.metaculus.com)

Fore­cast With GiveWell

christian23 Oct 2024 19:50 UTC
35 points
6 comments2 min readEA link
(www.metaculus.com)

Fore­casts about EA or­gani­sa­tions which are cur­rently on Me­tac­u­lus.

alex lawsen29 Dec 2020 17:42 UTC
65 points
4 comments3 min readEA link

Col­lab­o­ra­tive In­tel­li­gence: Har­ness­ing Crowd Fore­cast­ing for Na­tional Security

Deger Turan9 Dec 2024 22:12 UTC
6 points
1 comment4 min readEA link
(fas.org)

[Question] What ques­tions would you like to see fore­casts on from the Me­tac­u­lus com­mu­nity?

alex lawsen26 Jul 2020 14:40 UTC
32 points
30 comments2 min readEA link

An­nual AGI Bench­mark­ing Event

Metaculus26 Aug 2022 21:31 UTC
20 points
2 comments2 min readEA link
(www.metaculus.com)

Me­tac­u­lus seek­ing An­a­lyt­i­cal Sto­ry­tel­lers to write es­says for­tified with testable predictions

christian6 Oct 2021 4:39 UTC
16 points
4 comments1 min readEA link

Me­tac­u­lus is seek­ing ex­pe­rienced lead­ers, re­searchers & op­er­a­tors for high-im­pact roles

christian10 Jul 2022 14:29 UTC
13 points
0 comments1 min readEA link
(apply.workable.com)

[job] Me­tac­u­lus has new soft­ware roles

dschwarz7 Nov 2022 21:19 UTC
9 points
0 comments1 min readEA link
(apply.workable.com)

Me­tac­u­lus is hiring

aaguirre9 Dec 2020 20:58 UTC
30 points
0 comments1 min readEA link

How does fore­cast quan­tity im­pact fore­cast qual­ity on Me­tac­u­lus?

Charles Dillon 🔸1 Oct 2021 18:57 UTC
60 points
14 comments8 min readEA link

Prac­ti­cal ethics re­quires meta­phys­i­cal Free Will

Astra Kamratowski7 Apr 2022 14:47 UTC
2 points
1 comment13 min readEA link

Re­duc­ing Nu­clear Risk Through Im­proved US-China Relations

Metaculus21 Mar 2022 11:50 UTC
31 points
19 comments5 min readEA link

A quick and crude com­par­i­son of epi­demiolog­i­cal ex­pert fore­casts ver­sus Me­tac­u­lus fore­casts for COVID-19

Jotto2 Apr 2020 19:29 UTC
9 points
7 comments1 min readEA link

Me­tac­u­lus An­nounces The Million Pre­dic­tions Hackathon

christian10 Nov 2022 20:00 UTC
20 points
0 comments1 min readEA link
(metaculus.medium.com)

Top open Me­tac­u­lus forecasts

EA Handbook20 Jul 2022 23:00 UTC
14 points
1 comment1 min readEA link
(www.metaculus.com)

Prob­a­bil­ity of ex­tinc­tion for var­i­ous types of catastrophes

Vasco Grilo🔸9 Oct 2022 15:30 UTC
16 points
0 comments10 min readEA link

Pre­dic­tion Bank: A way around cur­rent pre­dic­tion mar­ket reg­u­la­tions?

gvst25 Jan 2022 4:21 UTC
25 points
10 comments1 min readEA link

[Question] What stops you do­ing more fore­cast­ing?

Nathan Young16 Nov 2021 0:26 UTC
11 points
6 comments1 min readEA link

Me­tac­u­lus Year in Re­view: 2022

christian6 Jan 2023 1:23 UTC
25 points
2 comments4 min readEA link
(metaculus.medium.com)

Me­tac­u­lus Launches Cli­mate Tip­ping Points Tour­na­ment With The Fed­er­a­tion of Amer­i­can Scientists

christian27 Jan 2023 19:33 UTC
21 points
2 comments1 min readEA link
(www.metaculus.com)

More Is Prob­a­bly More—Fore­cast­ing Ac­cu­racy and Num­ber of Fore­cast­ers on Metaculus

nikos31 Jan 2023 17:20 UTC
36 points
11 comments10 min readEA link

Fore­cast­ing Our World in Data: The Next 100 Years

AlexLeader1 Feb 2023 22:13 UTC
97 points
8 comments66 min readEA link
(www.metaculus.com)

[Question] Pre­dic­tive Perfor­mance on Me­tac­u­lus vs. Man­i­fold Markets

nikos3 Mar 2023 19:39 UTC
111 points
8 comments5 min readEA link

Fore­cast Which Psy­chol­ogy Stud­ies Repli­cate With Me­tac­u­lus for the Trans­par­ent Repli­ca­tions Project

christian29 Aug 2023 20:24 UTC
21 points
2 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Con­di­tional Cup to Ex­plore Linked Forecasts

christian18 Oct 2023 20:41 UTC
11 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus’s Cli­mate Tip­ping Points Tour­na­ment En­ters Round 2

christian16 Mar 2023 18:48 UTC
10 points
0 comments1 min readEA link
(www.metaculus.com)

[Event] Join Me­tac­u­lus for Fore­cast Fri­day on March 24th!

christian17 Mar 2023 22:47 UTC
8 points
0 comments1 min readEA link
(www.eventbrite.com)

Me­tac­u­lus’s Series ‘Shared Vi­sion: Pro Fore­caster Es­says on Pre­dict­ing the Fu­ture Bet­ter’

christian13 Jul 2023 1:24 UTC
16 points
0 comments1 min readEA link
(www.metaculus.com)

Take­aways from the Me­tac­u­lus AI Progress Tournament

Javier Prieto🔸27 Jul 2023 14:37 UTC
85 points
6 comments4 min readEA link

Pro­posal: Con­nect Me­tac­u­lus to the EA Fo­rum to In­cen­tivize Bet­ter Research

Damien Laird25 Mar 2023 12:13 UTC
19 points
5 comments8 min readEA link
(damienlaird.substack.com)

Pro­posal + Demo: Con­nect Guessti­mate and Me­tac­u­lus and Turn them into Trees

Charlie_Guthmann25 Mar 2023 17:15 UTC
15 points
0 comments3 min readEA link

[Event] Join Me­tac­u­lus To­mor­row, March 31st, for Fore­cast Fri­day!

christian30 Mar 2023 20:58 UTC
29 points
1 comment1 min readEA link
(www.metaculus.com)

Wis­dom of the Crowd vs. “the Best of the Best of the Best”

nikos4 Apr 2023 15:32 UTC
101 points
11 comments12 min readEA link

Me­tac­u­lus’s Keep Virginia Safe II Tour­na­ment En­ters 2nd Round

christian4 Apr 2023 21:51 UTC
11 points
0 comments1 min readEA link
(www.metaculus.com)

🔰Me­tac­u­lus Launches New Begin­ner Fore­cast­ing Tour­na­ment🔰

christian5 Apr 2023 20:08 UTC
21 points
0 comments1 min readEA link
(www.metaculus.com)

Pro­ject Idea: Pro­files Ag­gre­gat­ing Fore­cast­ing Perfor­mance Metrics

Damien Laird17 Apr 2023 10:29 UTC
2 points
2 comments4 min readEA link
(damienlaird.substack.com)

[Opz­ionale] Pre­vi­sioni aperte più popo­lari su Metaculus

EA Italy17 Jan 2023 20:15 UTC
1 point
0 comments1 min readEA link
(www.metaculus.com)

Share Your Feed­back and Help Us Refine Me­tac­u­lus’s Scor­ing Sys­tem

christian7 Aug 2023 23:09 UTC
15 points
0 comments1 min readEA link
(docs.google.com)

Me­tac­u­lus Launches 2023/​2024 FluSight Challenge Sup­port­ing CDC, $5K in Prizes

christian27 Sep 2023 21:35 UTC
9 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus An­nounces Win­ners of the Alt-Protein Fore­cast­ing Tournament

christian15 Sep 2023 17:59 UTC
25 points
0 comments1 min readEA link

Fore­cast­ing for Policy (FORPOL) - Main take­aways, prac­ti­cal learn­ings & report

janklenha18 Sep 2023 12:27 UTC
36 points
2 comments4 min readEA link

Me­tac­u­lus In­tro­duces Con­di­tional Con­tin­u­ous Ques­tions to Ex­plore Re­la­tion­ships Between Events

christian19 May 2023 19:24 UTC
14 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Pre­sents — View From the En­ter­prise Suite: How Ap­plied AI Gover­nance Works Today

christian20 Jun 2023 22:24 UTC
4 points
0 comments1 min readEA link

Me­tac­u­lus In­tro­duces Bet­ter Ques­tion Discovery

christian1 Mar 2024 3:24 UTC
5 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus In­tro­duces Mul­ti­ple Choice Questions

christian20 Dec 2023 19:00 UTC
8 points
1 comment1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Hosts ACX 2024 Pre­dic­tion Contest

christian1 Jan 2024 16:38 UTC
16 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Q1 2024 Quar­terly Cup for Cur­rent Events-Fo­cused, Fast-Re­solv­ing Questions

christian9 Jan 2024 4:28 UTC
6 points
0 comments1 min readEA link
(www.metaculus.com)

AI Bench­marks Series — Me­tac­u­lus Ques­tions on Eval­u­a­tions of AI Models Against Tech­ni­cal Benchmarks

christian27 Mar 2024 23:05 UTC
10 points
0 comments1 min readEA link
(www.metaculus.com)

‘Scale is All You Need’ — Wri­teup of a Syn­thetic Biol­ogy Work­shop With the Astera In­sti­tute & Metaculus

christian5 Sep 2024 20:11 UTC
6 points
1 comment5 min readEA link
(www.metaculus.com)