RSS

Metaculus

TagLast edit: Dec 1, 2022, 10:41 AM by brook

Metaculus is a reputation-based site for soliciting and aggregating predictions. It was founded in November 2015 by astrophysicist Anthony Aguirre, cosmologist Greg Laughlin and data scientist Max Wainwright.[1][2]

Tutorials

The official Metaculus tutorials[3] are suitable for beginners as a way to introduce forecasting, probability distributions, and a basic understanding of how to behave under uncertainty.

Tutorial #1 is over-explaining for those with a basic understanding of the forecasting, but still a good introduction to using Metaculus specifically.

For those familiar with forecasting, tutorials #2 and #3 are more useful as a (small) amount of calibration practice and building familiarity with the Metaculus interface.

A video introduction to Metaculus is also available[4].

Funding

As of August 2022, Metaculus has received $5.6 million in grants from Open Philanthropy,[5] and over $300,000 from Effective Altruism Funds.[6]

Further reading

Aguirre, Anthony (2021) A primer on the Metaculus scoring rule, Medium, February 27.

dan (2020) A preliminary look at Metaculus and expert forecasts, Metaculus, June 2.

Hobbhahn, Marius (2022) How good were our AI timeline predictions so far?, Metaculus, April 10.

External links

Metaculus official website

Metaculus official tutorials

Apply for a job

Related entries

AI forecasting | Forecasting | Prediction markets | Manifold markets

  1. ^

    Mann, Adam (2016) The power of prediction markets, Nature, October 18.

  2. ^
  3. ^

    Metaculus, Tutorials

  4. ^

    Brook, Victoria (2022), Metaculus Tutorial, Loom

  5. ^

    Open Philanthropy (2022) Grants database: Metaculus, Open Philanthropy.

  6. ^

    Effective Altruism Infrastructure Fund (2021) May-August 2021: EA Infrastructure Fund grants, Effective Altruism Funds, August.

Ex­plor­ing Me­tac­u­lus’ com­mu­nity predictions

Vasco Grilo🔸Mar 24, 2023, 7:59 AM
95 points
17 comments10 min readEA link

Launch­ing the AI Fore­cast­ing Bench­mark Series Q3 | $30k in Prizes

christianJul 8, 2024, 5:20 PM
17 points
0 comments1 min readEA link
(www.metaculus.com)

Analysing In­di­vi­d­ual Con­tri­bu­tions to the Me­tac­u­lus Com­mu­nity Prediction

nikosMay 8, 2023, 10:58 PM
28 points
1 comment12 min readEA link

Get­ting GPT-3 to pre­dict Me­tac­u­lus questions

MathiasKB🔸May 6, 2022, 12:12 PM
59 points
6 comments2 min readEA link

Fore­cast 2025 With Vox’s Fu­ture Perfect Team — $2,500 Prize Pool

christianDec 20, 2024, 11:00 PM
10 points
0 comments1 min readEA link
(www.metaculus.com)

Data on fore­cast­ing ac­cu­racy across differ­ent time hori­zons and lev­els of fore­caster experience

Charles Dillon 🔸May 27, 2021, 6:51 PM
125 points
7 comments23 min readEA link

AI Risk & Policy Fore­casts from Me­tac­u­lus & FLI’s AI Path­ways Workshop

Will AldredMay 16, 2023, 8:53 AM
41 points
0 comments8 min readEA link

Me­tac­u­lus’ pre­dic­tions are much bet­ter than low-in­for­ma­tion priors

Vasco Grilo🔸Apr 11, 2023, 8:36 AM
53 points
0 comments6 min readEA link

Me­tac­u­lus Launches Q4 Quar­terly Cup!

christianOct 9, 2023, 9:36 PM
8 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Space Tech­nol­ogy & Cli­mate Fore­cast­ing Ini­ti­a­tive

christianOct 11, 2023, 1:29 AM
11 points
1 comment1 min readEA link
(www.metaculus.com)

Me­tac­u­lus In­tro­duces New Fore­cast Scores, New Leader­board & Medals

christianNov 20, 2023, 8:33 PM
13 points
0 comments3 min readEA link
(www.metaculus.com)

Me­tac­u­lus is build­ing a team ded­i­cated to AI forecasting

christianOct 18, 2022, 4:08 PM
35 points
0 comments1 min readEA link
(apply.workable.com)

Me­tac­u­lus In­tro­duces New ‘Con­di­tional Pair’ Fore­cast Ques­tions for Mak­ing Con­di­tional Predictions

christianFeb 20, 2023, 1:36 PM
60 points
2 comments2 min readEA link
(www.metaculus.com)

Me­tac­u­lus An­nounces Fore­cast­ing Tour­na­ment to Eval­u­ate Fo­cused Re­search Or­ga­ni­za­tions, in Part­ner­ship With the Fed­er­a­tion of Amer­i­can Scien­tists

christianOct 3, 2023, 4:44 PM
21 points
0 comments2 min readEA link
(www.metaculus.com)

[Me­tac­u­lus Event] April 7 Fore­cast Fri­day: A Pro Fore­caster Pre­sents on Longevity Trends in G7 Coun­tries

christianApr 7, 2023, 2:01 AM
11 points
0 comments1 min readEA link
(www.metaculus.com)

An es­ti­mate of the value of Me­tac­u­lus questions

NunoSempereOct 22, 2021, 5:45 PM
45 points
19 comments8 min readEA link

Bot­tle­necks to more im­pact­ful crowd forecasting

eliflandDec 26, 2021, 10:48 PM
47 points
2 comments10 min readEA link
(www.foxy-scout.com)

Types of speci­fi­ca­tion prob­lems in forecasting

Juan GilJul 20, 2021, 4:17 AM
35 points
1 comment12 min readEA link

Ex­plor­ing Me­tac­u­lus’s AI Track Record

Peter ScoblicMay 1, 2023, 9:02 PM
52 points
5 comments5 min readEA link

Long-Term Fu­ture Fund: April 2019 grant recommendations

HabrykaApr 23, 2019, 7:00 AM
142 points
242 comments46 min readEA link

[Question] Me­tac­u­lus Ques­tions Suggest Money Will Do More Good in the Future

MichaelDickensJul 22, 2021, 1:56 AM
20 points
17 comments2 min readEA link

Me­tac­u­lus Launches Chi­nese AI Chips Tour­na­ment, Sup­port­ing In­sti­tute for AI Policy and Strat­egy Research

christianDec 6, 2023, 11:26 AM
27 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Fu­ture of AI Series, Based on Re­search Ques­tions by Arb

christianMar 13, 2024, 9:14 PM
34 points
0 comments1 min readEA link
(www.metaculus.com)

EA In­fras­truc­ture Fund: May–Au­gust 2021 grant recommendations

Max_DanielDec 24, 2021, 10:42 AM
85 points
19 comments21 min readEA link
(funds.effectivealtruism.org)

Im­pact­ful Fore­cast­ing Prize Re­sults and Reflections

eliflandMar 29, 2022, 4:16 PM
40 points
10 comments3 min readEA link

An ex­am­i­na­tion of Me­tac­u­lus’ re­solved AI pre­dic­tions and their im­pli­ca­tions for AI timelines

Charles Dillon 🔸Jul 20, 2021, 9:07 AM
81 points
2 comments7 min readEA link

Me­tac­u­lus Pre­dicts Weak AGI in 2 Years and AGI in 10

Chris LeongMar 24, 2023, 7:43 PM
27 points
12 comments1 min readEA link

6 Year De­crease of Me­tac­u­lus AGI Prediction

Chris LeongApr 12, 2022, 5:36 AM
40 points
6 comments1 min readEA link

An­nounc­ing the Nu­clear Risk Fore­cast­ing Tournament

MichaelA🔸Jun 16, 2021, 4:12 PM
38 points
0 comments2 min readEA link

Long-Term Fu­ture Fund: April 2020 grants and recommendations

HabrykaSep 18, 2020, 10:28 AM
40 points
7 comments9 min readEA link
(app.effectivealtruism.org)

Com­pe­ti­tion for “For­tified Es­says” on nu­clear risk

MichaelA🔸Nov 17, 2021, 8:55 PM
35 points
0 comments3 min readEA link
(www.metaculus.com)

900+ Fore­cast­ers on Whether Rus­sia Will In­vade Ukraine

MetaculusFeb 19, 2022, 1:29 PM
51 points
0 comments4 min readEA link
(metaculus.medium.com)

Event on Oct 9: Fore­cast­ing Nu­clear Risk with Re­think Pri­ori­ties’ Michael Aird

MichaelA🔸Sep 29, 2021, 5:45 PM
24 points
3 comments2 min readEA link
(www.eventbrite.com)

[Me­tac­u­lus Event] April 14 Fore­cast Fri­day: A Pro Fore­caster on Shift­ing Ter­ri­to­rial Con­trol in Ukraine

christianApr 14, 2023, 12:40 AM
5 points
0 comments1 min readEA link

Rus­sia-Ukraine Con­flict: Fore­cast­ing Nu­clear Risk in 2022

MetaculusMar 24, 2022, 9:03 PM
23 points
1 comment12 min readEA link

Im­pact­ful Fore­cast­ing Prize for fore­cast write­ups on cu­rated Me­tac­u­lus questions

eliflandFeb 4, 2022, 8:06 PM
91 points
13 comments4 min readEA link

“Nu­clear risk re­search, fore­cast­ing, & im­pact” [pre­sen­ta­tion]

MichaelA🔸Oct 21, 2021, 10:54 AM
20 points
0 comments1 min readEA link
(www.youtube.com)

[Question] What im­por­tant ques­tions are miss­ing from Me­tac­u­lus?

Charles Dillon 🔸May 26, 2021, 2:03 PM
38 points
12 comments1 min readEA link

An anal­y­sis of Me­tac­u­lus pre­dic­tions of fu­ture EA re­sources, 2025 and 2030

Charles Dillon 🔸Sep 22, 2021, 10:24 AM
50 points
2 comments8 min readEA link

Me­tac­u­lus Pre­sents: Does Gen­er­a­tive AI In­fringe Copy­right?

christianNov 6, 2023, 11:41 PM
5 points
0 comments1 min readEA link

Me­tac­u­lus Pre­sents: Trans­for­ma­tive Science at Startup Speed

christianOct 31, 2023, 9:12 PM
5 points
0 comments1 min readEA link

Me­tac­u­lus Launches the ‘Fore­cast­ing Our World In Data’ Pro­ject to Probe the Long-Term Future

christianOct 14, 2022, 5:00 PM
65 points
6 comments1 min readEA link
(www.metaculus.com)

An­nounc­ing the AI Fore­cast­ing Bench­mark Series | July 8, $120k in Prizes

christianJun 19, 2024, 9:37 PM
52 points
4 comments5 min readEA link
(www.metaculus.com)

Open Sourc­ing Metaculus

christianJun 25, 2024, 6:40 PM
75 points
4 comments2 min readEA link
(www.metaculus.com)

An­nounc­ing the Ben­tham Prize

PabloJan 21, 2020, 10:23 PM
33 points
1 comment1 min readEA link

Me­tac­u­lus Begin­ner Tour­na­ment for New Forecasters

AnastasiaJan 6, 2023, 2:35 AM
33 points
5 comments1 min readEA link

Fore­cast­ing Newslet­ter: Fe­bru­ary 2022

NunoSempereMar 5, 2022, 7:16 PM
25 points
2 comments9 min readEA link

Pre­dict­ing Poly­genic Selec­tion for IQ

MetaculusMar 28, 2022, 6:00 PM
41 points
4 comments20 min readEA link

Com­par­ing Fore­cast­ing Track Records for AI Bench­mark­ing and Beyond

Tom LiptaySep 25, 2024, 9:01 PM
14 points
1 comment19 min readEA link
(www.metaculus.com)

De­sign­ing Ar­tifi­cial Wis­dom: De­ci­sion Fore­cast­ing AI & Futarchy

Jordan ArelJul 14, 2024, 5:10 AM
5 points
1 comment6 min readEA link

Me­tac­u­lus World Map Experiment

Ryan BeckJul 16, 2024, 6:19 PM
20 points
1 comment1 min readEA link
(www.metaculus.com)

Can AI Out­pre­dict Hu­mans? Re­sults From Me­tac­u­lus’s Q3 AI Fore­cast­ing Benchmark

Tom LiptayOct 10, 2024, 6:58 PM
32 points
1 comment6 min readEA link
(www.metaculus.com)

Me­tac­u­lus’s ‘Mini­tac­u­lus’ Ex­per­i­ments — Col­lab­o­rate With Us

christianAug 26, 2024, 8:44 PM
10 points
0 comments1 min readEA link
(www.metaculus.com)

OpenAI and An­thropic Donate Cred­its for AI Fore­cast­ing Bench­mark Tournament

christianJul 17, 2024, 9:50 PM
2 points
0 comments1 min readEA link

Launch­ing the Re­s­pi­ra­tory Out­look 2024/​25 Fore­cast­ing Series

christianJul 17, 2024, 7:51 PM
8 points
0 comments1 min readEA link
(www.metaculus.com)

Pre­dic­tions as Public Works Pro­ject — What Me­tac­u­lus Is Build­ing Next

SylvainOct 22, 2024, 4:35 PM
24 points
4 comments5 min readEA link
(www.metaculus.com)

Fore­cast With GiveWell

christianOct 23, 2024, 7:50 PM
35 points
6 comments2 min readEA link
(www.metaculus.com)

Bridge­wa­ter x Me­tac­u­lus Fore­cast­ing Con­test Goes Global — Feb 3, $25k, Opportunities

christianJan 7, 2025, 9:40 PM
6 points
0 comments1 min readEA link
(www.metaculus.com)

AI Fore­cast­ing Bench­mark: Con­grat­u­la­tions to Q4 Win­ners + Q1 Prac­tice Ques­tions Open

christianJan 10, 2025, 3:02 AM
6 points
0 comments2 min readEA link
(www.metaculus.com)

Fore­casts about EA or­gani­sa­tions which are cur­rently on Me­tac­u­lus.

alex lawsenDec 29, 2020, 5:42 PM
65 points
4 comments3 min readEA link

Col­lab­o­ra­tive In­tel­li­gence: Har­ness­ing Crowd Fore­cast­ing for Na­tional Security

Deger TuranDec 9, 2024, 10:12 PM
6 points
1 comment4 min readEA link
(fas.org)

[Question] What ques­tions would you like to see fore­casts on from the Me­tac­u­lus com­mu­nity?

alex lawsenJul 26, 2020, 2:40 PM
32 points
30 comments2 min readEA link

An­nual AGI Bench­mark­ing Event

MetaculusAug 26, 2022, 9:31 PM
20 points
2 comments2 min readEA link
(www.metaculus.com)

Me­tac­u­lus seek­ing An­a­lyt­i­cal Sto­ry­tel­lers to write es­says for­tified with testable predictions

christianOct 6, 2021, 4:39 AM
16 points
4 comments1 min readEA link

Me­tac­u­lus is seek­ing ex­pe­rienced lead­ers, re­searchers & op­er­a­tors for high-im­pact roles

christianJul 10, 2022, 2:29 PM
13 points
0 comments1 min readEA link
(apply.workable.com)

[job] Me­tac­u­lus has new soft­ware roles

dschwarzNov 7, 2022, 9:19 PM
9 points
0 comments1 min readEA link
(apply.workable.com)

Me­tac­u­lus is hiring

aaguirreDec 9, 2020, 8:58 PM
30 points
0 comments1 min readEA link

How does fore­cast quan­tity im­pact fore­cast qual­ity on Me­tac­u­lus?

Charles Dillon 🔸Oct 1, 2021, 6:57 PM
60 points
14 comments8 min readEA link

Prac­ti­cal ethics re­quires meta­phys­i­cal Free Will

Astra KamratowskiApr 7, 2022, 2:47 PM
2 points
1 comment13 min readEA link

Re­duc­ing Nu­clear Risk Through Im­proved US-China Relations

MetaculusMar 21, 2022, 11:50 AM
31 points
19 comments5 min readEA link

A quick and crude com­par­i­son of epi­demiolog­i­cal ex­pert fore­casts ver­sus Me­tac­u­lus fore­casts for COVID-19

JottoApr 2, 2020, 7:29 PM
9 points
7 comments1 min readEA link

Me­tac­u­lus An­nounces The Million Pre­dic­tions Hackathon

christianNov 10, 2022, 8:00 PM
20 points
0 comments1 min readEA link
(metaculus.medium.com)

Top open Me­tac­u­lus forecasts

EA HandbookJul 20, 2022, 11:00 PM
14 points
1 comment1 min readEA link
(www.metaculus.com)

Prob­a­bil­ity of ex­tinc­tion for var­i­ous types of catastrophes

Vasco Grilo🔸Oct 9, 2022, 3:30 PM
16 points
0 comments10 min readEA link

Pre­dic­tion Bank: A way around cur­rent pre­dic­tion mar­ket reg­u­la­tions?

gvstJan 25, 2022, 4:21 AM
25 points
10 comments1 min readEA link

[Question] What stops you do­ing more fore­cast­ing?

Nathan YoungNov 16, 2021, 12:26 AM
11 points
6 comments1 min readEA link

Me­tac­u­lus Year in Re­view: 2022

christianJan 6, 2023, 1:23 AM
25 points
2 comments4 min readEA link
(metaculus.medium.com)

Me­tac­u­lus Launches Cli­mate Tip­ping Points Tour­na­ment With The Fed­er­a­tion of Amer­i­can Scientists

christianJan 27, 2023, 7:33 PM
21 points
2 comments1 min readEA link
(www.metaculus.com)

More Is Prob­a­bly More—Fore­cast­ing Ac­cu­racy and Num­ber of Fore­cast­ers on Metaculus

nikosJan 31, 2023, 5:20 PM
36 points
11 comments10 min readEA link

Fore­cast­ing Our World in Data: The Next 100 Years

AlexLeaderFeb 1, 2023, 10:13 PM
97 points
8 comments66 min readEA link
(www.metaculus.com)

[Question] Pre­dic­tive Perfor­mance on Me­tac­u­lus vs. Man­i­fold Markets

nikosMar 3, 2023, 7:39 PM
111 points
8 comments5 min readEA link

Fore­cast Which Psy­chol­ogy Stud­ies Repli­cate With Me­tac­u­lus for the Trans­par­ent Repli­ca­tions Project

christianAug 29, 2023, 8:24 PM
21 points
2 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Con­di­tional Cup to Ex­plore Linked Forecasts

christianOct 18, 2023, 8:41 PM
11 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus’s Cli­mate Tip­ping Points Tour­na­ment En­ters Round 2

christianMar 16, 2023, 6:48 PM
10 points
0 comments1 min readEA link
(www.metaculus.com)

[Event] Join Me­tac­u­lus for Fore­cast Fri­day on March 24th!

christianMar 17, 2023, 10:47 PM
8 points
0 comments1 min readEA link
(www.eventbrite.com)

Me­tac­u­lus’s Series ‘Shared Vi­sion: Pro Fore­caster Es­says on Pre­dict­ing the Fu­ture Bet­ter’

christianJul 13, 2023, 1:24 AM
16 points
0 comments1 min readEA link
(www.metaculus.com)

Take­aways from the Me­tac­u­lus AI Progress Tournament

Javier Prieto🔸Jul 27, 2023, 2:37 PM
85 points
6 comments4 min readEA link

Pro­posal: Con­nect Me­tac­u­lus to the EA Fo­rum to In­cen­tivize Bet­ter Research

Damien LairdMar 25, 2023, 12:13 PM
19 points
5 comments8 min readEA link
(damienlaird.substack.com)

Pro­posal + Demo: Con­nect Guessti­mate and Me­tac­u­lus and Turn them into Trees

Charlie_GuthmannMar 25, 2023, 5:15 PM
15 points
0 comments3 min readEA link

[Event] Join Me­tac­u­lus To­mor­row, March 31st, for Fore­cast Fri­day!

christianMar 30, 2023, 8:58 PM
29 points
1 comment1 min readEA link
(www.metaculus.com)

Wis­dom of the Crowd vs. “the Best of the Best of the Best”

nikosApr 4, 2023, 3:32 PM
101 points
11 comments12 min readEA link

Me­tac­u­lus’s Keep Virginia Safe II Tour­na­ment En­ters 2nd Round

christianApr 4, 2023, 9:51 PM
11 points
0 comments1 min readEA link
(www.metaculus.com)

🔰Me­tac­u­lus Launches New Begin­ner Fore­cast­ing Tour­na­ment🔰

christianApr 5, 2023, 8:08 PM
21 points
0 comments1 min readEA link
(www.metaculus.com)

Pro­ject Idea: Pro­files Ag­gre­gat­ing Fore­cast­ing Perfor­mance Metrics

Damien LairdApr 17, 2023, 10:29 AM
2 points
2 comments4 min readEA link
(damienlaird.substack.com)

[Opz­ionale] Pre­vi­sioni aperte più popo­lari su Metaculus

EA ItalyJan 17, 2023, 8:15 PM
1 point
0 comments1 min readEA link
(www.metaculus.com)

Share Your Feed­back and Help Us Refine Me­tac­u­lus’s Scor­ing Sys­tem

christianAug 7, 2023, 11:09 PM
15 points
0 comments1 min readEA link
(docs.google.com)

Me­tac­u­lus Launches 2023/​2024 FluSight Challenge Sup­port­ing CDC, $5K in Prizes

christianSep 27, 2023, 9:35 PM
9 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus An­nounces Win­ners of the Alt-Protein Fore­cast­ing Tournament

christianSep 15, 2023, 5:59 PM
25 points
0 comments1 min readEA link

Fore­cast­ing for Policy (FORPOL) - Main take­aways, prac­ti­cal learn­ings & report

janklenhaSep 18, 2023, 12:27 PM
36 points
2 comments4 min readEA link

Me­tac­u­lus In­tro­duces Con­di­tional Con­tin­u­ous Ques­tions to Ex­plore Re­la­tion­ships Between Events

christianMay 19, 2023, 7:24 PM
14 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Pre­sents — View From the En­ter­prise Suite: How Ap­plied AI Gover­nance Works Today

christianJun 20, 2023, 10:24 PM
4 points
0 comments1 min readEA link

Me­tac­u­lus In­tro­duces Bet­ter Ques­tion Discovery

christianMar 1, 2024, 3:24 AM
5 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus In­tro­duces Mul­ti­ple Choice Questions

christianDec 20, 2023, 7:00 PM
8 points
1 comment1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Hosts ACX 2024 Pre­dic­tion Contest

christianJan 1, 2024, 4:38 PM
16 points
0 comments1 min readEA link
(www.metaculus.com)

Me­tac­u­lus Launches Q1 2024 Quar­terly Cup for Cur­rent Events-Fo­cused, Fast-Re­solv­ing Questions

christianJan 9, 2024, 4:28 AM
6 points
0 comments1 min readEA link
(www.metaculus.com)

AI Bench­marks Series — Me­tac­u­lus Ques­tions on Eval­u­a­tions of AI Models Against Tech­ni­cal Benchmarks

christianMar 27, 2024, 11:05 PM
10 points
0 comments1 min readEA link
(www.metaculus.com)

‘Scale is All You Need’ — Wri­teup of a Syn­thetic Biol­ogy Work­shop With the Astera In­sti­tute & Metaculus

christianSep 5, 2024, 8:11 PM
6 points
1 comment5 min readEA link
(www.metaculus.com)

Me­tac­u­lus Is Open Source

christianOct 7, 2024, 7:55 PM
33 points
1 comment1 min readEA link
(www.metaculus.com)