I’m building tools to make sense of the future at Sage. Currently building AI Digest, Fatebook and Quantified Intuitions.
Previously I was doing a PhD in HCI at St Andrews, and worked at Clearer Thinking.
Website: https://binksmith.com
Tweeting, sometimes about EA: https://twitter.com/adambinksmith
In many ways Fatebook is a successor to PredictionBook (now >11 years old!) If you’ve used PredictionBook in the past, you can import all your PredictionBook questions and scores to Fatebook.
In a perfect world, this would also integrate with Alfred on my mac so that it becomes extremely easy and quick to create a new private question
I’m thinking of creating a Chrome extension that will let you type /forecast Will x happen? anywhere on the internet, and it’ll create and embed an interactive Fatebook question. EDIT: we created this, the Fatebook browser extension.
I’m thinking of primarily focussing on Google Docs, because I think the EA community could get a lot of mileage out of making and tracking predictions embedded in reports, strategy docs, etc. This extension would also work in messaging apps, on social media, and even here on the forum (though first-party support might be better for the forum!).
Great, thanks!
The format could be “[question text]? [resolve date]” where the question mark serves as the indicator for the end of the question text, and the resolve date part can interpret things like “1w”, “1y”, “eoy”, “5d”
I’m interested in adding power user shortcuts like this!
Currently, if your question text includes a date that Fatebook can recognise, it’ll prepopulate the “Resolve by” field with that date. This works for a bunch of common phrases, e.g. “in two weeks” “by next month” “by Jan 2025″ “by February” “by tomorrow”.
If you play around with the site, I’d be interested to hear if you find yourself still keen for the addition of concise shortcuts like “2w” or if the current natural language date parsing works well for you.
The June Estimation Game is animal welfare + alt proteins themed! 10 Fermi estimation questions. You can play here: quantifiedintuitions.org/estimation-game/june
Seems like academic research groups would be a better reference class than YC companies for most alignment labs.
If they’re trying to build an org that scales a lot, and is funded by selling products, YC companies is a good reference class, but if they’re an org of researchers working somewhat independently or collaborating on hard technical problems, funded by grants, that sounds much more similar to an academic research group.
Unsure how to define success for an academic research group, any ideas? They seem to more often be exploratory and less goal-oriented.
Hmm, I’m not aware of a way to do this (but there might be one). Maybe you could generate two versions of the deck from your orgmode file, one with the Anki with Uncertainty card types and the other with plain card types?
I’m excited to see the return of the careers guide as the core 80k resource (vs the key ideas series)! I think it’s a better way to provide value to people, because a careers guide is about the individual (“how can I think about what to do with my career?”) rather than about 80k (“what are the key ideas of 80k/EA?”)
Great :)
Nice! Thanks for the heads up Elliot—which page are you seeing a missing certificate on? Seems to be working for me
I’ve added a basic calibration curve, thanks for the suggestion!
You can find it in the app’s Home tab (click on Fatebook in the left sidebar > Home tab at the top) once at least one question you’ve forecasted on has resolved.
Great, glad to hear it!
Aggregation choices (e.g. geo mean of odds would be nice)
Geo mean of odds is a good idea—it’s probably a more sensible default. How would you feel about us using that everywhere, instead of the current arithmetic mean?
Brier scores for users
You can see your own absolute and relative Brier score in the app home (click Fatebook in the sidebar). If you’re thinking of a team-wide leaderboard—that’s on our list! Though some users said they wouldn’t like this to avoid Goodharting, so I’ve not prioritised it so far, and will include a team-wide toggle if we add it.
Calibration curves for users
We’ll add this soon!
I think you could implement a spaced repetition feature based on how many orders of magnitude you’re off, where the more OOMs you’re off, the earlier it prompts you with the same question again
This is a great idea, so we made Anki with Uncertainty to do exactly this!
Thank you Hauke for the suggestion :D
I think we’ll keep the calibration app as a pure calibration training game, where you see each question only once. Anki is already the king of spaced repetition, so adding calibration features to it seemed like a natural fit.
Super interesting to see this analysis, especially the table of current capabilities—thank you!
I have interpreted [feasible] as, one year after the forecasted date, have AI labs achieved these milestones, and disclosed this publicly?
It seems to me that this ends up being more conservative than the original “Ignore the question of whether they would choose to” , which presumably makes the expert forecasts worse than they seem to be here.
For example, a task like “win angry birds” seems pretty achievable to me, just that no one’s thinking about angry birds these days so it probably hasn’t been attempted. Does that sound right to you?
I’m curious if you have a rough estimate of how many of these tasks would be achievable within a year if top labs attempted them?
Thanks for the feedback Forslack! I’m curious whether you’d prefer to play without logging in because you don’t have a Google account or because you don’t want to share your email?
Thank you! I’m interested to hear how you find it!
Very relatable! The 10 Conditions for Change framework might be helping for thinking of ways to do it more consistently (if on reflection you really want to!) Fatebook aims to help with 1, 2, 4, 7, and 8, I think.
One way to do more prediction I’m interested in is integrating prediction into workflows. Here are some made-up examples:
At the start of a work project, you always forecast how long it’ll take (I think this is almost always an important question, and getting good at predicting this is powerful)
When you notice you’re concerned about some uncertainty (e.g. some risk) you operationalise it and write it down as a question
In your weekly review with your manager, you make forecasts about how likely you are to meet each of your goals. Then you discuss strategies to raise the P(success) on the important goals
When there’s a disagreement two team members about what to prioritise, you make operationalise it as a forecasting question, and get the whole team’s view. If the team as a whole disagrees, you look for ways to get more information, or if the team agrees (after sharing info) you follow that prioitisation
If anyone that either has prediction as part of workflows or would like to do so would be interested in chatting, lmk!