Thanks. Fixed.
Hmash
Karma: 639
EffectiveAltruismData.com: A Website for Aggregating and Visualising EA Data
Statistics for Lazy People, Part 2
Thank you. I have corrected the mistake.
The relationship between Lindy, Doomsday, and Copernicus is as follows:
The “Copernican Principle” is that “we” are not special. This is a generalisation of how the Earth is not special: it’s just another planet in the solar system, not the centre of the universe.
In John Gott’s famous paper on the Doomsday Argument, he appeals to the the Copernican Principle to assert “we are also not special in time”, meaning that we should expect ourselves to be in a typical point in the history of humanity.
The “most typical” point in history is exactly in the middle. Thus your best guess of the longevity of humanity is twice its current age: Lindy’s Law.
Statistics for Lazy People, Part 1
This is brilliant!
I think we can actually do an explicit expected-utility and value-of-information calculation here:
Let one five-star book = one util
Each book’s quality can be modelled as a rate of producing stars.
The star rating you give a book is the sum of 5 Bernoulli trials with rate .
The book will produce utils of value per read in expectation.
To estimate , sum up the total stars awarded and total possible stars .
The probability distribution is then (assuming uniform prior for simplicity).
For any pair of books, we can compute the probability that book 1 is more valuable than book 2 as .
Let’s say there’s a prescribed EA reading list.
Let people who encounter the list be probabilistic automata.
These automata start at the top of the list, then iteratively either: 1) read the book they are currently looking at, 2) move down to the next item on the list, 3) quit.
Intuitively, I think this process will result in books being read geometrically less as you move down the list.
For simplicity, let’s say the first book is guaranteed to be read, the next book has a 50% chance of being read, then 25%, …, and then -th book has chance of being read (with starting at zero).
The expected value of the list is then
To calculate the value of information for reading a given book, you enumerate all the possible outcomes (one-star, two-stars, …., five-stars), calculate the probability of each one, look at how the rankings would change, and re-calculate the the expected value of the list. Multiply the expected values by the probabilities et voila.
Can I get the data please?
It just occurred to me that you don’t actually need to convert the forecaster’s odds to bits. You can just take the ceiling of the odds themselves:
Which is more useful for calibrating in the low-confidence range.
Additional note: BitBets is a proper scoring rule, but not strictly proper. If you round report odds which are rounded up to the next power of two you will achieve the same scores in expectation.
Thanks for the insightful comments.
One other thought I’ve had in this area is “auctioning” predictions, so that whoever is willing to assign the most bits of confidence to their prediction (the highest “bitter”) gets exclusive payoffs. I’m not sure what this would be useful for though. Maybe you award a contract for some really important task to whoever is most confident they will be able to fulfill the contract.
BitBets: A Simple Scoring System for Forecaster Training
Let me know if you do. I’d be keen to help!
Very cute. 🙂
I’m curious about your thinking on colour symbolism. On the one hand, ravens are smart and crafty, so “black bird = smart/strategic bird” makes sense. But on the other hand, blue is kinda an EA colour, so at first I thought the blue bird would represent EA. Why did you choose to make the lay-bird a blue bird?