On your main point, this was the kind of thing we were trying to make clearer, so it’s disappointing that hasn’t come through.
Just on the particular VC example:
I’m suspicious you can do a good job of predicting ex ante outcomes. After all, that’s what VCs would want to do and they have enormous resources. Their strategy is basically to pick as many plausible winners as they can fund.
YC having a low acceptance rate could mean they are highly confident in their ability to predict ex ante outcomes. It could also mean that they get a lot of unserious applications. Essays such as this one by Paul Graham bemoaning the difficulty of predicting ex ante outcomes make me think it is more the latter. (“it’s mostly luck once you get down to the top 1-5%” makes it sound to me like ultra-successful startups should have elite founders, but my take on Graham’s essay is that ultra-successful startups tend to be unusual, often in a way that makes them look non-elite according to traditional metrics—I tend to suspect this is true of exceptionally innovative people more generally)
I’m not trying to be obtuse, it wasn’t super clear to me on a quick-ish skim; maybe if I’d paid more attention I’ve have clocked it.
Yup, I was too hasty on VCs. It seems like they are pretty confident they know what the top >5% are, but not that can say anything more precise than. (Although I wonder what evidence indicates they can reliably tell the top 5% from those below, rather than they just think they can).
(Although I wonder what evidence indicates they can reliably tell the top 5% from those below, rather than they just think they can).
The Canadian inventors assistance program provides a rating of how good an invention is to inventors for a nominal fee. A large fraction of the people who get a bad rating try to make a company anyway, so we can judge the accuracy of their evaluations.
55% of the inventions which they give the highest rating to achieve commercial success, compared to 0% for the lowest rating.
ah, this is great. evidence the selectors could tell the top 2% from the rest, but 2%-20% was much of a muchness. Shame that it doesn’t give any more information on ‘commercial success’.
I’m not trying to be obtuse, it wasn’t super clear to me on a quick-ish skim; maybe if I’d paid more attention I’ve have clocked it.
FWIW I think it’s the authors’ job to anticipate how their audience is going to engage with their writing, where they’re coming from etc. - You were not the only one who reacted by pushing back against our framing as evident e.g. from Khorton’s much upvoted comment.
So no matter what we tried to convey, and what info is in the post or document if one reads closely enough, I think this primarily means that I (as main author of the wording in the post) could have done a better job, not that you or anyone else is being obtuse.
On your main point, this was the kind of thing we were trying to make clearer, so it’s disappointing that hasn’t come through.
Just on the particular VC example:
Most VCs only pick from the top 1-5% of startups. E.g. YC’s acceptance rate is 1%, and very few startups they reject make it to series A. More data on VC acceptance rates here: https://80000hours.org/2014/06/the-payoff-and-probability-of-obtaining-venture-capital/
So, I think that while it’s mostly luck once you get down to the top 1-5%, I think there’s a lot of predictors before that.
Also see more on predictors of startup performance here: https://80000hours.org/2012/02/entrepreneurship-a-game-of-poker-not-roulette/
YC having a low acceptance rate could mean they are highly confident in their ability to predict ex ante outcomes. It could also mean that they get a lot of unserious applications. Essays such as this one by Paul Graham bemoaning the difficulty of predicting ex ante outcomes make me think it is more the latter. (“it’s mostly luck once you get down to the top 1-5%” makes it sound to me like ultra-successful startups should have elite founders, but my take on Graham’s essay is that ultra-successful startups tend to be unusual, often in a way that makes them look non-elite according to traditional metrics—I tend to suspect this is true of exceptionally innovative people more generally)
Hello Ben.
I’m not trying to be obtuse, it wasn’t super clear to me on a quick-ish skim; maybe if I’d paid more attention I’ve have clocked it.
Yup, I was too hasty on VCs. It seems like they are pretty confident they know what the top >5% are, but not that can say anything more precise than. (Although I wonder what evidence indicates they can reliably tell the top 5% from those below, rather than they just think they can).
The Canadian inventors assistance program provides a rating of how good an invention is to inventors for a nominal fee. A large fraction of the people who get a bad rating try to make a company anyway, so we can judge the accuracy of their evaluations.
55% of the inventions which they give the highest rating to achieve commercial success, compared to 0% for the lowest rating.
https://www.researchgate.net/publication/227611370_Profitable_Advice_The_Value_of_Information_Provided_by_Canadas_Inventors_Assistance_Program
ah, this is great. evidence the selectors could tell the top 2% from the rest, but 2%-20% was much of a muchness. Shame that it doesn’t give any more information on ‘commercial success’.
This is amazing data, and not what I would have expected—I’ve just had my mind changed on the predictability of invention success. Thanks!
This is really cool, thank you!
That’s very interesting, thanks for sharing!
ETA: I’ve added this to our doc acknowledging your comment.
FWIW I think it’s the authors’ job to anticipate how their audience is going to engage with their writing, where they’re coming from etc. - You were not the only one who reacted by pushing back against our framing as evident e.g. from Khorton’s much upvoted comment.
So no matter what we tried to convey, and what info is in the post or document if one reads closely enough, I think this primarily means that I (as main author of the wording in the post) could have done a better job, not that you or anyone else is being obtuse.