For example, they asked people to give 90% confidence intervals for various things, and recorded how frequently the true answer was outside that range. This seems to me to be a legitimate definition of “overconfident”.
I agree that this is a better measure of overconfidence. But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
That said, I also agree that startup founders tend to be unrealistically optimistic about their own prospects (or at least answer survey questions in ways that are unreallistically optimistic).
To me a good formalization of “luck” is “ex-ante variance”
Suppose that each startup’s valuation is a deterministic function of the founder’s skill (which is known to the founder). The we expect startup founders to be paid mostly in equity. And we could also see great variation in founder compensation. But I think we would agree that there is no luck in this case—it seems like exactly the kind of model you are trying to argue against.
Factor 3′s non-significance tells us that, if there is such a thing as founding skill it’s not a skill which improves through practice.
I’m skeptical that the study has enough power to detect a realistic effect for this factor, given that it’s a small piece of human capital and all of the measures are extremely noisy.
I also have concerns about confounding. For example, prior foundings had the largest negative coefficient of any variable on VC funding. The author’s interpretation is: “presidents who have experienced entrepreneurial failures in the past may find it more difficult to obtain VC financing for future startups.” I don’t know if that’s the real explanation, but I’m pretty hesitant to make causal inferences in this case—whatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
but it seems weird to assume perfect competition here?
I expect the real world is somewhere in between perfect competition and no competition.
But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
No, you are correct and I misread your statement. Entrepreneurs don’t appear to have this sort of overconfidence (at least according to this study’s results). They (we) appeared to have more self aggrandizing confidence.
But I think we would agree that there is no luck in this case—it seems like exactly the kind of model you are trying to argue against.
Cool, I thought you might be making this point but I didn’t want to put words in your mouth.
It’s unquestionably true that part of the variance is explained by the fact that any jackass can call themselves an entrepreneur and therefore the set of “entrepreneurs” has a greater range of skills than e.g. the set of software developers. But I’m skeptical that this is the entire reason; to cite a fact from above: an entrepreneur who has a successful exit only has a 30% chance of success in their next venture. It seems unlikely that entrepreneurs’ skills decline rapidly after a successful exit.
whatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
I would agree with this more generally: there are probably modest skill effects, but they are incredibly hard to define and are swamped by the nosiness of startups. Especially once you look at “qualified” founders (e.g. those with venture backing), the skill differentiation explains a very small piece of the variance.
Quoth Gompers et al.:
“While it may be better to be lucky than smart, the evidence presented here indicates that being smart has value too.”
I expect the real world is somewhere in between perfect competition and no competition.
Hard to argue with that :-)
I’m still not entirely sure what lesson you were drawing from the efficient market hypothesis, but perhaps it doesn’t matter.
Thanks for engaging with objections, and sorry for being so critical!
It’s possible we just have a quantitative disagreement.
For example, I agree that there are very few people who could start a successful startup with high reliability. But quantitatively, I don’t know how much variation in skill there is and how important it is. I think the most compelling statistic here is the 30% IPO rate for second startup (given success) vs. 18% IPO rate for first startups. But that seems to me like a pretty big effect, so I’m not sure quite what to make of it.
I think you could just as well argue “Submitting an academic paper to a good conference is a lottery ticket.” (In fact, the numbers are comparable). In some sense this is true, but I still wouldn’t say “The idea that there are some excellent papers is just factually wrong.”
Thanks Paul for the feedback, and for the reminder that we are criticizing ideas and not people :-)
The thing with academic papers was really interesting, and gave me pause. I would point out two similarities:
The set of all papers submitted to a specific conference is a lot more homogeneous than the set of all papers period. Similarly, the set of all entrepreneurs who get VC funding is a lot more homogenous than the set of all people who think about starting companies. So the statement “performance within some limited subset is mostly due to chance” isn’t necessarily conflicting with the idea that there is such a thing as entrepreneurial skill/paper quality.
Instead of drawing a lesson that there is no such thing as skill we might conclude that acceptance to a conference or having an IPO is just not a very good indicator of skill.
I also agree that this is largely a quantitative disagreement. I’ve spent the last year being surrounded by people who believe that variance in startups is completely determined by the founder’s skill, and that gives me a framing for what I write.
I agree that this is a better measure of overconfidence. But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
That said, I also agree that startup founders tend to be unrealistically optimistic about their own prospects (or at least answer survey questions in ways that are unreallistically optimistic).
Suppose that each startup’s valuation is a deterministic function of the founder’s skill (which is known to the founder). The we expect startup founders to be paid mostly in equity. And we could also see great variation in founder compensation. But I think we would agree that there is no luck in this case—it seems like exactly the kind of model you are trying to argue against.
I’m skeptical that the study has enough power to detect a realistic effect for this factor, given that it’s a small piece of human capital and all of the measures are extremely noisy.
I also have concerns about confounding. For example, prior foundings had the largest negative coefficient of any variable on VC funding. The author’s interpretation is: “presidents who have experienced entrepreneurial failures in the past may find it more difficult to obtain VC financing for future startups.” I don’t know if that’s the real explanation, but I’m pretty hesitant to make causal inferences in this case—whatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
I expect the real world is somewhere in between perfect competition and no competition.
No, you are correct and I misread your statement. Entrepreneurs don’t appear to have this sort of overconfidence (at least according to this study’s results). They (we) appeared to have more self aggrandizing confidence.
Cool, I thought you might be making this point but I didn’t want to put words in your mouth.
It’s unquestionably true that part of the variance is explained by the fact that any jackass can call themselves an entrepreneur and therefore the set of “entrepreneurs” has a greater range of skills than e.g. the set of software developers. But I’m skeptical that this is the entire reason; to cite a fact from above: an entrepreneur who has a successful exit only has a 30% chance of success in their next venture. It seems unlikely that entrepreneurs’ skills decline rapidly after a successful exit.
I would agree with this more generally: there are probably modest skill effects, but they are incredibly hard to define and are swamped by the nosiness of startups. Especially once you look at “qualified” founders (e.g. those with venture backing), the skill differentiation explains a very small piece of the variance.
Quoth Gompers et al.:
“While it may be better to be lucky than smart, the evidence presented here indicates that being smart has value too.”
Hard to argue with that :-)
I’m still not entirely sure what lesson you were drawing from the efficient market hypothesis, but perhaps it doesn’t matter.
Thanks for engaging with objections, and sorry for being so critical!
It’s possible we just have a quantitative disagreement.
For example, I agree that there are very few people who could start a successful startup with high reliability. But quantitatively, I don’t know how much variation in skill there is and how important it is. I think the most compelling statistic here is the 30% IPO rate for second startup (given success) vs. 18% IPO rate for first startups. But that seems to me like a pretty big effect, so I’m not sure quite what to make of it.
I think you could just as well argue “Submitting an academic paper to a good conference is a lottery ticket.” (In fact, the numbers are comparable). In some sense this is true, but I still wouldn’t say “The idea that there are some excellent papers is just factually wrong.”
Thanks Paul for the feedback, and for the reminder that we are criticizing ideas and not people :-)
The thing with academic papers was really interesting, and gave me pause. I would point out two similarities:
The set of all papers submitted to a specific conference is a lot more homogeneous than the set of all papers period. Similarly, the set of all entrepreneurs who get VC funding is a lot more homogenous than the set of all people who think about starting companies. So the statement “performance within some limited subset is mostly due to chance” isn’t necessarily conflicting with the idea that there is such a thing as entrepreneurial skill/paper quality.
Instead of drawing a lesson that there is no such thing as skill we might conclude that acceptance to a conference or having an IPO is just not a very good indicator of skill.
I also agree that this is largely a quantitative disagreement. I’ve spent the last year being surrounded by people who believe that variance in startups is completely determined by the founder’s skill, and that gives me a framing for what I write.