Iâm not too surprised that these factors had limited predictive ability, or that their sign was mixed. Factor 3 doesnât seem to incorporate anything about the success of those startups, just their number.
Factor 3â˛s non-significance tells us that, if there is such a thing as founding skill itâs not a skill which improves through practice. This certainly doesnât disprove the skill hypothesis, but itâs a mark against.
For what itâs worth: anecdotally these are major factors that VCs look at, so itâs very surprising to people in the startup world that they donât correlate with success. (To the extent that most investors bluntly tell me that they donât believe the result.)
I agree this why startup founders have such a wide spread in their compensation, but I donât think it helps attribute that spread to luck vs. skill.
I donât think I understand. To me a good formalization of âluckâ is âex-ante varianceâ â I think you disagree?
This looks like a surprisingly small effect! Even a 50-50 split would leave room for the best entrepreneurs to succeed with very high probability if dropped into a random industry.
Keep in mind that most of the variance is still unexplained; itâs not 19% industry and 81% founder.
But in efficient-market land, itâs not very important as a consideration about what startup to start, because there is more competition in industries that are poised to grow.
I agree that under perfect competition profits are zero, and that this mechanism is enforced by increased competition in profitable sectors, but it seems weird to assume perfect competition here? I think Iâm not understanding you again.
The study in question showed that entrepreneurs believe they are better at predicting things. It assumed that this was because they were more biased rather than being better predictors, but both explanations seem plausible. Similarly, it showed that entrepreneurs had better views of themselves, and assumed that was due to overconfidence rather than ability, but didnât try to distinguish the two possibilities.
I donât think this is a fair summary of their paper. For example, they asked people to give 90% confidence intervals for various things, and recorded how frequently the true answer was outside that range. This seems to me to be a legitimate definition of âoverconfidentâ.
The authors also cite evidence that peopleâs answers to questions like âI could succeed at making this venture a success, even though many other managers would failâ is uncorrelated with their actual ability, although I agree that itâs not 100% clear.
For example, they asked people to give 90% confidence intervals for various things, and recorded how frequently the true answer was outside that range. This seems to me to be a legitimate definition of âoverconfidentâ.
I agree that this is a better measure of overconfidence. But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
That said, I also agree that startup founders tend to be unrealistically optimistic about their own prospects (or at least answer survey questions in ways that are unreallistically optimistic).
To me a good formalization of âluckâ is âex-ante varianceâ
Suppose that each startupâs valuation is a deterministic function of the founderâs skill (which is known to the founder). The we expect startup founders to be paid mostly in equity. And we could also see great variation in founder compensation. But I think we would agree that there is no luck in this caseâit seems like exactly the kind of model you are trying to argue against.
Factor 3â˛s non-significance tells us that, if there is such a thing as founding skill itâs not a skill which improves through practice.
Iâm skeptical that the study has enough power to detect a realistic effect for this factor, given that itâs a small piece of human capital and all of the measures are extremely noisy.
I also have concerns about confounding. For example, prior foundings had the largest negative coefficient of any variable on VC funding. The authorâs interpretation is: âpresidents who have experienced entrepreneurial failures in the past may find it more difficult to obtain VC financing for future startups.â I donât know if thatâs the real explanation, but Iâm pretty hesitant to make causal inferences in this caseâwhatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
but it seems weird to assume perfect competition here?
I expect the real world is somewhere in between perfect competition and no competition.
But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
No, you are correct and I misread your statement. Entrepreneurs donât appear to have this sort of overconfidence (at least according to this studyâs results). They (we) appeared to have more self aggrandizing confidence.
But I think we would agree that there is no luck in this caseâit seems like exactly the kind of model you are trying to argue against.
Cool, I thought you might be making this point but I didnât want to put words in your mouth.
Itâs unquestionably true that part of the variance is explained by the fact that any jackass can call themselves an entrepreneur and therefore the set of âentrepreneursâ has a greater range of skills than e.g. the set of software developers. But Iâm skeptical that this is the entire reason; to cite a fact from above: an entrepreneur who has a successful exit only has a 30% chance of success in their next venture. It seems unlikely that entrepreneursâ skills decline rapidly after a successful exit.
whatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
I would agree with this more generally: there are probably modest skill effects, but they are incredibly hard to define and are swamped by the nosiness of startups. Especially once you look at âqualifiedâ founders (e.g. those with venture backing), the skill differentiation explains a very small piece of the variance.
Quoth Gompers et al.:
âWhile it may be better to be lucky than smart, the evidence presented here indicates that being smart has value too.â
I expect the real world is somewhere in between perfect competition and no competition.
Hard to argue with that :-)
Iâm still not entirely sure what lesson you were drawing from the efficient market hypothesis, but perhaps it doesnât matter.
Thanks for engaging with objections, and sorry for being so critical!
Itâs possible we just have a quantitative disagreement.
For example, I agree that there are very few people who could start a successful startup with high reliability. But quantitatively, I donât know how much variation in skill there is and how important it is. I think the most compelling statistic here is the 30% IPO rate for second startup (given success) vs. 18% IPO rate for first startups. But that seems to me like a pretty big effect, so Iâm not sure quite what to make of it.
I think you could just as well argue âSubmitting an academic paper to a good conference is a lottery ticket.â (In fact, the numbers are comparable). In some sense this is true, but I still wouldnât say âThe idea that there are some excellent papers is just factually wrong.â
Thanks Paul for the feedback, and for the reminder that we are criticizing ideas and not people :-)
The thing with academic papers was really interesting, and gave me pause. I would point out two similarities:
The set of all papers submitted to a specific conference is a lot more homogeneous than the set of all papers period. Similarly, the set of all entrepreneurs who get VC funding is a lot more homogenous than the set of all people who think about starting companies. So the statement âperformance within some limited subset is mostly due to chanceâ isnât necessarily conflicting with the idea that there is such a thing as entrepreneurial skill/âpaper quality.
Instead of drawing a lesson that there is no such thing as skill we might conclude that acceptance to a conference or having an IPO is just not a very good indicator of skill.
I also agree that this is largely a quantitative disagreement. Iâve spent the last year being surrounded by people who believe that variance in startups is completely determined by the founderâs skill, and that gives me a framing for what I write.
Thanks for the feedback Paul.
Factor 3â˛s non-significance tells us that, if there is such a thing as founding skill itâs not a skill which improves through practice. This certainly doesnât disprove the skill hypothesis, but itâs a mark against.
For what itâs worth: anecdotally these are major factors that VCs look at, so itâs very surprising to people in the startup world that they donât correlate with success. (To the extent that most investors bluntly tell me that they donât believe the result.)
I donât think I understand. To me a good formalization of âluckâ is âex-ante varianceâ â I think you disagree?
Keep in mind that most of the variance is still unexplained; itâs not 19% industry and 81% founder.
I agree that under perfect competition profits are zero, and that this mechanism is enforced by increased competition in profitable sectors, but it seems weird to assume perfect competition here? I think Iâm not understanding you again.
I donât think this is a fair summary of their paper. For example, they asked people to give 90% confidence intervals for various things, and recorded how frequently the true answer was outside that range. This seems to me to be a legitimate definition of âoverconfidentâ.
The authors also cite evidence that peopleâs answers to questions like âI could succeed at making this venture a success, even though many other managers would failâ is uncorrelated with their actual ability, although I agree that itâs not 100% clear.
I agree that this is a better measure of overconfidence. But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
That said, I also agree that startup founders tend to be unrealistically optimistic about their own prospects (or at least answer survey questions in ways that are unreallistically optimistic).
Suppose that each startupâs valuation is a deterministic function of the founderâs skill (which is known to the founder). The we expect startup founders to be paid mostly in equity. And we could also see great variation in founder compensation. But I think we would agree that there is no luck in this caseâit seems like exactly the kind of model you are trying to argue against.
Iâm skeptical that the study has enough power to detect a realistic effect for this factor, given that itâs a small piece of human capital and all of the measures are extremely noisy.
I also have concerns about confounding. For example, prior foundings had the largest negative coefficient of any variable on VC funding. The authorâs interpretation is: âpresidents who have experienced entrepreneurial failures in the past may find it more difficult to obtain VC financing for future startups.â I donât know if thatâs the real explanation, but Iâm pretty hesitant to make causal inferences in this caseâwhatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
I expect the real world is somewhere in between perfect competition and no competition.
No, you are correct and I misread your statement. Entrepreneurs donât appear to have this sort of overconfidence (at least according to this studyâs results). They (we) appeared to have more self aggrandizing confidence.
Cool, I thought you might be making this point but I didnât want to put words in your mouth.
Itâs unquestionably true that part of the variance is explained by the fact that any jackass can call themselves an entrepreneur and therefore the set of âentrepreneursâ has a greater range of skills than e.g. the set of software developers. But Iâm skeptical that this is the entire reason; to cite a fact from above: an entrepreneur who has a successful exit only has a 30% chance of success in their next venture. It seems unlikely that entrepreneursâ skills decline rapidly after a successful exit.
I would agree with this more generally: there are probably modest skill effects, but they are incredibly hard to define and are swamped by the nosiness of startups. Especially once you look at âqualifiedâ founders (e.g. those with venture backing), the skill differentiation explains a very small piece of the variance.
Quoth Gompers et al.:
âWhile it may be better to be lucky than smart, the evidence presented here indicates that being smart has value too.â
Hard to argue with that :-)
Iâm still not entirely sure what lesson you were drawing from the efficient market hypothesis, but perhaps it doesnât matter.
Thanks for engaging with objections, and sorry for being so critical!
Itâs possible we just have a quantitative disagreement.
For example, I agree that there are very few people who could start a successful startup with high reliability. But quantitatively, I donât know how much variation in skill there is and how important it is. I think the most compelling statistic here is the 30% IPO rate for second startup (given success) vs. 18% IPO rate for first startups. But that seems to me like a pretty big effect, so Iâm not sure quite what to make of it.
I think you could just as well argue âSubmitting an academic paper to a good conference is a lottery ticket.â (In fact, the numbers are comparable). In some sense this is true, but I still wouldnât say âThe idea that there are some excellent papers is just factually wrong.â
Thanks Paul for the feedback, and for the reminder that we are criticizing ideas and not people :-)
The thing with academic papers was really interesting, and gave me pause. I would point out two similarities:
The set of all papers submitted to a specific conference is a lot more homogeneous than the set of all papers period. Similarly, the set of all entrepreneurs who get VC funding is a lot more homogenous than the set of all people who think about starting companies. So the statement âperformance within some limited subset is mostly due to chanceâ isnât necessarily conflicting with the idea that there is such a thing as entrepreneurial skill/âpaper quality.
Instead of drawing a lesson that there is no such thing as skill we might conclude that acceptance to a conference or having an IPO is just not a very good indicator of skill.
I also agree that this is largely a quantitative disagreement. Iâve spent the last year being surrounded by people who believe that variance in startups is completely determined by the founderâs skill, and that gives me a framing for what I write.