Human capital variables have limited impact on startup performance, and the few significant effects are split equally between enhancing and impeding performance
Isn’t this what you would see if your “human capital variables” just don’t have much predictive power? Their four human capital variables, described in section 4.3.3, seem to be the following: 1. number of people in management roles, 2. number of management roles held by the president, 3. number of prior startups which the founder was president of, 4. number of roles at other companies currently held by president.
I’m not too surprised that these factors had limited predictive ability, or that their sign was mixed. Factor 3 doesn’t seem to incorporate anything about the success of those startups, just their number.
Entrepreneurship must involve an element of risk for the founder (i.e. luck). If it didn’t, no one would invest.
I don’t understand the logic here. In any domain with principal agent problems, you expect the principal to pay the agent a fraction of the output, i.e. for the agent to retain equity. In fact, financial theory suggests that startup founders should take more equity when performance was more dependent on their behavior / skill, and should take less equity when performance is more out of their control. So the fact that startup founders receive variable payouts is direct evidence against the luck hypothesis.
I agree this why startup founders have such a wide spread in their compensation, but I don’t think it helps attribute that spread to luck vs. skill.
But the point is that skill in entrepreneurship is less about being able to steer your ship through stormy waters and more about checking the weather before you set off.
It seems like this is mostly a claim about what skills are useful, rather than about the importance of skill. The fact that smartphone use was high was not out of Uber’s control, as you point out, since they founded the businesses when it looked reasonable. So it’s strange to call it a “factor completely outside of the founder’s control.”
Your industry’s performance explains 19% of your company’s performance.
This looks like a surprisingly small effect! Even a 50-50 split would leave room for the best entrepreneurs to succeed with very high probability if dropped into a random industry. You say:
my intuition is that small companies are even more affected by industry downturns.
This may be true (though I have the opposite intuition), but even if so it seems almost guaranteed to be a smaller share of the variance—just because there is so much more variance.
More importantly, variation in performance of an industry has a huge impact on the returns to industry-specific capital, even in efficient-market land. But in efficient-market land, it’s not very important as a consideration about what startup to start, because there is more competition in industries that are poised to grow.
One of the differentiators between entrepreneurs and everyone else is that entrepreneurs have a stronger illusion of control, believing that they can accurately predict things like when competitors will enter the market.
The study in question showed that entrepreneurs believe they are better at predicting things. It assumed that this was because they were more biased rather than being better predictors, but both explanations seem plausible. Similarly, it showed that entrepreneurs had better views of themselves, and assumed that was due to overconfidence rather than ability, but didn’t try to distinguish the two possibilities.
I’m not too surprised that these factors had limited predictive ability, or that their sign was mixed. Factor 3 doesn’t seem to incorporate anything about the success of those startups, just their number.
Factor 3′s non-significance tells us that, if there is such a thing as founding skill it’s not a skill which improves through practice. This certainly doesn’t disprove the skill hypothesis, but it’s a mark against.
For what it’s worth: anecdotally these are major factors that VCs look at, so it’s very surprising to people in the startup world that they don’t correlate with success. (To the extent that most investors bluntly tell me that they don’t believe the result.)
I agree this why startup founders have such a wide spread in their compensation, but I don’t think it helps attribute that spread to luck vs. skill.
I don’t think I understand. To me a good formalization of “luck” is “ex-ante variance” – I think you disagree?
This looks like a surprisingly small effect! Even a 50-50 split would leave room for the best entrepreneurs to succeed with very high probability if dropped into a random industry.
Keep in mind that most of the variance is still unexplained; it’s not 19% industry and 81% founder.
But in efficient-market land, it’s not very important as a consideration about what startup to start, because there is more competition in industries that are poised to grow.
I agree that under perfect competition profits are zero, and that this mechanism is enforced by increased competition in profitable sectors, but it seems weird to assume perfect competition here? I think I’m not understanding you again.
The study in question showed that entrepreneurs believe they are better at predicting things. It assumed that this was because they were more biased rather than being better predictors, but both explanations seem plausible. Similarly, it showed that entrepreneurs had better views of themselves, and assumed that was due to overconfidence rather than ability, but didn’t try to distinguish the two possibilities.
I don’t think this is a fair summary of their paper. For example, they asked people to give 90% confidence intervals for various things, and recorded how frequently the true answer was outside that range. This seems to me to be a legitimate definition of “overconfident”.
The authors also cite evidence that people’s answers to questions like “I could succeed at making this venture a success, even though many other managers would fail” is uncorrelated with their actual ability, although I agree that it’s not 100% clear.
For example, they asked people to give 90% confidence intervals for various things, and recorded how frequently the true answer was outside that range. This seems to me to be a legitimate definition of “overconfident”.
I agree that this is a better measure of overconfidence. But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
That said, I also agree that startup founders tend to be unrealistically optimistic about their own prospects (or at least answer survey questions in ways that are unreallistically optimistic).
To me a good formalization of “luck” is “ex-ante variance”
Suppose that each startup’s valuation is a deterministic function of the founder’s skill (which is known to the founder). The we expect startup founders to be paid mostly in equity. And we could also see great variation in founder compensation. But I think we would agree that there is no luck in this case—it seems like exactly the kind of model you are trying to argue against.
Factor 3′s non-significance tells us that, if there is such a thing as founding skill it’s not a skill which improves through practice.
I’m skeptical that the study has enough power to detect a realistic effect for this factor, given that it’s a small piece of human capital and all of the measures are extremely noisy.
I also have concerns about confounding. For example, prior foundings had the largest negative coefficient of any variable on VC funding. The author’s interpretation is: “presidents who have experienced entrepreneurial failures in the past may find it more difficult to obtain VC financing for future startups.” I don’t know if that’s the real explanation, but I’m pretty hesitant to make causal inferences in this case—whatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
but it seems weird to assume perfect competition here?
I expect the real world is somewhere in between perfect competition and no competition.
But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
No, you are correct and I misread your statement. Entrepreneurs don’t appear to have this sort of overconfidence (at least according to this study’s results). They (we) appeared to have more self aggrandizing confidence.
But I think we would agree that there is no luck in this case—it seems like exactly the kind of model you are trying to argue against.
Cool, I thought you might be making this point but I didn’t want to put words in your mouth.
It’s unquestionably true that part of the variance is explained by the fact that any jackass can call themselves an entrepreneur and therefore the set of “entrepreneurs” has a greater range of skills than e.g. the set of software developers. But I’m skeptical that this is the entire reason; to cite a fact from above: an entrepreneur who has a successful exit only has a 30% chance of success in their next venture. It seems unlikely that entrepreneurs’ skills decline rapidly after a successful exit.
whatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
I would agree with this more generally: there are probably modest skill effects, but they are incredibly hard to define and are swamped by the nosiness of startups. Especially once you look at “qualified” founders (e.g. those with venture backing), the skill differentiation explains a very small piece of the variance.
Quoth Gompers et al.:
“While it may be better to be lucky than smart, the evidence presented here indicates that being smart has value too.”
I expect the real world is somewhere in between perfect competition and no competition.
Hard to argue with that :-)
I’m still not entirely sure what lesson you were drawing from the efficient market hypothesis, but perhaps it doesn’t matter.
Thanks for engaging with objections, and sorry for being so critical!
It’s possible we just have a quantitative disagreement.
For example, I agree that there are very few people who could start a successful startup with high reliability. But quantitatively, I don’t know how much variation in skill there is and how important it is. I think the most compelling statistic here is the 30% IPO rate for second startup (given success) vs. 18% IPO rate for first startups. But that seems to me like a pretty big effect, so I’m not sure quite what to make of it.
I think you could just as well argue “Submitting an academic paper to a good conference is a lottery ticket.” (In fact, the numbers are comparable). In some sense this is true, but I still wouldn’t say “The idea that there are some excellent papers is just factually wrong.”
Thanks Paul for the feedback, and for the reminder that we are criticizing ideas and not people :-)
The thing with academic papers was really interesting, and gave me pause. I would point out two similarities:
The set of all papers submitted to a specific conference is a lot more homogeneous than the set of all papers period. Similarly, the set of all entrepreneurs who get VC funding is a lot more homogenous than the set of all people who think about starting companies. So the statement “performance within some limited subset is mostly due to chance” isn’t necessarily conflicting with the idea that there is such a thing as entrepreneurial skill/paper quality.
Instead of drawing a lesson that there is no such thing as skill we might conclude that acceptance to a conference or having an IPO is just not a very good indicator of skill.
I also agree that this is largely a quantitative disagreement. I’ve spent the last year being surrounded by people who believe that variance in startups is completely determined by the founder’s skill, and that gives me a framing for what I write.
My thoughts exactly—except
“Your industry’s performance explains 19% of your company’s performance.
This looks like a surprisingly small effect! Even a 50-50 split would leave room for the best entrepreneurs to succeed with very high probability if dropped into a random industry.”
doesn’t take account of the proportion of businesses that fail—if 10% (as I understand it) succeed according to some measure, then a 50-50 split would leave a fairly small probability if dropped into a random industry, as would 80-20.
Isn’t this what you would see if your “human capital variables” just don’t have much predictive power? Their four human capital variables, described in section 4.3.3, seem to be the following: 1. number of people in management roles, 2. number of management roles held by the president, 3. number of prior startups which the founder was president of, 4. number of roles at other companies currently held by president.
I’m not too surprised that these factors had limited predictive ability, or that their sign was mixed. Factor 3 doesn’t seem to incorporate anything about the success of those startups, just their number.
I don’t understand the logic here. In any domain with principal agent problems, you expect the principal to pay the agent a fraction of the output, i.e. for the agent to retain equity. In fact, financial theory suggests that startup founders should take more equity when performance was more dependent on their behavior / skill, and should take less equity when performance is more out of their control. So the fact that startup founders receive variable payouts is direct evidence against the luck hypothesis.
I agree this why startup founders have such a wide spread in their compensation, but I don’t think it helps attribute that spread to luck vs. skill.
It seems like this is mostly a claim about what skills are useful, rather than about the importance of skill. The fact that smartphone use was high was not out of Uber’s control, as you point out, since they founded the businesses when it looked reasonable. So it’s strange to call it a “factor completely outside of the founder’s control.”
This looks like a surprisingly small effect! Even a 50-50 split would leave room for the best entrepreneurs to succeed with very high probability if dropped into a random industry. You say:
This may be true (though I have the opposite intuition), but even if so it seems almost guaranteed to be a smaller share of the variance—just because there is so much more variance.
More importantly, variation in performance of an industry has a huge impact on the returns to industry-specific capital, even in efficient-market land. But in efficient-market land, it’s not very important as a consideration about what startup to start, because there is more competition in industries that are poised to grow.
The study in question showed that entrepreneurs believe they are better at predicting things. It assumed that this was because they were more biased rather than being better predictors, but both explanations seem plausible. Similarly, it showed that entrepreneurs had better views of themselves, and assumed that was due to overconfidence rather than ability, but didn’t try to distinguish the two possibilities.
Thanks for the feedback Paul.
Factor 3′s non-significance tells us that, if there is such a thing as founding skill it’s not a skill which improves through practice. This certainly doesn’t disprove the skill hypothesis, but it’s a mark against.
For what it’s worth: anecdotally these are major factors that VCs look at, so it’s very surprising to people in the startup world that they don’t correlate with success. (To the extent that most investors bluntly tell me that they don’t believe the result.)
I don’t think I understand. To me a good formalization of “luck” is “ex-ante variance” – I think you disagree?
Keep in mind that most of the variance is still unexplained; it’s not 19% industry and 81% founder.
I agree that under perfect competition profits are zero, and that this mechanism is enforced by increased competition in profitable sectors, but it seems weird to assume perfect competition here? I think I’m not understanding you again.
I don’t think this is a fair summary of their paper. For example, they asked people to give 90% confidence intervals for various things, and recorded how frequently the true answer was outside that range. This seems to me to be a legitimate definition of “overconfident”.
The authors also cite evidence that people’s answers to questions like “I could succeed at making this venture a success, even though many other managers would fail” is uncorrelated with their actual ability, although I agree that it’s not 100% clear.
I agree that this is a better measure of overconfidence. But it looks like startup founders did insignificantly better at this task, rather than doing worse at it. I may be misreading Table 1.
That said, I also agree that startup founders tend to be unrealistically optimistic about their own prospects (or at least answer survey questions in ways that are unreallistically optimistic).
Suppose that each startup’s valuation is a deterministic function of the founder’s skill (which is known to the founder). The we expect startup founders to be paid mostly in equity. And we could also see great variation in founder compensation. But I think we would agree that there is no luck in this case—it seems like exactly the kind of model you are trying to argue against.
I’m skeptical that the study has enough power to detect a realistic effect for this factor, given that it’s a small piece of human capital and all of the measures are extremely noisy.
I also have concerns about confounding. For example, prior foundings had the largest negative coefficient of any variable on VC funding. The author’s interpretation is: “presidents who have experienced entrepreneurial failures in the past may find it more difficult to obtain VC financing for future startups.” I don’t know if that’s the real explanation, but I’m pretty hesitant to make causal inferences in this case—whatever confounder produced that large negative effect, I expect it swamps a modest practice effect.
I expect the real world is somewhere in between perfect competition and no competition.
No, you are correct and I misread your statement. Entrepreneurs don’t appear to have this sort of overconfidence (at least according to this study’s results). They (we) appeared to have more self aggrandizing confidence.
Cool, I thought you might be making this point but I didn’t want to put words in your mouth.
It’s unquestionably true that part of the variance is explained by the fact that any jackass can call themselves an entrepreneur and therefore the set of “entrepreneurs” has a greater range of skills than e.g. the set of software developers. But I’m skeptical that this is the entire reason; to cite a fact from above: an entrepreneur who has a successful exit only has a 30% chance of success in their next venture. It seems unlikely that entrepreneurs’ skills decline rapidly after a successful exit.
I would agree with this more generally: there are probably modest skill effects, but they are incredibly hard to define and are swamped by the nosiness of startups. Especially once you look at “qualified” founders (e.g. those with venture backing), the skill differentiation explains a very small piece of the variance.
Quoth Gompers et al.:
“While it may be better to be lucky than smart, the evidence presented here indicates that being smart has value too.”
Hard to argue with that :-)
I’m still not entirely sure what lesson you were drawing from the efficient market hypothesis, but perhaps it doesn’t matter.
Thanks for engaging with objections, and sorry for being so critical!
It’s possible we just have a quantitative disagreement.
For example, I agree that there are very few people who could start a successful startup with high reliability. But quantitatively, I don’t know how much variation in skill there is and how important it is. I think the most compelling statistic here is the 30% IPO rate for second startup (given success) vs. 18% IPO rate for first startups. But that seems to me like a pretty big effect, so I’m not sure quite what to make of it.
I think you could just as well argue “Submitting an academic paper to a good conference is a lottery ticket.” (In fact, the numbers are comparable). In some sense this is true, but I still wouldn’t say “The idea that there are some excellent papers is just factually wrong.”
Thanks Paul for the feedback, and for the reminder that we are criticizing ideas and not people :-)
The thing with academic papers was really interesting, and gave me pause. I would point out two similarities:
The set of all papers submitted to a specific conference is a lot more homogeneous than the set of all papers period. Similarly, the set of all entrepreneurs who get VC funding is a lot more homogenous than the set of all people who think about starting companies. So the statement “performance within some limited subset is mostly due to chance” isn’t necessarily conflicting with the idea that there is such a thing as entrepreneurial skill/paper quality.
Instead of drawing a lesson that there is no such thing as skill we might conclude that acceptance to a conference or having an IPO is just not a very good indicator of skill.
I also agree that this is largely a quantitative disagreement. I’ve spent the last year being surrounded by people who believe that variance in startups is completely determined by the founder’s skill, and that gives me a framing for what I write.
My thoughts exactly—except “Your industry’s performance explains 19% of your company’s performance.
This looks like a surprisingly small effect! Even a 50-50 split would leave room for the best entrepreneurs to succeed with very high probability if dropped into a random industry.”
doesn’t take account of the proportion of businesses that fail—if 10% (as I understand it) succeed according to some measure, then a 50-50 split would leave a fairly small probability if dropped into a random industry, as would 80-20.