In which the authors performed a linear regression on companies’ profitability which included a coefficient for every single company (i.e. it was insanely biased towards finding that individual performance matters).
This lead to the following conclusions:
Your industry’s performance explains 19% of your company’s performance.
Everything else about your company combined (growth, employees, IP,…) explains 32% of performance.
I have some concerns with this paper:
It’s not about startups (average assets of included companies were something like $500m) and is highly weighted towards commoditized industries.
They allow persistent shocks in unexplained variance, which is going to seriously attenuate any firm-specific effect, basically shifting some of the “growth, employees, IP, …” effects into the unexplained variance.
Finally, they don’t mention logging dollar variables, which makes me highly suspicious (perhaps it’s so obvious they don’t need to mention it?)--dollars are power-law distributed, which means large values have an outsized influence on unlogged linear regression fits.
Thanks for the comments Ben! This paper is the canonical response to the poorly researched “All great CEOs do these 10 things” sorts of articles that Harvard Business Review etc. loves to publish, which is why I included it.
Fair enough. If you look at table 4, you can see that the basic trend of “industry matters more than segment specific effects” holds for everything except, confusingly, for manufacturing. So while it may not be robust across different company sizes, it’s at least robust across different industries. (And my intuition is that small companies are even more affected by industry downturns.)
If I’m understanding you correctly, you think they should be using the Rumelt model described on pages 17 and 18? They ran that model on the data in table 4 with basically the same result (17% of variance explained by industry, 29% by company versus 19% and 32% with their model).
Interesting, good point. I skimmed some citing articles and couldn’t find anyone talking about this, so I’m not sure.
So while it may not be robust across different company sizes, it’s at least robust across different industries.
This is irrelevant to your point if it doesn’t hold across sizes, though. Compustat’s data is entirely on public companies. I really don’t see any reason to expect this to generalize to tech startups.
In fact, one would expect that most of the influence of founder/IP/team/whatever would be concentrated in navigating from startup to IPO, rather than in post-IPO profits. If the fixed effect is already on the order of 30% of variance post-IPO, I think it’s hard to defend the primarily-lottery-ticket interpretation in the pre-IPO regime.
Overall, I agree with the point that this data set may not be valid, and I sincerely appreciate you pointing that out. I will have to do more research and figure out what can be done with that section.
However, I disagree with the statement:
In fact, one would expect that most of the influence of founder/IP/team/whatever would be concentrated in navigating from startup to IPO, rather than in post-IPO profits.
Startups pivot all the time, and are constantly changing their products. Therefore we should expect that a startup’s profitability in year T doesn’t predict very well their profitability in year T+1.
Big companies almost never change, and my experience is that a large part of the reason they make sales this year is because of the sales, products etc. they made last year. Therefore we should expect their revenue stream to be highly correlated over time.
So I think the big company numbers place an upper bound on the results you would see from small companies, not a lower one.
(One way of coming to this conclusion that I assume is uncontroversial is just to note that the valuations of startups fluctuate a lot more rapidly than the valuations of public companies.)
Let me know what you think about this, and your thoughts around including some language about this in that section.
Startups pivot all the time, and are constantly changing their products. Therefore we should expect that a startup’s profitability in year T doesn’t predict very well their profitability in year T+1.
Yearly profitability isn’t the right measure before IPO; most startups aren’t optimizing for profitability (and indeed never become profitable until after IPO). High losses can indicate failure, or they can indicate rapid scaling of a successful model. So unpredictable profits aren’t really evidence of eventual success being unpredictable. Even if eventual success was relatively predictable we would expect profits to fluctuate along the way depending on the company’s specific growth trajectory.
Sure. For better or for worse that’s what the paper I cited measured, so that’s why I was using that metric.
If you prefer though: startups have significantly higher fluctuations in valuation than public companies, and presumably valuation captures whatever metrics are relevant.
I have some concerns with this paper:
It’s not about startups (average assets of included companies were something like $500m) and is highly weighted towards commoditized industries.
They allow persistent shocks in unexplained variance, which is going to seriously attenuate any firm-specific effect, basically shifting some of the “growth, employees, IP, …” effects into the unexplained variance.
Finally, they don’t mention logging dollar variables, which makes me highly suspicious (perhaps it’s so obvious they don’t need to mention it?)--dollars are power-law distributed, which means large values have an outsized influence on unlogged linear regression fits.
Thanks for the comments Ben! This paper is the canonical response to the poorly researched “All great CEOs do these 10 things” sorts of articles that Harvard Business Review etc. loves to publish, which is why I included it.
Fair enough. If you look at table 4, you can see that the basic trend of “industry matters more than segment specific effects” holds for everything except, confusingly, for manufacturing. So while it may not be robust across different company sizes, it’s at least robust across different industries. (And my intuition is that small companies are even more affected by industry downturns.)
If I’m understanding you correctly, you think they should be using the Rumelt model described on pages 17 and 18? They ran that model on the data in table 4 with basically the same result (17% of variance explained by industry, 29% by company versus 19% and 32% with their model).
Interesting, good point. I skimmed some citing articles and couldn’t find anyone talking about this, so I’m not sure.
This is irrelevant to your point if it doesn’t hold across sizes, though. Compustat’s data is entirely on public companies. I really don’t see any reason to expect this to generalize to tech startups.
In fact, one would expect that most of the influence of founder/IP/team/whatever would be concentrated in navigating from startup to IPO, rather than in post-IPO profits. If the fixed effect is already on the order of 30% of variance post-IPO, I think it’s hard to defend the primarily-lottery-ticket interpretation in the pre-IPO regime.
Overall, I agree with the point that this data set may not be valid, and I sincerely appreciate you pointing that out. I will have to do more research and figure out what can be done with that section.
However, I disagree with the statement:
Startups pivot all the time, and are constantly changing their products. Therefore we should expect that a startup’s profitability in year T doesn’t predict very well their profitability in year T+1.
Big companies almost never change, and my experience is that a large part of the reason they make sales this year is because of the sales, products etc. they made last year. Therefore we should expect their revenue stream to be highly correlated over time.
So I think the big company numbers place an upper bound on the results you would see from small companies, not a lower one.
(One way of coming to this conclusion that I assume is uncontroversial is just to note that the valuations of startups fluctuate a lot more rapidly than the valuations of public companies.)
Let me know what you think about this, and your thoughts around including some language about this in that section.
Yearly profitability isn’t the right measure before IPO; most startups aren’t optimizing for profitability (and indeed never become profitable until after IPO). High losses can indicate failure, or they can indicate rapid scaling of a successful model. So unpredictable profits aren’t really evidence of eventual success being unpredictable. Even if eventual success was relatively predictable we would expect profits to fluctuate along the way depending on the company’s specific growth trajectory.
Sure. For better or for worse that’s what the paper I cited measured, so that’s why I was using that metric.
If you prefer though: startups have significantly higher fluctuations in valuation than public companies, and presumably valuation captures whatever metrics are relevant.