Brian, there are several serious sampling biases in most estimates of long run real returns which tend to overestimate the returns. These include:
(1) Time selection bias. The 20th century was unprecedentedly good for stocks. If we instead averaged over wider periods or over the 21st century so far, we get much lower numbers. It is unclear what is the best period to use, but many estimates use the most optimistic one which is suspect.
(2) Country selection bias. The US has done unprecedentedly well with stocks. International comparisons give lower returns and are probably more representative of the future (we don’t know which country will do best this time round).
(3) Within-index selection bias. The major indices are of the top stocks rather than a fixed set, so for example if all the stocks in the S&P 500 went to zero tomorrow, this would really change the real rate of return, but wouldn’t change the index that much as the next 500 stocks would replace them—we need to adjust for that.
(4) Between-exchange selection bias. Even attempts to adjust for the country selection bias by using a range of stock markets or indices in different countries often overestimate returns because failed stock markets typically don’t appear in the later data for they have ceased to exist. One needs to carefully adjust for this.
I don’t recall the exact real returns when these things are adjusted for and can’t quickly find a good estimate, but I seem to recall it comes down to less than 3%. If someone has a pointer to a good estimate, I’d love to see it.
Regarding risk adjustment, I didn’t mean risk aversion, just that you have to adjust for the chance of losses as well as gains to get an expected rate. Any sensible aggregate will do this.
http://economics.mit.edu/files/637 says the US Social Security Administration used a 7% real rate of return, but the paper goes on to explain why that seems too high.
https://en.wikipedia.org/wiki/Equity_premium_puzzle says the equity premium for stocks “is generally accepted to be in the range of 3–7% in the long-run.” That piece lists reasons to deny an equity premium, similar to those you enumerate, but it also says “most mainstream economists agree that the evidence [for an equity premium] shows substantial statistical power.” I don’t know enough to evaluate this debate without further investigation, but your concerns about biases seem significant.
Brian, there are several serious sampling biases in most estimates of long run real returns which tend to overestimate the returns. These include:
(1) Time selection bias. The 20th century was unprecedentedly good for stocks. If we instead averaged over wider periods or over the 21st century so far, we get much lower numbers. It is unclear what is the best period to use, but many estimates use the most optimistic one which is suspect. (2) Country selection bias. The US has done unprecedentedly well with stocks. International comparisons give lower returns and are probably more representative of the future (we don’t know which country will do best this time round). (3) Within-index selection bias. The major indices are of the top stocks rather than a fixed set, so for example if all the stocks in the S&P 500 went to zero tomorrow, this would really change the real rate of return, but wouldn’t change the index that much as the next 500 stocks would replace them—we need to adjust for that. (4) Between-exchange selection bias. Even attempts to adjust for the country selection bias by using a range of stock markets or indices in different countries often overestimate returns because failed stock markets typically don’t appear in the later data for they have ceased to exist. One needs to carefully adjust for this.
I don’t recall the exact real returns when these things are adjusted for and can’t quickly find a good estimate, but I seem to recall it comes down to less than 3%. If someone has a pointer to a good estimate, I’d love to see it.
Regarding risk adjustment, I didn’t mean risk aversion, just that you have to adjust for the chance of losses as well as gains to get an expected rate. Any sensible aggregate will do this.
Thanks for those notes. :)
http://economics.mit.edu/files/637 says the US Social Security Administration used a 7% real rate of return, but the paper goes on to explain why that seems too high.
https://en.wikipedia.org/wiki/Equity_premium_puzzle says the equity premium for stocks “is generally accepted to be in the range of 3–7% in the long-run.” That piece lists reasons to deny an equity premium, similar to those you enumerate, but it also says “most mainstream economists agree that the evidence [for an equity premium] shows substantial statistical power.” I don’t know enough to evaluate this debate without further investigation, but your concerns about biases seem significant.