Good questions Thomas. The point of the blog series is to highlight papers that ask the right questions and use the right methods to have consequentialist value. I am not arguing that the Gerring paper is the last word. I’ll answer a few of your questions, though.
We know they aren’t p-hacking in the selection of dependent variables because there are very few such variables that cover every country-year of interest. How many organizations measured the governance quality of Liberia, Columbia and Denmark in 1953. I’m working on introducing a new one using weather station quality.
I didn’t want to dive into the regression table in my blog post. All models used adjust for continent. They also adjust for distance to financial center. I would also point out that if the continents with lots of presidential regimes have less cross-border trade, this is evidence against quality of governance of presidentialism.
There is a later study with an expanded dataset that supported the null on GDP, but I didn’t include it because it ignored the 13 other governance indicators. This isn’t my main research area so I won’t do a full literature review for this blog post. In municipalities the same result is robustly observed.
If robustly establishing causation means “adjusting for every factor which could possibly affect governance outcomes at the country level”, then the question is clearly unanswerable. There are hundreds of such factors and RCT’s are impossible. But as consequentialists our goal isn’t to achieve some arbitrary degree of confidence in our beliefs. The goal is to make better decisions. Since your prior on pres v. parl should be near .5, this evidence compellingly moves us toward the parl side, maybe to .7 . For a constitutional designer, that’s a hugely valuable update. There remains a 30% chance of making the wrong decision, but that’s way better than a 50% chance of making the wrong decision. Therefore if even one constitutional designer reads this paper, the QALY’s that Gerring et al. have made is huge.
Good questions Thomas. The point of the blog series is to highlight papers that ask the right questions and use the right methods to have consequentialist value. I am not arguing that the Gerring paper is the last word. I’ll answer a few of your questions, though.
We know they aren’t p-hacking in the selection of dependent variables because there are very few such variables that cover every country-year of interest. How many organizations measured the governance quality of Liberia, Columbia and Denmark in 1953. I’m working on introducing a new one using weather station quality.
I didn’t want to dive into the regression table in my blog post. All models used adjust for continent. They also adjust for distance to financial center. I would also point out that if the continents with lots of presidential regimes have less cross-border trade, this is evidence against quality of governance of presidentialism.
There is a later study with an expanded dataset that supported the null on GDP, but I didn’t include it because it ignored the 13 other governance indicators. This isn’t my main research area so I won’t do a full literature review for this blog post. In municipalities the same result is robustly observed.
If robustly establishing causation means “adjusting for every factor which could possibly affect governance outcomes at the country level”, then the question is clearly unanswerable. There are hundreds of such factors and RCT’s are impossible. But as consequentialists our goal isn’t to achieve some arbitrary degree of confidence in our beliefs. The goal is to make better decisions. Since your prior on pres v. parl should be near .5, this evidence compellingly moves us toward the parl side, maybe to .7 . For a constitutional designer, that’s a hugely valuable update. There remains a 30% chance of making the wrong decision, but that’s way better than a 50% chance of making the wrong decision. Therefore if even one constitutional designer reads this paper, the QALY’s that Gerring et al. have made is huge.
Thanks for the elaboration! I’m just glad to hear that the researchers didn’t make any obvious mistakes.