The conclusion seems reasonable, but I have some concerns about taking this at face value. The large number of dependent variables also makes me a bit skeptical. How do we know they werenât p-hacking by, say, choosing the best 14 of 25 possible dependent variables? More importantly, it doesnât seem to robustly establish causation. What if Latin America, the US and Africa have worse outcomes due to lack of trade or something?
Furthermore no subsequent article (afaik) has found evidence supporting presidentialism.
Good questions Thomas. The point of the blog series is to highlight papers that ask the right questions and use the right methods to have consequentialist value. I am not arguing that the Gerring paper is the last word. Iâll answer a few of your questions, though.
We know they arenât p-hacking in the selection of dependent variables because there are very few such variables that cover every country-year of interest. How many organizations measured the governance quality of Liberia, Columbia and Denmark in 1953. Iâm working on introducing a new one using weather station quality.
I didnât want to dive into the regression table in my blog post. All models used adjust for continent. They also adjust for distance to financial center. I would also point out that if the continents with lots of presidential regimes have less cross-border trade, this is evidence against quality of governance of presidentialism.
There is a later study with an expanded dataset that supported the null on GDP, but I didnât include it because it ignored the 13 other governance indicators. This isnât my main research area so I wonât do a full literature review for this blog post. In municipalities the same result is robustly observed.
If robustly establishing causation means âadjusting for every factor which could possibly affect governance outcomes at the country levelâ, then the question is clearly unanswerable. There are hundreds of such factors and RCTâs are impossible. But as consequentialists our goal isnât to achieve some arbitrary degree of confidence in our beliefs. The goal is to make better decisions. Since your prior on pres v. parl should be near .5, this evidence compellingly moves us toward the parl side, maybe to .7 . For a constitutional designer, thatâs a hugely valuable update. There remains a 30% chance of making the wrong decision, but thatâs way better than a 50% chance of making the wrong decision. Therefore if even one constitutional designer reads this paper, the QALYâs that Gerring et al. have made is huge.
Epistemic status: have not read the paper
The conclusion seems reasonable, but I have some concerns about taking this at face value. The large number of dependent variables also makes me a bit skeptical. How do we know they werenât p-hacking by, say, choosing the best 14 of 25 possible dependent variables? More importantly, it doesnât seem to robustly establish causation. What if Latin America, the US and Africa have worse outcomes due to lack of trade or something?
How many such articles have there been?
Good questions Thomas. The point of the blog series is to highlight papers that ask the right questions and use the right methods to have consequentialist value. I am not arguing that the Gerring paper is the last word. Iâll answer a few of your questions, though.
We know they arenât p-hacking in the selection of dependent variables because there are very few such variables that cover every country-year of interest. How many organizations measured the governance quality of Liberia, Columbia and Denmark in 1953. Iâm working on introducing a new one using weather station quality.
I didnât want to dive into the regression table in my blog post. All models used adjust for continent. They also adjust for distance to financial center. I would also point out that if the continents with lots of presidential regimes have less cross-border trade, this is evidence against quality of governance of presidentialism.
There is a later study with an expanded dataset that supported the null on GDP, but I didnât include it because it ignored the 13 other governance indicators. This isnât my main research area so I wonât do a full literature review for this blog post. In municipalities the same result is robustly observed.
If robustly establishing causation means âadjusting for every factor which could possibly affect governance outcomes at the country levelâ, then the question is clearly unanswerable. There are hundreds of such factors and RCTâs are impossible. But as consequentialists our goal isnât to achieve some arbitrary degree of confidence in our beliefs. The goal is to make better decisions. Since your prior on pres v. parl should be near .5, this evidence compellingly moves us toward the parl side, maybe to .7 . For a constitutional designer, thatâs a hugely valuable update. There remains a 30% chance of making the wrong decision, but thatâs way better than a 50% chance of making the wrong decision. Therefore if even one constitutional designer reads this paper, the QALYâs that Gerring et al. have made is huge.
Thanks for the elaboration! Iâm just glad to hear that the researchers didnât make any obvious mistakes.