Thanks very much for sharing this, it seems like a very interesting project.
I note on your website you reference V-DEM:
The overwhelming majority of empirical evidence shows a significant decline in the number of democracies and, to an even greater extent, a decline in the number of people who live in democracies. And the trend continues: In 2022, according to the latest V-Dem report, the level of democracy enjoyed by the average global citizen is down to 1986 levels. And only 13% of world population live in liberal democracies.
I would caution you on relying on their data too much. The methodology is not particularly scientific—it’s basically just what some random academics say. In particular, this leaves it quite vulnerable to the biases of academic political scientists.
An example I just checked (possibly there are other better ones) is the US:
A couple of things jump out at me here:
The election of Trump is a huge outlier, giving a major reduction to ‘Liberal Democracy’.
This impact occurs immediately despite the fact that he didn’t actually pass that many laws or make that many changes in 2017; I think this is more about perception than reality.
The impact appears to be larger than the abolition of slavery, the passage of the civil rights act, the second world war, conscription or female suffrage. This seems very implausible to me.
Freedom of domestic movement increased in 2020, despite the introduction of essentially unprecedented covid-related restrictions. (Other countries with even more draconian rules, like the UK, also do not see major declines here, even though the entire population was essentially under house arrest for much of the year)
The subnational civil liberties unevenness index does not seem to reflect the fact that covid restrictions were very different in rural and urban areas.
On the whole I think these ratings often tell us more about the political views of the authors (pro-lockdown, anti-trump) than they do about the actual levels of liberty or democracy in a country.
Until quite recently the existing body of knowledge on the desription and measurement of democracy was somewhat hard to access. Fortunately, donors advised by https://legacies-now.de/en and https://effektiv-spenden.org/ enabled https://ourworldindata.org/ to upgrade their democracy section. Since August of 2022 the most important datasets can be accessed via the “Democracy Explorer” (scroll down on https://ourworldindata.org/democracy ). Hence, you can change the Dataset in display via the dropdown at the top left. Thus, one can choose between e.g., V-Dem, Polity, EIU etc. and you don’t have to go with V-Dem.
My take so far, from browisng myself and from talking to the team at OWiD is that, generally speaking, the datasets—provided by very different institutions and sets of people / authors from around the world—tend to converge on most key issues/ trends. (I haven’t checked the conrete US example; there are literally millions of interesting specific data points to look at.)
Thanks for this reply, but I don’t feel like it really gets at the core of my comment. Looking again at the US data, the different datasets show quite different pictures. Many (correctly, in my opinion) show essentially no recent change; the Polity dataset (screenshotted below) instead shows US democracy as being as low now as in 1798: an era with slavery, no female sufferage, etc.
Even then the different datasets do not agree with your statement though—using the Polity dataset again solely because it’s the one I showed above, we see world levels of democracy at almost their highest levels ever, in contrast to your VDEM claim that democracy has fallen to mid-1980s levels. One of the biggest drivers for the reduction in capita-weighted-democracy for VDEM is India, but Polity actually records India as being extremely democratic—actually more democratic than the US now! Again, this seems clearly absurd to me.
So the mere fact that OWID has a lot of different datasets doesn’t seem very reassuring to me, because 1) the one you specifically highlighted as motivating the project seems problematic, and 2) the others seem to vary significantly and also have significant problems.
I also disagree with your implication that because there are “literally millions of interesting specific data points to look at” that this is some sort of implicit p-hacking. I looked at the US first because it is the most important country, and the problems are apparent on the headline measure; you don’t have to dive into the sub-components. Similarly, I looked at covid because it is clearly one of the most important events of recent years for human freedom, not because I was going around looking for an issue to pick holes over.
Thanks very much for sharing this, it seems like a very interesting project.
I note on your website you reference V-DEM:
I would caution you on relying on their data too much. The methodology is not particularly scientific—it’s basically just what some random academics say. In particular, this leaves it quite vulnerable to the biases of academic political scientists.
An example I just checked (possibly there are other better ones) is the US:
A couple of things jump out at me here:
The election of Trump is a huge outlier, giving a major reduction to ‘Liberal Democracy’.
This impact occurs immediately despite the fact that he didn’t actually pass that many laws or make that many changes in 2017; I think this is more about perception than reality.
The impact appears to be larger than the abolition of slavery, the passage of the civil rights act, the second world war, conscription or female suffrage. This seems very implausible to me.
Freedom of domestic movement increased in 2020, despite the introduction of essentially unprecedented covid-related restrictions. (Other countries with even more draconian rules, like the UK, also do not see major declines here, even though the entire population was essentially under house arrest for much of the year)
The subnational civil liberties unevenness index does not seem to reflect the fact that covid restrictions were very different in rural and urban areas.
On the whole I think these ratings often tell us more about the political views of the authors (pro-lockdown, anti-trump) than they do about the actual levels of liberty or democracy in a country.
Maybe the following is helpful:
Until quite recently the existing body of knowledge on the desription and measurement of democracy was somewhat hard to access. Fortunately, donors advised by https://legacies-now.de/en and https://effektiv-spenden.org/ enabled https://ourworldindata.org/ to upgrade their democracy section.
Since August of 2022 the most important datasets can be accessed via the “Democracy Explorer” (scroll down on https://ourworldindata.org/democracy ). Hence, you can change the Dataset in display via the dropdown at the top left. Thus, one can choose between e.g., V-Dem, Polity, EIU etc. and you don’t have to go with V-Dem.
My take so far, from browisng myself and from talking to the team at OWiD is that, generally speaking, the datasets—provided by very different institutions and sets of people / authors from around the world—tend to converge on most key issues/ trends. (I haven’t checked the conrete US example; there are literally millions of interesting specific data points to look at.)
Thanks for this reply, but I don’t feel like it really gets at the core of my comment. Looking again at the US data, the different datasets show quite different pictures. Many (correctly, in my opinion) show essentially no recent change; the Polity dataset (screenshotted below) instead shows US democracy as being as low now as in 1798: an era with slavery, no female sufferage, etc.
So the mere fact that OWID has a lot of different datasets doesn’t seem very reassuring to me, because 1) the one you specifically highlighted as motivating the project seems problematic, and 2) the others seem to vary significantly and also have significant problems.
I also disagree with your implication that because there are “literally millions of interesting specific data points to look at” that this is some sort of implicit p-hacking. I looked at the US first because it is the most important country, and the problems are apparent on the headline measure; you don’t have to dive into the sub-components. Similarly, I looked at covid because it is clearly one of the most important events of recent years for human freedom, not because I was going around looking for an issue to pick holes over.