2020 Economists’ Comparisons of Welfare Policies: A few economists (Hendren, Sprung-Keyser) aggregated the results of studies that calculate the impact and costs of various welfare policies (US only). They develop a method to compare the impact of these policies. This paper has a lengthy description of this approach and contains the comparisons themselves (reference Table II. in the appendix for the full list and know that “MVPF” is the primary comparison metric). Hendren and, another economist, Finkelstein provide a less lengthy description here. There’s also a great 7-minute explainer video by econimate here (I’d start with this!).
Federal Government CBAs: Additionally, as @evelynciara mentioned earlier, the federal government uses cost-benefit analyses (CBAs) to evaluate regulations. Agencies (e.g. the Department of Energy) conduct these CBAs, which are then reviewed by OIRA (a group housed within OMB). You can find spreadsheets with aggregated results for different regulations here and can explore regulations.gov for specific CBAs.
Other: State and local governments will publish CBAs too (and economists frequently perform CBAs on state/local programs obviously), but I’m not aware of any aggregation of these studies beyond the work by Hendren, Sprung-Keyser, Finkelstein, et al.
|||||
MVPFs and CBAs calculate the ratio of benefits-to-costs (or costs-to-benefits). On the other hand, GiveWell generally calculates “$/life saved” and/or “cost-effectiveness in multiples of cash transfers” for each program they analyze. At first glance, the”cost-effectiveness in multiples of cash transfers” metric appears to approximate a ratio of benefits-to-costs. However, this ratio should not be compared to MVPF calculations by Hendren, nor the US CBAs for multiple reasons. One primary reason they shouldn’t be compared is that there are issues comparing cost-benefit ratios of programs that affect different income groups (a friend and I made a post about that issue here).
Happy to provide more info if you’re interested (for example, I think there are some limitations to the approach used by Hendren/Sprung-Keyser, but overall I think it’s awesome).
This is super interesting! The Policy Impacts-project at Harvard University, and their method for evaluating the impacts of public policies that you refer to, the Marginal Value of Public Funds (MVPF), is new to me but seems potentially very valuable!
I recommend any policy-interested EA to check out the links I have provided here (very simple reading), and those from @Schethik above.
I see that the Policy Impacts project have also created a Policy Impacts Library, a database of MVPF estimates for different public policies derived from rigorous empirical research.
Thanks for this. I agree, the MVPF approach does seem to be somewhat limiting, but it’s got me thinking in the right direction. It seems like the US Federal CBA system is pretty ad-hoc. As far as I can tell each department is responsible for producing their own CBAs, and they each use their own methodology for quantifying things. Do you know if that’s right, or is there some standardized methodology that I’m not seeing?
Yes, you’re correct that each department produces its own CBAs. However, the Office of Management and Budget (OMB) does help standardize these CBAs to some extent. For example, the federal government sets up “interagency working groups” to standardize specific inputs (e.g. the social cost of carbon) across departments (more info on social cost of carbon standardization). And, when OIRA reviews CBAs, they make sure departments are following some general standard processes too.
2020 Economists’ Comparisons of Welfare Policies: A few economists (Hendren, Sprung-Keyser) aggregated the results of studies that calculate the impact and costs of various welfare policies (US only). They develop a method to compare the impact of these policies. This paper has a lengthy description of this approach and contains the comparisons themselves (reference Table II. in the appendix for the full list and know that “MVPF” is the primary comparison metric). Hendren and, another economist, Finkelstein provide a less lengthy description here. There’s also a great 7-minute explainer video by econimate here (I’d start with this!).
Federal Government CBAs: Additionally, as @evelynciara mentioned earlier, the federal government uses cost-benefit analyses (CBAs) to evaluate regulations. Agencies (e.g. the Department of Energy) conduct these CBAs, which are then reviewed by OIRA (a group housed within OMB). You can find spreadsheets with aggregated results for different regulations here and can explore regulations.gov for specific CBAs.
Other: State and local governments will publish CBAs too (and economists frequently perform CBAs on state/local programs obviously), but I’m not aware of any aggregation of these studies beyond the work by Hendren, Sprung-Keyser, Finkelstein, et al.
|||||
MVPFs and CBAs calculate the ratio of benefits-to-costs (or costs-to-benefits). On the other hand, GiveWell generally calculates “$/life saved” and/or “cost-effectiveness in multiples of cash transfers” for each program they analyze. At first glance, the”cost-effectiveness in multiples of cash transfers” metric appears to approximate a ratio of benefits-to-costs. However, this ratio should not be compared to MVPF calculations by Hendren, nor the US CBAs for multiple reasons. One primary reason they shouldn’t be compared is that there are issues comparing cost-benefit ratios of programs that affect different income groups (a friend and I made a post about that issue here).
Happy to provide more info if you’re interested (for example, I think there are some limitations to the approach used by Hendren/Sprung-Keyser, but overall I think it’s awesome).
This is super interesting! The Policy Impacts-project at Harvard University, and their method for evaluating the impacts of public policies that you refer to, the Marginal Value of Public Funds (MVPF), is new to me but seems potentially very valuable!
I recommend any policy-interested EA to check out the links I have provided here (very simple reading), and those from @Schethik above.
I see that the Policy Impacts project have also created a Policy Impacts Library, a database of MVPF estimates for different public policies derived from rigorous empirical research.
Thanks for this. I agree, the MVPF approach does seem to be somewhat limiting, but it’s got me thinking in the right direction. It seems like the US Federal CBA system is pretty ad-hoc. As far as I can tell each department is responsible for producing their own CBAs, and they each use their own methodology for quantifying things. Do you know if that’s right, or is there some standardized methodology that I’m not seeing?
Yes, you’re correct that each department produces its own CBAs. However, the Office of Management and Budget (OMB) does help standardize these CBAs to some extent. For example, the federal government sets up “interagency working groups” to standardize specific inputs (e.g. the social cost of carbon) across departments (more info on social cost of carbon standardization). And, when OIRA reviews CBAs, they make sure departments are following some general standard processes too.
I see. I’ve got some more research to do. Thanks again.