Correct me if I am wrong but you seem to be implying that the “theoretical reasons” why a policy idea will work are necessary and more important than empirical evidence that a system has worked in some case (which may be misleading due to confounding factors like good people).
If so I strongly disagree:
Based on my 7 years experience working in UK policy would lead me to say the opposite. Theoretical reasons are great but actual evidence that a particular system has worked is super great, and in most cases more important.
Of course both can be useful. The world is complicated and policy is complicated and both evidence and theory can lead you down the wrong path. Good theoretical policy ideas can turn out to be wrong and well-evidence policy idea may not replicate as expected.
Consider international development. The effective altruism community has been saying for years (and backing up these claims) that in development you cannot just do things that theoretically sound like they will work (like building schools) but you need to do things that have empirical evidence of working well.
People are very very good at persuading themselves in what they believe (eg confirmation bias). A risk with policies driven by theoretical reasoning is that its adherents have ideological baggage and motivated reasoning and do not shift in line with new evidence. This is less of a risk for policy driven based on what works.
ON PRISONS
I have not considered all the details but I do think you have a decent policy idea here. I would be interested to see it tried. I would make the following, hopefully constructive, suggestions to you.
1.
Focus on countries where the prison system is actually broken
There is a lot of failings in policy and limited capacity to address them all. I do think “if it is not broke don’t fix it” is often a good maxim in policy and countries with working systems should not be the first to shift to the system you describe.
2.
Be wary of the risk of motivated reasoning
If the UK system currently works well, I suspect that you have good regulators who are manually handling the shortcomings of the underlying system.
Nothing you said substantiates this claim and from what I know about the UK system (which is admittedly minimal) I don’t think this is the case. Now this claim might be true and you might have good evidence for it that you didn’t state, but it did raise a red flag in my mind when I red it.
3.
Don’t under-value evidence, you might miss things. An underplayed strength of your case for fixing private prisons is that the solution you suggest is testable. A single pilot prison could be run and data collected and lessons learned. To some degree this could even be done by an committed entrepreneur with minimal government support.
4.
Look at what can be learned from systems that work elsewhere. Eg a feature of the UK system is that there are both state-run and private prisons. These can and have been compared and can and have acted as a check on each other. If one is clearly failing it motivates change in the other. This learning can make you case for trailing auction based prisons stronger as you can highlight how different systems running in parallel act as a check on each other. Yet at the same time this learning also makes the case for running 100% private auction based prisons weaker as maybe some amount of state-run prisons can provide a useful check on the system.
Theoretical reasons are great but actual evidence [is] more important.
Good theoretical evidence is “actual evidence”. No amount of empirical evidence is up to the task of proving there are an infinite number of primes. Our theoretical argument showing there are an infinite number of primes is the strongest form of evidence that can be given.
That’s not to say I think my argument is airtight, however. My argument could probably be made with more realistic assumptions (alternatively, more realistic assumptions might show my proposed system is fundamentally mistaken). My model also just describes an end state, rather than provide any help on how to build this market from scratch (implementing the system overnight would produce a market that’s too thin and inefficient).
Theory can go wrong if the assumptions are false or the inferences are invalid. Both of these errors happen all the time of course, and in subtle ways too, so I agree that empirical evidence is important. But even with data, you need a model to properly interpret it. Data without a model only tells you what was measured, which usually isn’t that informative. No matter what the numbers are, no one can say the UK prison system is better than that of the US without some assumptions. And no one can get data on a system that hasn’t been tried before (depending on the reference classes available to you).
Consider international development. The effective altruism community has been saying for years (and backing up these claims) that in development you cannot just do things that theoretically sound like they will work (like building schools) but you need to do things that have empirical evidence of working well.
Can you show me a theoretical model of school building that would convince me that it would work when it would, in fact, fail? I don’t think I would be convinced. (How can you be sure the teachers are good? How can you be sure good teachers will stay? How can you be sure students will show up?) You can’t bundle all theory in the same basket (otherwise I could point to mathematics and say theory is always sound). Whether a theory is good evidence hinges on whether the assumptions are close enough to reality.
People are very very good at persuading themselves in what they believe [...]
Be wary of the risk of motivated reasoning
[But your] claim might be true and you might have good evidence for it
The process of building the argument changed what I believed the system should be. I have no dog in this fight. I claimed “If the UK system currently works well, I suspect that you have good regulators who are manually handling the shortcomings of the underlying system” for several reasons:
it is true of so many systems I’ve seen, including all of the prison systems I’ve seen (reference class)
For a system to work well and be robust, externalities need to be internalized. Any prison system that does so will look similar to mine, e.g. prisons would need to get paid when convicts pay tax. You would have mentioned if the UK system did so.
The data I’ve seen on UK prisons is about 50% recidivism, which I don’t think would be the case under a functional system.
Don’t under-value evidence, you might miss things. An underplayed strength of your case for fixing private prisons is that the solution you suggest is testable. A single pilot prison could be run and data collected and lessons learned. To some degree this could even be done by an committed entrepreneur with minimal government support.
A pilot prison wouldn’t work because it wouldn’t have competitive bidding. I did mention in Aaron Gertler’s comment what data should be collated prior to implementation. I’m all for looking at the relevant data.
If one [UK public prisons or UK private prisons are] clearly failing it motivates change in the other.
That doesn’t seem like a good system.
The bidding process and the actualisation of losses (tied to real social interests) keep the prisons in check.
That doesn’t seem like a good system. The bidding process and the actualisation of losses (tied to real social interests) keep the prisons in check.
I strongly disagree. Additional checks and balances that prevent serious problems occurring are good. You have already said your system could go wrong (you said “more realistic assumptions might show my proposed system is fundamentally mistaken”) and maybe it could go wrong in subtle ways that take years to manifest as companies learn how they can twist the rules.
You should be in favour of checks and balances, and might want to explore what additional systems of checks would work best for your proposal. Options include: A few prisons running on a different system (eg state-run). A regulator for your auction based prisons. Transparency. The prisons being on 10 year loans from the state with contacts the need regular renewing so they would default to state ownership. Human rights laws. Etc. Maybe all of the above are things to have.
As an example, one thing that could go wrong (although it looks like you have touched on this elsewhere in the comments) is prisons may not have a strong incentive to care about the welfare of the prisoners whilst they are in the prison.
Ah yes, I have mentioned in other comments about regulation keeping private prisons in check too. I should have restated it here. I am in favour of checks and balances, which is why my goal for the system contains ”...within the limits of the law”. I agree with almost everything you say here (I’d keep some public prisons until confident that the market is mature enough to handle all cases better than the public system, but I wouldn’t implement your 10-year loan).
Human rights laws. Etc.
Yep, I’m all for that. One thing that people are missing is that the goal of a system kind of… “compresses” the space of things you want to happen. That compression is lossy. You want that goal to lose as few things as possible, but you will lose some things. To fix that, you will need some regulation to make sure the system works in the important edge cases.
prisons may not have a strong incentive to care about the welfare of the prisoners whilst they are in the prison
This is incorrect. They do have a strong incentive, since the contract comes into effect immediately after the auction: If a crime happens in their prison, the prison has to pay the government. The resulting problem is that prisons have an incentive to hide these crimes. So I recommended that prisons be outfitted with cameras and microphones that are externally audited.
Looks like we are mostly on the same page. We both recognise the need for theoretical data and empirical data to play a role and we both think that you have a good idea for prison reform.
I still get the impression that you undervalue empirical evidence of existent systems compared to theoretical evidence and may under invest in understanding evidence that goes against the theory or could improve the model. (Or may be I am being too harsh and we agree here too, hard to judge from a short exchange like this.) I am not sure I can persuade you to change much on this but I go into detail on a few points below.
Anyway even if you are not persuaded I expect (well hope) that you would need to gather the empirical evidence before any senior policy makers look to implement this so either way that seems like a good next step. Good luck :-)
TO ADDRESS THE POINTS RAISED:
Good theoretical evidence is “actual evidence”
Firstly, apologies. I am not sure I explained things very well. Was late and I minced my words a bit. By “actual evidence” I was trying to just encompass the case of a similar policy already being in place and working. Eg we know tobacco tax works well at achieving the policy aim of reducing smoking because we can see it working. Sorry for any confusion caused.
Can you show me a theoretical model of school building that would convince me that it would work when it would, in fact, fail?
A better example from development is microcredit (mirofinace). Basically everyone was convinced by the theory of small loans to those too poor to receive finance. The guy who came up with the idea got a freking Nobel Prize. Super-skeptics GiveWell used to have a page on the best microcredit charity. But turns out (from multiple meta-analyses) that there was basically no way to make it work in practice (not for the worlds poorest).
Any prison system that does so [works] will look similar to mine, e.g. prisons would need to get paid when convicts pay tax.
Blanket statements like this – suggesting your idea or similar is the ONLY way prisons can work still concerns me and makes me think that you value theoretical data too highly compared to empirical data. I don’t know much about prisons systems but I would be shocked if there was NO other good way to have a well managed prison system.
A pilot prison wouldn’t work because it wouldn’t have competitive bidding.
I still think it could help the case to think about how a pilot prison could be made to produce useful data. Could the prison bid against the state somehow? Or could it work with two prisons? Or one prison and two price streams?
Basically everyone was convinced by the theory of small loans to those too poor to receive finance.
I was against microfinance, but I also don’t know how they justified the idea. I think empirical evidence should be used to undermine certain assumptions of models, such as “People will only take out a loan if it’s in their own interests”. Empirically, that’s clearly not always the case (e.g. people go bankrupt from credit cards), and any model that relies on that statement being true may fail because of it. A theoretical argument with bad assumptions is a bad argument.
“Any prison system that does so [works] will look similar to mine, e.g. prisons would need to get paid when convicts pay tax.”
Blanket statements like this – suggesting your idea or similar is the ONLY way prisons can work still concerns me and makes me think that you value theoretical data too highly compared to empirical data.
That’s not what I said. I said “Most systems can work well in the short run”, but systems that don’t internalize externalities are brittle (“For a system to work well and be robust”). Internalizing externalities has a very strong evidence base (theoretical and empirical). If anyone can show me a system that internalizes externalities and doesn’t look like my proposal, I will concede.
I still think it could help the case to think about how a pilot prison could be made to produce useful data.
I think we agree that you want “cheap and informative tests”. Some of the data you’re talking about already exists (income tax data, welfare payments, healthcare subsidies), which is cheap (because it already exists) and informative (because it’s close to the data prisons would need to operate effectively).
Social impact bonds for prisons are already in place, and that system is similar to mine in some respects (though it has poor, ad hoc justification, and so the system should fail in predictable ways). You’re right about me not being interested those systems. Social impact bonds are probably the best reference class we have for my proposal. But if they failed, I wouldn’t update my beliefs very much since the theory for the impact bonds is so poor.
Hmm. Actually, you’re right, you can make a small trial work.
1. Randomly select enough inmates to fill n US prisons. Randomly put 50 percent of those inmates in set A, and the other inmates in set B.
2. Allow any prison to bid on the contracts of set B. The participating prisons have to bid on all of those inmates’ contracts.
3. Select the n/2 prisons in such a way that those prisons are filled, and that the sum of the bids is maximized. Because all prisons are bidding, you get a thick market, and the assumptions of my argument should hold (you may have to compensate prisons who make reasonable bids).
4. Select n/2 random prisons to take on set A. Does this introduce selection bias? Yes, and that’s exactly the point. In my proposal, the best prisons self-select by making higher bids (criterion 1).
Unfortunately I don’t have much useful to contribute on this. I don’t have experience running trials and pilots. I would think through the various scenarios by which a pilot could get started and then adapt to that. Eg what if you had the senior management of one prison that was keen. What about a single state. What about a few prisons. Also worth recognising that data might take years.
I used to know someone who worked on prison data collection and assessing success of prisons, if I see her at some point I could raise this and message you.
Hi,
THEORY AND EVIDENCE
Correct me if I am wrong but you seem to be implying that the “theoretical reasons” why a policy idea will work are necessary and more important than empirical evidence that a system has worked in some case (which may be misleading due to confounding factors like good people).
If so I strongly disagree:
Based on my 7 years experience working in UK policy would lead me to say the opposite. Theoretical reasons are great but actual evidence that a particular system has worked is super great, and in most cases more important.
Of course both can be useful. The world is complicated and policy is complicated and both evidence and theory can lead you down the wrong path. Good theoretical policy ideas can turn out to be wrong and well-evidence policy idea may not replicate as expected.
Consider international development. The effective altruism community has been saying for years (and backing up these claims) that in development you cannot just do things that theoretically sound like they will work (like building schools) but you need to do things that have empirical evidence of working well.
People are very very good at persuading themselves in what they believe (eg confirmation bias). A risk with policies driven by theoretical reasoning is that its adherents have ideological baggage and motivated reasoning and do not shift in line with new evidence. This is less of a risk for policy driven based on what works.
ON PRISONS
I have not considered all the details but I do think you have a decent policy idea here. I would be interested to see it tried. I would make the following, hopefully constructive, suggestions to you.
1.
Focus on countries where the prison system is actually broken
There is a lot of failings in policy and limited capacity to address them all. I do think “if it is not broke don’t fix it” is often a good maxim in policy and countries with working systems should not be the first to shift to the system you describe.
2.
Be wary of the risk of motivated reasoning
Nothing you said substantiates this claim and from what I know about the UK system (which is admittedly minimal) I don’t think this is the case. Now this claim might be true and you might have good evidence for it that you didn’t state, but it did raise a red flag in my mind when I red it.
3.
Don’t under-value evidence, you might miss things. An underplayed strength of your case for fixing private prisons is that the solution you suggest is testable. A single pilot prison could be run and data collected and lessons learned. To some degree this could even be done by an committed entrepreneur with minimal government support.
4.
Look at what can be learned from systems that work elsewhere. Eg a feature of the UK system is that there are both state-run and private prisons. These can and have been compared and can and have acted as a check on each other. If one is clearly failing it motivates change in the other. This learning can make you case for trailing auction based prisons stronger as you can highlight how different systems running in parallel act as a check on each other. Yet at the same time this learning also makes the case for running 100% private auction based prisons weaker as maybe some amount of state-run prisons can provide a useful check on the system.
Hope that helps.
Good theoretical evidence is “actual evidence”. No amount of empirical evidence is up to the task of proving there are an infinite number of primes. Our theoretical argument showing there are an infinite number of primes is the strongest form of evidence that can be given.
That’s not to say I think my argument is airtight, however. My argument could probably be made with more realistic assumptions (alternatively, more realistic assumptions might show my proposed system is fundamentally mistaken). My model also just describes an end state, rather than provide any help on how to build this market from scratch (implementing the system overnight would produce a market that’s too thin and inefficient).
Theory can go wrong if the assumptions are false or the inferences are invalid. Both of these errors happen all the time of course, and in subtle ways too, so I agree that empirical evidence is important. But even with data, you need a model to properly interpret it. Data without a model only tells you what was measured, which usually isn’t that informative. No matter what the numbers are, no one can say the UK prison system is better than that of the US without some assumptions. And no one can get data on a system that hasn’t been tried before (depending on the reference classes available to you).
Can you show me a theoretical model of school building that would convince me that it would work when it would, in fact, fail? I don’t think I would be convinced. (How can you be sure the teachers are good? How can you be sure good teachers will stay? How can you be sure students will show up?) You can’t bundle all theory in the same basket (otherwise I could point to mathematics and say theory is always sound). Whether a theory is good evidence hinges on whether the assumptions are close enough to reality.
The process of building the argument changed what I believed the system should be. I have no dog in this fight. I claimed “If the UK system currently works well, I suspect that you have good regulators who are manually handling the shortcomings of the underlying system” for several reasons:
it is true of so many systems I’ve seen, including all of the prison systems I’ve seen (reference class)
For a system to work well and be robust, externalities need to be internalized. Any prison system that does so will look similar to mine, e.g. prisons would need to get paid when convicts pay tax. You would have mentioned if the UK system did so.
The data I’ve seen on UK prisons is about 50% recidivism, which I don’t think would be the case under a functional system.
A pilot prison wouldn’t work because it wouldn’t have competitive bidding. I did mention in Aaron Gertler’s comment what data should be collated prior to implementation. I’m all for looking at the relevant data.
That doesn’t seem like a good system.
The bidding process and the actualisation of losses (tied to real social interests) keep the prisons in check.
I strongly disagree. Additional checks and balances that prevent serious problems occurring are good. You have already said your system could go wrong (you said “more realistic assumptions might show my proposed system is fundamentally mistaken”) and maybe it could go wrong in subtle ways that take years to manifest as companies learn how they can twist the rules.
You should be in favour of checks and balances, and might want to explore what additional systems of checks would work best for your proposal. Options include: A few prisons running on a different system (eg state-run). A regulator for your auction based prisons. Transparency. The prisons being on 10 year loans from the state with contacts the need regular renewing so they would default to state ownership. Human rights laws. Etc. Maybe all of the above are things to have.
As an example, one thing that could go wrong (although it looks like you have touched on this elsewhere in the comments) is prisons may not have a strong incentive to care about the welfare of the prisoners whilst they are in the prison.
Ah yes, I have mentioned in other comments about regulation keeping private prisons in check too. I should have restated it here. I am in favour of checks and balances, which is why my goal for the system contains ”...within the limits of the law”. I agree with almost everything you say here (I’d keep some public prisons until confident that the market is mature enough to handle all cases better than the public system, but I wouldn’t implement your 10-year loan).
Yep, I’m all for that. One thing that people are missing is that the goal of a system kind of… “compresses” the space of things you want to happen. That compression is lossy. You want that goal to lose as few things as possible, but you will lose some things. To fix that, you will need some regulation to make sure the system works in the important edge cases.
This is incorrect. They do have a strong incentive, since the contract comes into effect immediately after the auction: If a crime happens in their prison, the prison has to pay the government. The resulting problem is that prisons have an incentive to hide these crimes. So I recommended that prisons be outfitted with cameras and microphones that are externally audited.
MAIN POINTS:
Looks like we are mostly on the same page. We both recognise the need for theoretical data and empirical data to play a role and we both think that you have a good idea for prison reform.
I still get the impression that you undervalue empirical evidence of existent systems compared to theoretical evidence and may under invest in understanding evidence that goes against the theory or could improve the model. (Or may be I am being too harsh and we agree here too, hard to judge from a short exchange like this.) I am not sure I can persuade you to change much on this but I go into detail on a few points below.
Anyway even if you are not persuaded I expect (well hope) that you would need to gather the empirical evidence before any senior policy makers look to implement this so either way that seems like a good next step. Good luck :-)
TO ADDRESS THE POINTS RAISED:
Firstly, apologies. I am not sure I explained things very well. Was late and I minced my words a bit. By “actual evidence” I was trying to just encompass the case of a similar policy already being in place and working. Eg we know tobacco tax works well at achieving the policy aim of reducing smoking because we can see it working. Sorry for any confusion caused.
A better example from development is microcredit (mirofinace). Basically everyone was convinced by the theory of small loans to those too poor to receive finance. The guy who came up with the idea got a freking Nobel Prize. Super-skeptics GiveWell used to have a page on the best microcredit charity. But turns out (from multiple meta-analyses) that there was basically no way to make it work in practice (not for the worlds poorest).
Blanket statements like this – suggesting your idea or similar is the ONLY way prisons can work still concerns me and makes me think that you value theoretical data too highly compared to empirical data. I don’t know much about prisons systems but I would be shocked if there was NO other good way to have a well managed prison system.
I still think it could help the case to think about how a pilot prison could be made to produce useful data. Could the prison bid against the state somehow? Or could it work with two prisons? Or one prison and two price streams?
I was against microfinance, but I also don’t know how they justified the idea. I think empirical evidence should be used to undermine certain assumptions of models, such as “People will only take out a loan if it’s in their own interests”. Empirically, that’s clearly not always the case (e.g. people go bankrupt from credit cards), and any model that relies on that statement being true may fail because of it. A theoretical argument with bad assumptions is a bad argument.
That’s not what I said. I said “Most systems can work well in the short run”, but systems that don’t internalize externalities are brittle (“For a system to work well and be robust”). Internalizing externalities has a very strong evidence base (theoretical and empirical). If anyone can show me a system that internalizes externalities and doesn’t look like my proposal, I will concede.
I think we agree that you want “cheap and informative tests”. Some of the data you’re talking about already exists (income tax data, welfare payments, healthcare subsidies), which is cheap (because it already exists) and informative (because it’s close to the data prisons would need to operate effectively).
Social impact bonds for prisons are already in place, and that system is similar to mine in some respects (though it has poor, ad hoc justification, and so the system should fail in predictable ways). You’re right about me not being interested those systems. Social impact bonds are probably the best reference class we have for my proposal. But if they failed, I wouldn’t update my beliefs very much since the theory for the impact bonds is so poor.
Hmm. Actually, you’re right, you can make a small trial work.
1. Randomly select enough inmates to fill n US prisons. Randomly put 50 percent of those inmates in set A, and the other inmates in set B.
2. Allow any prison to bid on the contracts of set B. The participating prisons have to bid on all of those inmates’ contracts.
3. Select the n/2 prisons in such a way that those prisons are filled, and that the sum of the bids is maximized. Because all prisons are bidding, you get a thick market, and the assumptions of my argument should hold (you may have to compensate prisons who make reasonable bids).
4. Select n/2 random prisons to take on set A. Does this introduce selection bias? Yes, and that’s exactly the point. In my proposal, the best prisons self-select by making higher bids (criterion 1).
5. Observe.
I’m interested to hear what you think.
Unfortunately I don’t have much useful to contribute on this. I don’t have experience running trials and pilots. I would think through the various scenarios by which a pilot could get started and then adapt to that. Eg what if you had the senior management of one prison that was keen. What about a single state. What about a few prisons. Also worth recognising that data might take years.
I used to know someone who worked on prison data collection and assessing success of prisons, if I see her at some point I could raise this and message you.