Looks like we are mostly on the same page. We both recognise the need for theoretical data and empirical data to play a role and we both think that you have a good idea for prison reform.
I still get the impression that you undervalue empirical evidence of existent systems compared to theoretical evidence and may under invest in understanding evidence that goes against the theory or could improve the model. (Or may be I am being too harsh and we agree here too, hard to judge from a short exchange like this.) I am not sure I can persuade you to change much on this but I go into detail on a few points below.
Anyway even if you are not persuaded I expect (well hope) that you would need to gather the empirical evidence before any senior policy makers look to implement this so either way that seems like a good next step. Good luck :-)
TO ADDRESS THE POINTS RAISED:
Good theoretical evidence is “actual evidence”
Firstly, apologies. I am not sure I explained things very well. Was late and I minced my words a bit. By “actual evidence” I was trying to just encompass the case of a similar policy already being in place and working. Eg we know tobacco tax works well at achieving the policy aim of reducing smoking because we can see it working. Sorry for any confusion caused.
Can you show me a theoretical model of school building that would convince me that it would work when it would, in fact, fail?
A better example from development is microcredit (mirofinace). Basically everyone was convinced by the theory of small loans to those too poor to receive finance. The guy who came up with the idea got a freking Nobel Prize. Super-skeptics GiveWell used to have a page on the best microcredit charity. But turns out (from multiple meta-analyses) that there was basically no way to make it work in practice (not for the worlds poorest).
Any prison system that does so [works] will look similar to mine, e.g. prisons would need to get paid when convicts pay tax.
Blanket statements like this – suggesting your idea or similar is the ONLY way prisons can work still concerns me and makes me think that you value theoretical data too highly compared to empirical data. I don’t know much about prisons systems but I would be shocked if there was NO other good way to have a well managed prison system.
A pilot prison wouldn’t work because it wouldn’t have competitive bidding.
I still think it could help the case to think about how a pilot prison could be made to produce useful data. Could the prison bid against the state somehow? Or could it work with two prisons? Or one prison and two price streams?
Basically everyone was convinced by the theory of small loans to those too poor to receive finance.
I was against microfinance, but I also don’t know how they justified the idea. I think empirical evidence should be used to undermine certain assumptions of models, such as “People will only take out a loan if it’s in their own interests”. Empirically, that’s clearly not always the case (e.g. people go bankrupt from credit cards), and any model that relies on that statement being true may fail because of it. A theoretical argument with bad assumptions is a bad argument.
“Any prison system that does so [works] will look similar to mine, e.g. prisons would need to get paid when convicts pay tax.”
Blanket statements like this – suggesting your idea or similar is the ONLY way prisons can work still concerns me and makes me think that you value theoretical data too highly compared to empirical data.
That’s not what I said. I said “Most systems can work well in the short run”, but systems that don’t internalize externalities are brittle (“For a system to work well and be robust”). Internalizing externalities has a very strong evidence base (theoretical and empirical). If anyone can show me a system that internalizes externalities and doesn’t look like my proposal, I will concede.
I still think it could help the case to think about how a pilot prison could be made to produce useful data.
I think we agree that you want “cheap and informative tests”. Some of the data you’re talking about already exists (income tax data, welfare payments, healthcare subsidies), which is cheap (because it already exists) and informative (because it’s close to the data prisons would need to operate effectively).
Social impact bonds for prisons are already in place, and that system is similar to mine in some respects (though it has poor, ad hoc justification, and so the system should fail in predictable ways). You’re right about me not being interested those systems. Social impact bonds are probably the best reference class we have for my proposal. But if they failed, I wouldn’t update my beliefs very much since the theory for the impact bonds is so poor.
Hmm. Actually, you’re right, you can make a small trial work.
1. Randomly select enough inmates to fill n US prisons. Randomly put 50 percent of those inmates in set A, and the other inmates in set B.
2. Allow any prison to bid on the contracts of set B. The participating prisons have to bid on all of those inmates’ contracts.
3. Select the n/2 prisons in such a way that those prisons are filled, and that the sum of the bids is maximized. Because all prisons are bidding, you get a thick market, and the assumptions of my argument should hold (you may have to compensate prisons who make reasonable bids).
4. Select n/2 random prisons to take on set A. Does this introduce selection bias? Yes, and that’s exactly the point. In my proposal, the best prisons self-select by making higher bids (criterion 1).
Unfortunately I don’t have much useful to contribute on this. I don’t have experience running trials and pilots. I would think through the various scenarios by which a pilot could get started and then adapt to that. Eg what if you had the senior management of one prison that was keen. What about a single state. What about a few prisons. Also worth recognising that data might take years.
I used to know someone who worked on prison data collection and assessing success of prisons, if I see her at some point I could raise this and message you.
MAIN POINTS:
Looks like we are mostly on the same page. We both recognise the need for theoretical data and empirical data to play a role and we both think that you have a good idea for prison reform.
I still get the impression that you undervalue empirical evidence of existent systems compared to theoretical evidence and may under invest in understanding evidence that goes against the theory or could improve the model. (Or may be I am being too harsh and we agree here too, hard to judge from a short exchange like this.) I am not sure I can persuade you to change much on this but I go into detail on a few points below.
Anyway even if you are not persuaded I expect (well hope) that you would need to gather the empirical evidence before any senior policy makers look to implement this so either way that seems like a good next step. Good luck :-)
TO ADDRESS THE POINTS RAISED:
Firstly, apologies. I am not sure I explained things very well. Was late and I minced my words a bit. By “actual evidence” I was trying to just encompass the case of a similar policy already being in place and working. Eg we know tobacco tax works well at achieving the policy aim of reducing smoking because we can see it working. Sorry for any confusion caused.
A better example from development is microcredit (mirofinace). Basically everyone was convinced by the theory of small loans to those too poor to receive finance. The guy who came up with the idea got a freking Nobel Prize. Super-skeptics GiveWell used to have a page on the best microcredit charity. But turns out (from multiple meta-analyses) that there was basically no way to make it work in practice (not for the worlds poorest).
Blanket statements like this – suggesting your idea or similar is the ONLY way prisons can work still concerns me and makes me think that you value theoretical data too highly compared to empirical data. I don’t know much about prisons systems but I would be shocked if there was NO other good way to have a well managed prison system.
I still think it could help the case to think about how a pilot prison could be made to produce useful data. Could the prison bid against the state somehow? Or could it work with two prisons? Or one prison and two price streams?
I was against microfinance, but I also don’t know how they justified the idea. I think empirical evidence should be used to undermine certain assumptions of models, such as “People will only take out a loan if it’s in their own interests”. Empirically, that’s clearly not always the case (e.g. people go bankrupt from credit cards), and any model that relies on that statement being true may fail because of it. A theoretical argument with bad assumptions is a bad argument.
That’s not what I said. I said “Most systems can work well in the short run”, but systems that don’t internalize externalities are brittle (“For a system to work well and be robust”). Internalizing externalities has a very strong evidence base (theoretical and empirical). If anyone can show me a system that internalizes externalities and doesn’t look like my proposal, I will concede.
I think we agree that you want “cheap and informative tests”. Some of the data you’re talking about already exists (income tax data, welfare payments, healthcare subsidies), which is cheap (because it already exists) and informative (because it’s close to the data prisons would need to operate effectively).
Social impact bonds for prisons are already in place, and that system is similar to mine in some respects (though it has poor, ad hoc justification, and so the system should fail in predictable ways). You’re right about me not being interested those systems. Social impact bonds are probably the best reference class we have for my proposal. But if they failed, I wouldn’t update my beliefs very much since the theory for the impact bonds is so poor.
Hmm. Actually, you’re right, you can make a small trial work.
1. Randomly select enough inmates to fill n US prisons. Randomly put 50 percent of those inmates in set A, and the other inmates in set B.
2. Allow any prison to bid on the contracts of set B. The participating prisons have to bid on all of those inmates’ contracts.
3. Select the n/2 prisons in such a way that those prisons are filled, and that the sum of the bids is maximized. Because all prisons are bidding, you get a thick market, and the assumptions of my argument should hold (you may have to compensate prisons who make reasonable bids).
4. Select n/2 random prisons to take on set A. Does this introduce selection bias? Yes, and that’s exactly the point. In my proposal, the best prisons self-select by making higher bids (criterion 1).
5. Observe.
I’m interested to hear what you think.
Unfortunately I don’t have much useful to contribute on this. I don’t have experience running trials and pilots. I would think through the various scenarios by which a pilot could get started and then adapt to that. Eg what if you had the senior management of one prison that was keen. What about a single state. What about a few prisons. Also worth recognising that data might take years.
I used to know someone who worked on prison data collection and assessing success of prisons, if I see her at some point I could raise this and message you.