Huh, yeah, I disagree. It seems to me pretty fundamental to a justice system’s credibility that it not imprison one person and free another when the only difference between them is the color of their skin (or, yes, their height), and it makes a lot of sense to me that U.S. law mandates sacrificing predictive power in order to maintain this feature of the system.
Similarly, I don’t think all of the restrictions the legal system imposes on what kinds of evidence to use are, in fact, motivated by long-term harm-reduction considerations. I think they’re motivated by wanting the system to embody the ideal of justice. EAs are mostly consequentialists (I am) and mostly only interested in harm, not in fairness, but I think it’s important to realize that the overwhelming majority of people care a lot about whether a justice system is fair in addition to whether it is harm-reducing, and that this is the actual motivation for the laws I discuss above, even if you can technically propose a defense of them in harm-reduction terms.
but I think it’s important to realize that the overwhelming majority of people care a lot about whether a justice system is fair in addition to whether it is harm-reducing, and that this is the actual motivation for the laws I discuss above, even if you can technically propose a defense of them in harm-reduction terms.
I take the same stance towards moral arguments as I do towards epistemic ones. I would be very sad if EAs make moral arguments that are based on bad moral reasoning just because they appeal to the preconceived notions of some parts of society. I think most arguments in favor of naive conceptions of fairness fall into this category, and I would strongly prefer to advocate for moral stances that we feel confident in, have checked their consistency and feel comfortable defending on our own grounds.
Hmm. I think I’m thinking of concern for justice-system outcomes as a values difference rather than a reasoning error, and so treating it as legitimate feels appropriate in the same way it feels appropriate to say ‘an AI with poorly specified goals could wirehead everyone, which is an example of optimizing for one thing we wanted at the expense of other things we wanted’ even though I don’t actually feel that confident that my preferences against wireheading everyone are principled and consistent.
I agree that most peoples’ conceptions of fairness are inconsistent, but that’s only because most peoples’ values are inconsistent in general; I don’t think it means they’d necessarily have my values if they thought about it more. I also think that ‘the U.S. government should impose the same prison sentence for the same crime regardless of the race of the defendant’ is probably correct under my value system, which probably influences me towards thinking that other people who value it would still value it if they were less confused.
Some instrumental merits of imposing the same prison sentence for the same crime regardless of the race of the defendant:
I want to gesture at something in the direction of pluralism: we agree to treat all religions the same, not because they are of equal social value or because we think they are equally correct, but because this is social technology to prevent constantly warring over whose religion is correct/of the most social value. I bet some religious beliefs predict less recidivism, but I prefer not using religion to determine sentencing because I think there are a lot of practical benefits to the pluralistic compromise the U.S. uses here. This generalizes to race.
There are ways you can greatly exacerbate an initially fairly small difference by updating on it in ways that are all technically correct. I think the classic example is a career path with lots of promotions, where one thing people are optimizing for at each level is the odds of being promoted at the next level; this will result in a very small difference in average ability producing a huge difference in odds of reaching the highest level. I think it is good for systems like the U.S. justice system to try to adopt procedures that avoid this, where this is sane and the tradeoffs relatively small.
(least important): Justice systems run on social trust. If they use processes which undermine social trust, even if they do this because the public is objectively unreasonable, they will work less well; people will be less likely to report crimes, cooperate with police, testify, serve on juries, make truthful decisions on juries, etc. I know that when crimes are committed against me, I weigh whether I expect the justice system to behave according to my values when deciding whether to report the crimes. If this is common, there’s reason for justice systems to use processes that people consider aligned. If we want to change what people value, we should use instruments for this other than the justice system.
Expanding on this: I don’t think ‘fairness’ is a fundamental part of morality. It’s better for good things to happen than bad ones, regardless of how they’re distributed, and it’s bad to sacrifice utility for fairness.
However, I think there are some aspects of policy where fairness is instrumentally really useful, and I think the justice system is the single place where it’s most useful, and the will/preferences of the American populace is demonstrably for a justice system to embody fairness, and so it seems to me that we’re missing a really important point if we decide that it’s not a problem for a justice system to badly fail to embody the values it was intended to embody just because we don’t non-instrumentally value fairness.
My perspective here is that many forms of fairness are inconsistent, and fall apart on significant moral introspection as you try to make your moral preferences consistent. I think the skin-color thing is one of them, which is really hard to maintain as something that you shouldn’t pay attention to, as you realize that it can’t be causally disentangled from other factors that you feel like you definitely should pay attention to (such as the person’s physical strength, or their height, or the speed at which they can run).
Not paying attention to skin color has to mean that you don’t pay attention to physical strength, since those are causally entangled in a way that makes it impossible to pay attention to one without the other. You won’t ever have full information on someone’s physical strength, so hearing about their ethnic background will always give you additional evidence. Skin color is not an isolated epiphenomenal node in the causal structure of the world, and you can’t just decide to “not discriminate on it” without stopping to pay attention to every single phenomenon that is correlated with it and that you can’t fully screen off, which is a really large range of things that you would definitely want to know in a criminal investigation.
My perspective here is that many forms of fairness are inconsistent, and fall apart on significant moral introspection as you try to make your moral preferences consistent. I think the skin-color thing is one of them, which is really hard to maintain as something that you shouldn’t pay attention to, as you realize that it can’t be causally disentangled from other factors that you feel like you definitely should pay attention to (such as the person’s physical strength, or their height, or the speed at which they can run).
I think that a sensible interpretation of “is the justice system (or society in general) fair” is “does the justice system (or society) reward behaviors that are good overall, and punish behaviors that are bad overall”; in other words, can you count on society to cooperate with you rather than defect on you if you cooperate with it. If you get jailed based (in part) on your skin color, then if you have the wrong skin color (which you can’t affect), there’s an increased probability of society defecting on you regardless of whether you cooperate or defect. This means that you have an extra incentive to defect since you might get defected on anyway. This feels like a sensible thing to try to avoid.
This is not for criminal investigation. This is for, when a person has been convicted of a crime, estimating when to release them (by estimating how likely they are to commit another crime).
Will write a longer reply later, since I am about to board a plane.
I was indeed thinking of a criminal investigation context, but I think the question of how likely someone is to commit further crimes is likely to be directly related to their ability to commit further crimes, which will depend on many of the variables I mentioned above, and so the same argument holds.
I expect those variables to still be highly relevant when you want to assess the likelihood of another crime, and there are many more that are more obviously relevant and also correlated with race (such as their impulsivity, their likelihood to get addicted to drugs, etc.). Do you think we should not take into account someone’s impulsivity when predicting whether they will commit more crimes?
Huh, yeah, I disagree. It seems to me pretty fundamental to a justice system’s credibility that it not imprison one person and free another when the only difference between them is the color of their skin (or, yes, their height), and it makes a lot of sense to me that U.S. law mandates sacrificing predictive power in order to maintain this feature of the system.
If the crime was performed by someone who had to be at least 2m tall, and one of the suspects is 2.10m and other one is 1.60m tall, then it seems really obvious to me that you should use their height as evidence? I would be deeply surprised if you think otherwise.
If the crime was performed by someone who had to be at least 2m tall, and one of the suspects is 2.10m and other one is 1.60m tall, then it seems really obvious to me that you should use their height as evidence?
That’s not what these articles describe—the algorithm in question wasn’t being used to determine whether a suspect had committed a crime, it was being used for risk assessment, ie determine the probability that a person convicted of one crime will go on to commit another.
Huh, yeah, I disagree. It seems to me pretty fundamental to a justice system’s credibility that it not imprison one person and free another when the only difference between them is the color of their skin (or, yes, their height), and it makes a lot of sense to me that U.S. law mandates sacrificing predictive power in order to maintain this feature of the system.
Similarly, I don’t think all of the restrictions the legal system imposes on what kinds of evidence to use are, in fact, motivated by long-term harm-reduction considerations. I think they’re motivated by wanting the system to embody the ideal of justice. EAs are mostly consequentialists (I am) and mostly only interested in harm, not in fairness, but I think it’s important to realize that the overwhelming majority of people care a lot about whether a justice system is fair in addition to whether it is harm-reducing, and that this is the actual motivation for the laws I discuss above, even if you can technically propose a defense of them in harm-reduction terms.
I take the same stance towards moral arguments as I do towards epistemic ones. I would be very sad if EAs make moral arguments that are based on bad moral reasoning just because they appeal to the preconceived notions of some parts of society. I think most arguments in favor of naive conceptions of fairness fall into this category, and I would strongly prefer to advocate for moral stances that we feel confident in, have checked their consistency and feel comfortable defending on our own grounds.
Hmm. I think I’m thinking of concern for justice-system outcomes as a values difference rather than a reasoning error, and so treating it as legitimate feels appropriate in the same way it feels appropriate to say ‘an AI with poorly specified goals could wirehead everyone, which is an example of optimizing for one thing we wanted at the expense of other things we wanted’ even though I don’t actually feel that confident that my preferences against wireheading everyone are principled and consistent.
I agree that most peoples’ conceptions of fairness are inconsistent, but that’s only because most peoples’ values are inconsistent in general; I don’t think it means they’d necessarily have my values if they thought about it more. I also think that ‘the U.S. government should impose the same prison sentence for the same crime regardless of the race of the defendant’ is probably correct under my value system, which probably influences me towards thinking that other people who value it would still value it if they were less confused.
Some instrumental merits of imposing the same prison sentence for the same crime regardless of the race of the defendant:
I want to gesture at something in the direction of pluralism: we agree to treat all religions the same, not because they are of equal social value or because we think they are equally correct, but because this is social technology to prevent constantly warring over whose religion is correct/of the most social value. I bet some religious beliefs predict less recidivism, but I prefer not using religion to determine sentencing because I think there are a lot of practical benefits to the pluralistic compromise the U.S. uses here. This generalizes to race.
There are ways you can greatly exacerbate an initially fairly small difference by updating on it in ways that are all technically correct. I think the classic example is a career path with lots of promotions, where one thing people are optimizing for at each level is the odds of being promoted at the next level; this will result in a very small difference in average ability producing a huge difference in odds of reaching the highest level. I think it is good for systems like the U.S. justice system to try to adopt procedures that avoid this, where this is sane and the tradeoffs relatively small.
(least important): Justice systems run on social trust. If they use processes which undermine social trust, even if they do this because the public is objectively unreasonable, they will work less well; people will be less likely to report crimes, cooperate with police, testify, serve on juries, make truthful decisions on juries, etc. I know that when crimes are committed against me, I weigh whether I expect the justice system to behave according to my values when deciding whether to report the crimes. If this is common, there’s reason for justice systems to use processes that people consider aligned. If we want to change what people value, we should use instruments for this other than the justice system.
Expanding on this: I don’t think ‘fairness’ is a fundamental part of morality. It’s better for good things to happen than bad ones, regardless of how they’re distributed, and it’s bad to sacrifice utility for fairness.
However, I think there are some aspects of policy where fairness is instrumentally really useful, and I think the justice system is the single place where it’s most useful, and the will/preferences of the American populace is demonstrably for a justice system to embody fairness, and so it seems to me that we’re missing a really important point if we decide that it’s not a problem for a justice system to badly fail to embody the values it was intended to embody just because we don’t non-instrumentally value fairness.
My perspective here is that many forms of fairness are inconsistent, and fall apart on significant moral introspection as you try to make your moral preferences consistent. I think the skin-color thing is one of them, which is really hard to maintain as something that you shouldn’t pay attention to, as you realize that it can’t be causally disentangled from other factors that you feel like you definitely should pay attention to (such as the person’s physical strength, or their height, or the speed at which they can run).
Not paying attention to skin color has to mean that you don’t pay attention to physical strength, since those are causally entangled in a way that makes it impossible to pay attention to one without the other. You won’t ever have full information on someone’s physical strength, so hearing about their ethnic background will always give you additional evidence. Skin color is not an isolated epiphenomenal node in the causal structure of the world, and you can’t just decide to “not discriminate on it” without stopping to pay attention to every single phenomenon that is correlated with it and that you can’t fully screen off, which is a really large range of things that you would definitely want to know in a criminal investigation.
I think that a sensible interpretation of “is the justice system (or society in general) fair” is “does the justice system (or society) reward behaviors that are good overall, and punish behaviors that are bad overall”; in other words, can you count on society to cooperate with you rather than defect on you if you cooperate with it. If you get jailed based (in part) on your skin color, then if you have the wrong skin color (which you can’t affect), there’s an increased probability of society defecting on you regardless of whether you cooperate or defect. This means that you have an extra incentive to defect since you might get defected on anyway. This feels like a sensible thing to try to avoid.
This is not for criminal investigation. This is for, when a person has been convicted of a crime, estimating when to release them (by estimating how likely they are to commit another crime).
Will write a longer reply later, since I am about to board a plane.
I was indeed thinking of a criminal investigation context, but I think the question of how likely someone is to commit further crimes is likely to be directly related to their ability to commit further crimes, which will depend on many of the variables I mentioned above, and so the same argument holds.
I expect those variables to still be highly relevant when you want to assess the likelihood of another crime, and there are many more that are more obviously relevant and also correlated with race (such as their impulsivity, their likelihood to get addicted to drugs, etc.). Do you think we should not take into account someone’s impulsivity when predicting whether they will commit more crimes?
If the crime was performed by someone who had to be at least 2m tall, and one of the suspects is 2.10m and other one is 1.60m tall, then it seems really obvious to me that you should use their height as evidence? I would be deeply surprised if you think otherwise.
That’s not what these articles describe—the algorithm in question wasn’t being used to determine whether a suspect had committed a crime, it was being used for risk assessment, ie determine the probability that a person convicted of one crime will go on to commit another.