Thanks for this write-up, I thought it was really interesting and not something I’d ever considered—kudos!
I’ll now hone in on the bit of this I think needs most attention. :)
It seems you think that one of the essential things is developing and using manipulation-proof measures of malevolence. If you were very confident we couldn’t do this, how much of an issue would that be? I raise this because it’s not clear to me how such measures could be created or deployed. It seems you have (1) self-reports, (2) other-reports, (3) objective metrics, e.g. brain scans. If I were really sneaky, I would just lie or not take the test. If I were really sneaky, I would be able to con others, at least for a long-time—perhaps until I was in power. Regarding objective measures, there will be ‘Minority Report’ style objections to actually using them in advance, even if they have high predictive power (which might be tricky as it relies on collecting good data, which seems to require the consent of the malevolent).
The area where I see this sort of stuff working best is in large organisations, such as civil services, where the organisations have control over who gets promoted. I’m less optimistic this could work for the most important cases, political elections, where there is not a system that can enforce the use of such measures. But it’s not clear to me how much of an innovation malevolence tests are over the normal feedback processes used in large organisations. Even if they could be introduced in politics somehow, it’s unclear how much of an innovation this would be: the public already try to assess politicians for these negative traits.
It might be worth adding that the reason the Myers-Brigg style personality tests are, so I hear, more popular in large organisations than the (more predictive) “Big 5” personality test is that Myers-Briggs has no ostensibly negative dimensions. If you pass round a Big-5 test, people might score highly on neuroticism or low on openness and get annoyed. If this is the case, which seems likely, I find it hard e.g. Google will insist that staff take a test they know will assess them on their malevolence!
As a test for the plausibility of introducing and using malevolence tests, notice that we could already test for psychopathy but we don’t. That suggests there are strong barriers to overcome.
It seems you think that one of the essential things is developing and using manipulation-proof measures of malevolence. If you were very confident we couldn’t do this, how much of an issue would that be?
I wouldn’t say it’s “essential”—influencing genetic enhancement would still be feasible—though it would certainly be a big problem.
Regarding objective measures, there will be ‘Minority Report’ style objections to actually using them in advance, even if they have high predictive power
Yes, that would be an issue.
The area where I see this sort of stuff working best is in large organisations, such as civil services, where the organisations have control over who gets promoted. I’m less optimistic this could work for the most important cases, political elections, where there is not a system that can enforce the use of such measures.
That seems true though it seems at least conceivable that voters will demand such measures in the future. (As an aside, you mention large organisations but it seems such measures could also be valuable when used in smaller (non-profit) organizations?)
But it’s not clear to me how much of an innovation malevolence tests are over the normal feedback processes used in large organisations.
Yeah, true. I guess it’s also a matter of how much (negative) weight you put on malevolent traits, how much of an effort you make to detect them, and how attentive you are to potential signs of malevolence—most people seem to overestimate their ability to detect (strategic) malevolence (at least I did so before reality taught me a lesson).
It might be worth adding that the reason the Myers-Brigg style personality tests are, so I hear, more popular in large organisations than the (more predictive) “Big 5” personality test is that Myers-Briggs has no ostensibly negative dimensions.
Interesting, that seems plausible! I’ve always been somewhat bewildered by its popularity.
If this is the case, which seems likely, I find it hard e.g. Google will insist that staff take a test they know will assess them on their malevolence!
True. I guess measures of malevolence would work best as part of the hiring process (i.e., before one has formed close relationships).
As a test for the plausibility of introducing and using malevolence tests, notice that we could already test for psychopathy but we don’t. That suggests there are strong barriers to overcome.
I agree that there are probably substantial barriers to be overcome. On the other hand, it seems that many companies are using “integrity tests” which go in a similar direction. According to Sacket and Harris (1984), at least 5,000 companies used “honesty tests” in 1984. Companies were also often using polygraph examinations—in 1985, for example, about 1.7 million such tests were administered to (prospective) employees (Dalton & Metzger, 1993, p. 149)—until they became illegal in 1988. And this even though polygraph tests and integrity tests (as well as psychopathy tests) can be gamed (rather easily).
I could thus imagine that at least some companies and organizations would start using manipulation-proof measures of malevolence (which is somewhat similar to the inverse of integrity) if it was common knowledge that such tests actually had high predictive validity and could not be gamed.
Regarding objective measures, there will be ‘Minority Report’ style objections to actually using them in advance, even if they have high predictive power (which might be tricky as it relies on collecting good data, which seems to require the consent of the malevolent).
You could imagine scenarios where we apply objective measures to children, embryos, and/or couples who are considering having children.
This would avoid some of the free-choice problem around who takes the test, though it doesn’t really get around the Minority Report problem. Also there’d still be selection at the parent level (some would-be parents would decide to not get tested).
Interesting that we don’t do anything like this for psychopathy currently, as far as I know. (Psychopathy appears to be somewhat genetic.)
Thanks for this write-up, I thought it was really interesting and not something I’d ever considered—kudos!
I’ll now hone in on the bit of this I think needs most attention. :)
It seems you think that one of the essential things is developing and using manipulation-proof measures of malevolence. If you were very confident we couldn’t do this, how much of an issue would that be? I raise this because it’s not clear to me how such measures could be created or deployed. It seems you have (1) self-reports, (2) other-reports, (3) objective metrics, e.g. brain scans. If I were really sneaky, I would just lie or not take the test. If I were really sneaky, I would be able to con others, at least for a long-time—perhaps until I was in power. Regarding objective measures, there will be ‘Minority Report’ style objections to actually using them in advance, even if they have high predictive power (which might be tricky as it relies on collecting good data, which seems to require the consent of the malevolent).
The area where I see this sort of stuff working best is in large organisations, such as civil services, where the organisations have control over who gets promoted. I’m less optimistic this could work for the most important cases, political elections, where there is not a system that can enforce the use of such measures. But it’s not clear to me how much of an innovation malevolence tests are over the normal feedback processes used in large organisations. Even if they could be introduced in politics somehow, it’s unclear how much of an innovation this would be: the public already try to assess politicians for these negative traits.
It might be worth adding that the reason the Myers-Brigg style personality tests are, so I hear, more popular in large organisations than the (more predictive) “Big 5” personality test is that Myers-Briggs has no ostensibly negative dimensions. If you pass round a Big-5 test, people might score highly on neuroticism or low on openness and get annoyed. If this is the case, which seems likely, I find it hard e.g. Google will insist that staff take a test they know will assess them on their malevolence!
As a test for the plausibility of introducing and using malevolence tests, notice that we could already test for psychopathy but we don’t. That suggests there are strong barriers to overcome.
Thank you, great points.
I wouldn’t say it’s “essential”—influencing genetic enhancement would still be feasible—though it would certainly be a big problem.
Yes, that would be an issue.
That seems true though it seems at least conceivable that voters will demand such measures in the future. (As an aside, you mention large organisations but it seems such measures could also be valuable when used in smaller (non-profit) organizations?)
Yeah, true. I guess it’s also a matter of how much (negative) weight you put on malevolent traits, how much of an effort you make to detect them, and how attentive you are to potential signs of malevolence—most people seem to overestimate their ability to detect (strategic) malevolence (at least I did so before reality taught me a lesson).
Interesting, that seems plausible! I’ve always been somewhat bewildered by its popularity.
True. I guess measures of malevolence would work best as part of the hiring process (i.e., before one has formed close relationships).
I agree that there are probably substantial barriers to be overcome. On the other hand, it seems that many companies are using “integrity tests” which go in a similar direction. According to Sacket and Harris (1984), at least 5,000 companies used “honesty tests” in 1984. Companies were also often using polygraph examinations—in 1985, for example, about 1.7 million such tests were administered to (prospective) employees (Dalton & Metzger, 1993, p. 149)—until they became illegal in 1988. And this even though polygraph tests and integrity tests (as well as psychopathy tests) can be gamed (rather easily).
I could thus imagine that at least some companies and organizations would start using manipulation-proof measures of malevolence (which is somewhat similar to the inverse of integrity) if it was common knowledge that such tests actually had high predictive validity and could not be gamed.
You could imagine scenarios where we apply objective measures to children, embryos, and/or couples who are considering having children.
This would avoid some of the free-choice problem around who takes the test, though it doesn’t really get around the Minority Report problem. Also there’d still be selection at the parent level (some would-be parents would decide to not get tested).
Interesting that we don’t do anything like this for psychopathy currently, as far as I know. (Psychopathy appears to be somewhat genetic.)