1) Research suggestion: From afar, malevolence-detection techniques seem like a better version of the already-existing tool of top-secret security clearance (or tests similar to it). I am not confident about this, but it already seems that if top-secret security clearance was a requirement for holding important posts, a lot of grief would be avoided (at least where I am from). Yet we generally do not use this tool. Why is this? I suspect that whatever the answer is, it will apply to malevolence-detection techniques as well.
2) Potential bottleneck: Suppose you succeed and develop 100% accurate malevolence-detection technique. I think that, by default, you would have trouble convincing people to use it. (“I mean, what if I score high on it? You know, I am keeping my dark side in check and I don’t plan to become too influential either, so my malevolence doesn’t really hurt anybody. But the other people don’t know that! If I get branded as malevolent, nobody will talk to me ever, or hire me, or anything!”) I conjecture that the impact of this agenda will be bottlenecked on figuring out how to leave the malevolent people a line of retreat; making sure that if you score high on this, the implications aren’t that bad. I see three reasons for this:
a) non-malevolent people might not know they are non-malevolent, and hence be afraid of this,
b) malevolent-and-know-it people might have enough power to hinder this,
c) reasonable general concerns about any test like this getting out of hand.
3) Relatedly to (2), would it make sense to consider some alternative branding that more accurately suggests what you intend to do with the concept (and doesn’t suggest other things)? Unwieldly suggestion, to illustrate what I mean: Being publicly known as “potentially too risky to be in a position of great power” indicates that you shouldn’t be a president, but you might still have friends, a spouse, and a prestigeous job. Being publicly known as “malevolent”, however, … . (Also, it seems plausible that there are people who are malevolent, but do not endorse being so, similarly to how, I think, there are paedophiles who wish they weren’t so.)
(Also, it might not be obvious from my nitpicking, but I really like the post, thanks for it :-).)
(Also, it might not be obvious from my nitpicking, but I really like the post, thanks for it :-).)
Thank you. :) No worries, I didn’t think you were nitpicking. I agree with many of your points.
[...] if top-secret security clearance was a requirement for holding important posts, a lot of grief would be avoided (at least where I am from). Yet we generally do not use this tool. Why is this? I suspect that whatever the answer is, it will apply to malevolence-detection techniques as well.
One worry with security clearances is that they tend to mostly screen for impulsive behaviors such as crime and drug use (at least, according to my limited understanding of how these security clearances work) and would thus often fail to detect more strategic malevolent individuals.
Also, your claim that “we generally do not use this tool [i.e., security clearances]” feels too strong. For example, 5.1 million Americans seem to have a security clearance. Sounds like a lot to me. (Maybe you had a different country in mind.)
I conjecture that the impact of this agenda will be bottlenecked on figuring out how to leave the malevolent people a line of retreat; making sure that if you score high on this, the implications aren’t that bad.
Good point. I guess we weren’t sufficiently clear in the post about how we envision the usage of manipulation-proof measures of malevolence. My view is that their results should, as a general rule, not be made public and that individuals who are diagnosed as malevolent should not be publicly branded as such. (Similarly, my understanding is that if someone doesn’t get a top level security clearance because they, for instance, have a serious psychiatric disorder, they only don’t get the job requiring the security clearance—it’s not like the government makes their mental health problems public knowledge.)
My sense is that malevolent individuals should only be prevented from reaching highly influential positions like becoming senators, members of Congress, majors, CEOs of leading AGI companies, et cetera. In other words, the great majority of jobs would still be open to them.
Of course, my views on this issue are by no means set in stone and still evolving. I’m happy to elaborate on my reasons for preferring this more modest usage if you are interested.
Of course, my views on this issue are by no means set in stone and still evolving. I’m happy to elaborate on my reasons for preferring this more modest usage if you are interested.
I think the more modest usage is reasonable choice.
Maybe you had a different country in mind. [regarding top-secret security clearance]
I am Czech. We do have the institute, and use it. But, as far as I know, our president doesn’t have it, and a bunch of other people don’t have it. (I.e., it seems that people who need secret information on a daily basis have it. But you don’t need it for many other positions from which you could put pressure on people who have the clearance.)
Some thoughts that occured to me while reading:
1) Research suggestion: From afar, malevolence-detection techniques seem like a better version of the already-existing tool of top-secret security clearance (or tests similar to it). I am not confident about this, but it already seems that if top-secret security clearance was a requirement for holding important posts, a lot of grief would be avoided (at least where I am from). Yet we generally do not use this tool. Why is this? I suspect that whatever the answer is, it will apply to malevolence-detection techniques as well.
2) Potential bottleneck: Suppose you succeed and develop 100% accurate malevolence-detection technique. I think that, by default, you would have trouble convincing people to use it. (“I mean, what if I score high on it? You know, I am keeping my dark side in check and I don’t plan to become too influential either, so my malevolence doesn’t really hurt anybody. But the other people don’t know that! If I get branded as malevolent, nobody will talk to me ever, or hire me, or anything!”) I conjecture that the impact of this agenda will be bottlenecked on figuring out how to leave the malevolent people a line of retreat; making sure that if you score high on this, the implications aren’t that bad. I see three reasons for this:
a) non-malevolent people might not know they are non-malevolent, and hence be afraid of this,
b) malevolent-and-know-it people might have enough power to hinder this,
c) reasonable general concerns about any test like this getting out of hand.
3) Relatedly to (2), would it make sense to consider some alternative branding that more accurately suggests what you intend to do with the concept (and doesn’t suggest other things)? Unwieldly suggestion, to illustrate what I mean: Being publicly known as “potentially too risky to be in a position of great power” indicates that you shouldn’t be a president, but you might still have friends, a spouse, and a prestigeous job. Being publicly known as “malevolent”, however, … . (Also, it seems plausible that there are people who are malevolent, but do not endorse being so, similarly to how, I think, there are paedophiles who wish they weren’t so.)
(Also, it might not be obvious from my nitpicking, but I really like the post, thanks for it :-).)
Thank you. :) No worries, I didn’t think you were nitpicking. I agree with many of your points.
One worry with security clearances is that they tend to mostly screen for impulsive behaviors such as crime and drug use (at least, according to my limited understanding of how these security clearances work) and would thus often fail to detect more strategic malevolent individuals.
Also, your claim that “we generally do not use this tool [i.e., security clearances]” feels too strong. For example, 5.1 million Americans seem to have a security clearance. Sounds like a lot to me. (Maybe you had a different country in mind.)
Good point. I guess we weren’t sufficiently clear in the post about how we envision the usage of manipulation-proof measures of malevolence. My view is that their results should, as a general rule, not be made public and that individuals who are diagnosed as malevolent should not be publicly branded as such. (Similarly, my understanding is that if someone doesn’t get a top level security clearance because they, for instance, have a serious psychiatric disorder, they only don’t get the job requiring the security clearance—it’s not like the government makes their mental health problems public knowledge.)
My sense is that malevolent individuals should only be prevented from reaching highly influential positions like becoming senators, members of Congress, majors, CEOs of leading AGI companies, et cetera. In other words, the great majority of jobs would still be open to them.
Of course, my views on this issue are by no means set in stone and still evolving. I’m happy to elaborate on my reasons for preferring this more modest usage if you are interested.
I think the more modest usage is reasonable choice.
I am Czech. We do have the institute, and use it. But, as far as I know, our president doesn’t have it, and a bunch of other people don’t have it. (I.e., it seems that people who need secret information on a daily basis have it. But you don’t need it for many other positions from which you could put pressure on people who have the clearance.)