And yet, I think that very little AI safety work focuses on affecting P(things go really well | no AI takeover). Probably Forethought is doing the best work in this space.
Do you think this sort of work is related to AI safety? It seems to me that it’s more about philosophy (etc.) so I’m wondering what you had in mind.
Yup! Copying over from a LessWrong comment I made:
Roughly speaking, I’m interested in interventions that cause the people making the most important decisions about how advanced AI is used once it’s built to be smart, sane, and selfless. (Huh, that was some convenient alliteration.)
Smart: you need to be able to make really important judgment calls quickly. There will be a bunch of actors lobbying for all sorts of things, and you need to be smart enough to figure out what’s most important.
Sane: smart is not enough. For example, I wouldn’t trust Elon Musk with these decisions, because I think that he’d make rash decisions even though he’s smart, and even if he had humanity’s best interests at heart.
Selfless: even a smart and sane actor could curtail the future if they were selfish and opted to e.g. become world dictator.
And so I’m pretty keen on interventions that make it more likely that smart, sane, and selfless people are in a position to make the most important decisions. This includes things like:
Doing research to figure out the best way to govern advanced AI once it’s developed, and then disseminating those ideas.
Helping to positively shape internal governance at the big AI companies (I don’t have concrete suggestions in this bucket, but like, whatever led to Anthropic having a Long Term Benefit Trust, and whatever could have led to OpenAI’s non-profit board having actual power to fire the CEO).
Helping to staff governments with competent people.
Helping elect smart, sane, and selfless people to elected positions in governments (see 1, 2).
Hmm, I think if we are in a world where the people in charge of the company that have already built ASI need to be smart/sane/selfless for things to go well, then we’re already in a much worse situation than we should be, and things should have been done differently prior to this point.
I realize this is not a super coherent statement but I thought about it for a bit and I’m not sure how to express my thoughts more coherently so I’m just posting this comment as-is.
Do you think this sort of work is related to AI safety? It seems to me that it’s more about philosophy (etc.) so I’m wondering what you had in mind.
Yup! Copying over from a LessWrong comment I made:
Roughly speaking, I’m interested in interventions that cause the people making the most important decisions about how advanced AI is used once it’s built to be smart, sane, and selfless. (Huh, that was some convenient alliteration.)
Smart: you need to be able to make really important judgment calls quickly. There will be a bunch of actors lobbying for all sorts of things, and you need to be smart enough to figure out what’s most important.
Sane: smart is not enough. For example, I wouldn’t trust Elon Musk with these decisions, because I think that he’d make rash decisions even though he’s smart, and even if he had humanity’s best interests at heart.
Selfless: even a smart and sane actor could curtail the future if they were selfish and opted to e.g. become world dictator.
And so I’m pretty keen on interventions that make it more likely that smart, sane, and selfless people are in a position to make the most important decisions. This includes things like:
Doing research to figure out the best way to govern advanced AI once it’s developed, and then disseminating those ideas.
Helping to positively shape internal governance at the big AI companies (I don’t have concrete suggestions in this bucket, but like, whatever led to Anthropic having a Long Term Benefit Trust, and whatever could have led to OpenAI’s non-profit board having actual power to fire the CEO).
Helping to staff governments with competent people.
Helping elect smart, sane, and selfless people to elected positions in governments (see 1, 2).
Hmm, I think if we are in a world where the people in charge of the company that have already built ASI need to be smart/sane/selfless for things to go well, then we’re already in a much worse situation than we should be, and things should have been done differently prior to this point.
I realize this is not a super coherent statement but I thought about it for a bit and I’m not sure how to express my thoughts more coherently so I’m just posting this comment as-is.