Creating superintelligent artificial agents without a worldwide referendum is ethically unjustifiable. Until a consensus is reached on whether to bring into existence such technology, a global moratorium is required (n.b. we already have AGI).
yanni kyriacos
Something bouncing around my head recently … I think I agree with the notion that “you can’t solve a problem at the level it was created”.
A key point here is the difference between “solving” a problem and “minimising its harm”.
Solving a problem = engaging with a problem by going up a level from which is was createwd
Minimising its harm = trying to solve it at the level it was created
Why is this important? Because I think EA and AI Safety have historically focussed (and has their respective strengths in) harm-minimisation.
This applies obviously the micro. Here are some bad examples:
Problem: I’m experiencing intrusive + negative thoughts
Minimising its harm: engage with the thought using CBT
Attempting to solve it by going meta: apply meta cognitive therapy, see thoughts as empty of intrinsic value, as farts in the wind
Problem: I’m having fights with my partner about doing the dishes
Minimising its harm: create a spreadsheet and write down every thing each of us does around the house and calculate time spent
Attempting to solve it by going meta: discuss our communication styles and emotional states when frustration arises
But I also think this applies at the macro:
Problem: People love eating meat
Minimising harm by acting at the level the problem was created: asking them not to eat meat
Attempting to solve by going meta: replacing the meat with lab grown meat
Problem: Unaligned AI might kill us
Minimising harm by acting at the level the problem was created: understand the AI through mechanistic interpretability
Attempting to solve by going meta: probably just Governance
as of comment, 6 agrees and 6 disagrees. perfect :)
I think it is good to have some ratio of upvoted/agreed : downvotes/disagreed posts in your portfolio. I think if all of your posts are upvoted/high agreeance then you’re either playing it too safe or you’ve eaten the culture without chewing first.
Media is often bought on a CPM basis (cost per thousand views). A display ad on LinkedIn for e.g. might cost $30 CPM. So yeah I think merch is probably underrated.
Thanks for pointing this out :)
I think longview philanthropy might look after HNW individuals in the EA space?
Hi John—can you email me? yanni@aisafetyanz.com.au
Basically, I think there is a good chance we have 15% unemployment rates in less than two years caused primarily by digital agents.
Totally different. I had a call with a voice actor who has colleagues hearing their voices online without remuneration. Tip of the iceberg stuff.
Yeah the problem with some surveys is they measure prompted attitudes rather than salient ones.
I think the amount of AI caused turmoil I’m expecting in 3-5 years might be much higher than you, which accounts for most of this difference.
Hear hear to asking the obvious questions! Still so much low hanging fruit in longtermism / x-risk.
Thanks for writing this :)
I haven’t had a chance to read it all yet, so there is a chance this point is covered in the post, but I think more EA influence vs non-EA influence (let’s call them “normies”) within AI Safety could actually be bad.
For example, a year ago the majority of normies would have seen Lab jobs being promoted in EA spaces and gone “nah that’s off”, but most of EA needed more nuance and time to think about it.
This is a combo of style of thinking / level of conscientiousness that I think we need less of and leads me to think my initial point (there are other dimensions around which kind of normies I think we should target within AI Safety but that’s it’s own thing altogether).
I’m pretty confident that a majority of the population will soon have very negative attitudes towards big AI labs. I’m extremely unsure about what impact this will have on the AI Safety and EA communities (because we work with those labs in all sorts of ways). I think this could increase the likelihood of “Ethics” advocates becoming much more popular, but I don’t know if this necessarily increases catastrophic or existential risks.
Not sure if this type of concern has reached the meta yet, but if someone approached me asking for career advice, tossing up whether to apply for a job at a big AI lab, I would let them know that it could negatively affect their career prospects down the track because so many people now perceive such as a move as either morally wrong or just plain wrong-headed. And those perceptions might only increase over time. I am not making a claim here beyond this should be a career advice consideration.
Hey mate! Lovely to hear from you :)
Yeah I just think that most EAs assume that the message does most of the work in marketing when it is actually the medium: https://en.wikipedia.org/wiki/The_medium_is_the_message
I think this is a fair assumption to make if you believe people make decisions extremely rationally.
I basically don’t (I.e. the 80k brand is powering the OpenAI brand through a Halo Effect).
Unfortunately this is really hard to avoid!
Thanks for writing this up!
FWIW, even if AGI arrives ~ 2050 I still think it* would be the thing I’d want to work on right now. I would need to be really confident it wasn’t arriving before then for me not to want to work on it.
*AI Safety.
More thoughts on “80,000 hours should remove OpenAI from the Job Board”
- - - -
A broadly robust heuristic I walk around with in my head is “the more targeted the strategy, the more likely it will miss the target”.
Its origin is from when I worked in advertising agencies, where TV ads for car brands all looked the same because they needed to reach millions of people, while email campaigns were highly personalised.The thing is, if you get hit with a tv ad for a car, worst case scenario it can’t have a strongly negative effect because of its generic content (dude looks cool while driving his 4WD through rocky terrain). But when an email that attempts to be personalised misses the mark, you feel it (Hi {First Name : Last Name}!).
Anyway, effective altruism loves targeting. It is virtually built from the ground up on it (RCTs and all that, aversion to PR etc).
Anyway, I am thinking about this in relation to the recent “80,000 hours should remove OpenAI from the Job Board” post.
80k is applying an extremely targeted strategy by recommending people work in some jobs at the workplaces creating the possible death machines (caveat 1, caveat 2, caveat 3, caveatn).
There are so many failure modes involved with those caveats that makes it virtually impossible to not go catastrophically poorly in at least a few instances and take the mental health of a bunch of young and well-meaning EAs with it.
[IMAGE] there is something about the lack overlap between these two audiences that makes me uneasy. WYD?
I think an assumption 80k makes is something like “well if our audience thinks incredibly deeply about the Safety problem and what it would be like to work at a lab and the pressures they could be under while there, then we’re no longer accountable for how this could go wrong. After all, we provided vast amounts of information on why and how people should do their own research before making such a decision”.
The problem is, that is not how most people make decisions. No matter how much rational thinking is promoted, we’re first and foremost emotional creatures that care about things like status. So, if 80k decides to have a podcast with the Superalignment team lead, then they’re effectively promoting the work of OpenAI. That will make people want to work for OpenAI. This is an inescapable part of the Halo effect.Lastly, 80k is explicitly targeting very young people who, no offense, probably don’t have the life experience to imagine themselves in a workplace where they have to resist incredible pressures to not conform, such as not sharing interpretability insights with capabilities teams.
The whole exercise smacks of nativity and I’m very confident we’ll look back and see it as an incredibly obvious mistake in hindsight.
Good point. In that case the hypothetical user isn’t using it as a forum (i.e. for discourse)