brook
I think this kind of discussion is important, and I don’t want to discourage it, but I do think these discussions are more productive when they’re had in a calm manner. I appreciate this can be difficult with emotive topics, but it can be hard to change somebody’s mind if they could interpret your tone as attacking them.
In summary: I think it would be more productive if the discussion could be less hostile going forwards.
We’ve banned the user denyeverywhere for a month for promoting violence on the forum.
As a reminder, bans affect the user, not the account(s).
If anyone has questions or concerns, please feel free to reach out, and if you think we made a mistake here, you can appeal the decision
Speaking as a moderator, this comment seems to break a number of our norms. It isn’t on-topic, and it’s an unnecessary personal attack. I’d like to see better on the forum.
I think this comment is flamebait, and broke Forum norms. Examples below:
“I bet somewhere there’s a small group of rich elites who actually bet on the gang fights in Haiti, have their own private app for it, and I bet some of them are also in an EA circle”
“forget I mentioned the word ‘neocolonialism’ because you’ll be just like every other woke white person here and take offense that I said something true, you can go spend more time debating gender.”
I’d like if the discussion could be more civil going forwards.
Hi! Sorry for the delay in replying—we’ve now posted the metrics, if you’re interested.
Epistemic status: just a 5-minute collation of some useful sources, with a little explanatory text off the top of my head.
Stampy’s answers to “Why is AI dangerous?”and “Why might we expect a superintelligence to be hostile by default?” seem pretty good to me.
To elaborate a little:
Alignment seems hard. Humans value very complex things, which it seems both A) difficult to tell an AI to preserve and B) seem unlikely for AI to preserve by default.
A number of things seem to follow pretty directly from the idea of ‘creating an agent which is much more intelligent than humans’:
Non-human goals: we have a strong prior that its goals will not line up with human goals (See orthogonality thesis)
Optimising is Destructive: optimising for one value system will by default destroy value according to other value systems (see: instrumental convergence)
Intelligence is Dangerous: as it’s much smarter than humans, predicting its behaviour will be very difficult, as will containing or controlling it. (See AI boxing)
When you combine these things, you get an expectation that the default outcome of unaligned AGI is very bad for humans—and an idea of why AI alignment may be difficult.
To take a different approach:
Humans have a pretty bad track record of not using massively destructive technology. It seems at least plausible that COVID-19 was a lab leak (and its plausibility is enough for this argument). The other key example to me is the nuclear bomb.
What’s important is that both of these technologies are relatively difficult to get access to. At least right now, it’s relatively easy to get access to state-of-the-art AI.
Why is this important? It’s related to the unilateralist’s curse. If we think that AI has the potential to be very harmful (which deserves its own debate), then the more people that have access to it, the more likely that harm becomes. Given our track record with lower-access technologies, it seems likely from this frame that accelerationism will lead to non-general artificial intelligence being used to do massive harm by humans.
I would hope that good criticism of EA would “make the world better if taken seriously” by improving the EA ecosystem. That said, I do understand your concern—I hope people will submit good criticism to the journal, and that it will be published!
This is a really great point! Thank you for raising it. I’ll see about adding it to future posts.
Thank you for pointing that out! Worth noting that’s a limit on the videos you can have stored on their servers at once; if you want to download & delete them from the servers you can record as many as you like.
These look great, thanks for suggesting them! Would you be interested in writing tutorials for some/all of them that I could add to the sequence? If not, I think updating the topic page with links to tutorials you think are good would also be great!
The tool is here, there’ll also be a post in a few hours but it’s pretty self-explanatory
Any feedback you have as we go would be much appreciated! I’ve focussed on broadening use, so I’m hoping a good chunk of the value will be in new ways to use the tools as much as anything else—if you have any ways you think are missing they would also be great!
Thanks for making this! I also feel like I get a lot of value out of quarterly/yearly reviews, and this looks like a nice prompting tool. If you haven’t seen it already, you might like to look at Pete Slattery’s year-review question list too!
I think this is one reasonable avenue to explore alignment, but I don’t want everybody doing it.
My impression is that AI researchers exist on a spectrum from only doing empirical work (of the kind you describe) to only doing theoretical work (like Agent Foundations), and most fall in the middle, doing some theory to figure out what kind of experiment to run, and using empirical data to improve their theories (a lot of science looks like this!).
I think all (or even a majority of) AI safety researchers moving to doing empirical work on current AI systems is unwise, for two reasons:
Bigger models have bigger problems.
Lessons learned from current misalignment may be necessary for aligning future models, but will certainly not be sufficient. For instance, GPT-3 will (we assume) never demonstrate deceptive alignment, because its model of the world is not broad enough to do so, but more complex AIs may do.
This is particularly worrying because we may only get one shot at spotting deceptive alignment! Thinking about problems in this class before we have direct access to models that could, even in theory, exhibit these problems seems both mandatory and a key reason alignment seems hard to me.
AI researchers are sub-specialised.
Many current researchers working in non-technical alignment, while they presumably have a decent technical background, are not cutting-edge ML engineers. There’s not a 1:1 skill translation from ‘current alignment researcher’ to ‘GPT-3 alignment researcher’
There is maybe some claim here that you could save money on current alignment researchers and fund a whole bunch of GPT alignment researchers, but I expect the exchange rate is pretty poor, or it’s just not possible in the medium term to find sufficient people with a deep understanding of both ML and alignment.
The first one is the biggy. I can imagine this approach working (perhaps inefficiently) in a world were (1) were false and (2) were true, but I can’t imagine this approach working in any worlds where (1) holds.
I agree that publishing results of the form “it turns out that X can be done, though we won’t say how we did it” is clearly better than publishing your full results, but I think it’s much more harmful than publishing nothing in a world where other people are still doing capabilities research.
This is because it seems to me that knowing something is possible is often a first step to understanding how. This is especially true if you have any understanding of where this researcher or organisation were looking before publishing this result.
I also think there are worlds where it’s importantly harmful to too openly critique capabilities research, but I lean towards not thinking we are in this world, and think the tone of this post is a pretty good model for how this should look going forwards. +1!
This is a good point, and I think most EV in general is in X-risk. I’d include COVID-level or HIV-level emerging pandemics as being worth thinking about even if they don’t represent X-risk, though.
It’s not obvious to me that generalised solutions (for both natural & man-made andemics) are the most efficient answer. For instance as a random and un-researched example, it could be really cheap to encourage farmers to wear gloves or surgical masks when handling certain animals (or in certain regions), but this is only worth doing if we’re worried about farm animal pandemics.
Hi everybody! I’m Victoria, I’m currently based in Edinburgh and I heard about EA through LessWrong. I’ve been involved with the local EA group for almost a year now, and with rationalism for a few years longer than that. I’m only now getting around to being active on the forum here.
I was a medical student, but I’m taking a year out and seriously considering moving into either direct existential risk research/policy or something like operations/‘interpreting’ research. When I’ve had opportunities to do things like that I’ve really enjoyed it. I’ve also previously freelanced with Nonlinear and CEA for research and writing gigs.
Long-term I could see myself getting into AI, possibly something like helping build infrastructure for AI researchers to better communicate, or direct AI work (with my neuroscience degree).
See youse all around!
I think something like “only a minority of people [specific researchers, billionaires, etc.] are highly influential, so we should spend a lot of energy influencing them” is a reasonable claim that implies we maybe shouldn’t spend as much energy empowering everyday people. But I haven’t seen any strong evidence either way about how easy it is to (say) convert 1,000 non-billionaires to donate as much as one billionaire.
I do think the above view has some optics problems, and that many people who ‘aren’t highly influential’ obviously could become so if they e.g. changed careers.
As somebody strongly convinced by longtermist arguments, I do find it hard to ‘onboard’ new EAs without somebody asking “do you really think most people will sit and have a protracted philosophical discussion about longtermism?” at some point. I think there are two reasonable approaches to this:
If you start small (suggest donating to the AMF instead of some other charity, and maybe coming to some EA meetings), some people will become more invested and research longtermism on their own who would have otherwise been put off.
It’s useful to have two different pitches for EA for different audiences; discuss longtermism with people who are in philosophy or related fields, and something easier to explain the rest of the time. My impression is this is your pitch in this post?
I’m not currently convinced of either view, but would be interested to hear about other peoples’ experiences.
TLDR: Diverse EA contractor looking for a part-time operations or comms role, remote.
Skills & background: I currently run a local EA group with a grant from OpenPhil as the only paid member. This role includes tasks like organising coworking space and checking our compliance with local charity regulations as well as running events and getting feedback from our members.
In the past, I’ve done short-term projects or contractor work for EA organisations. One project I’m particularly proud of was the Tools For Collaborative Truth-Seeking sequence, on which we got feedback that a number of the tools were now being used as part of the regular workflow in some big EA organisations.
Finally, I also work currently as a moderator for the forum, which involves communicating carefully around sensitive topics on a regular basis.
Location/remote: I’m based in Edinburgh, Scotland, and a remote role would be ideal.
Availability & type of work: I’m in the GMT timezone, but am happy to work outside of these hours (though regularly working in e.g. PDT would be a challenge). My ideal role would be around 20h/week, but I’m open to roles with more or fewer hours available.
Resume/CV/LinkedIn: LinkedIn
Email/contact: DM me on the forum.
Other notes: I’m open to any operations or comms/outreach role, but have a preference for AI safety or pandemic prevention organisations.