Aspiring EA-adjacent trying to make the singularity further away
Pato
Karma: 111
I recomend AISafety.com and, if you are looking for introductions in video/ audio form, I like this selection (in part because I contributed to it).
None of them are that technical though, and given that you mentioned your math knowledge, seems that’s what you’re interested on.
In that case, the thing that you have to know is that it is said that the field is “pre-paradigmatic”, so there’s not any type of consensus about how is the best way to think about the problems and therefore how would even the potential solutions would look like or come to be. But there’s work outside ML that is purely mathematical and related to agent foundations, and this introductory post that I’ve just found seems to explain all that stuff better than I could and would probably be more useful for you than the other links.
I think something like that could be great! I also wish learning about EA wouldn’t be that difficult or time consuming. About your questions:
For the people who don’t want to change careers because of EA, I think you can still share with them a list of ways they still can have a positive impact in the world, something like what PauseAI has. That shouldn’t have a big cost and it would possible to expand about it later.
Adding to what you wrote, other ways they can still have an impact are:
Changing their habits of consumption, like becoming vegan and reducing their pollution
Participating in the community, like in the forums or volunteering
Sharing the ideas and content, like the video/ course itself
Small everyday actions caused by learning more about EA, like expanding the moral circle, the causes and having a more rationalist mindset
Stop working or investing on risky things, like AI capabilities
I don’t think so.
Sadly I couldn’t respond to this post two weeks ago but here I go.
First of all I’m not sure I understand your position, but I think that you believe that if we push for other types of regulation either:
that would be enough to be safe from dangerous AI or
we’ll be able to slow down AI development enough to develop measures to be safe from dangerous AI
I’m confused between the two because you write
Advanced AI systems pose grave threats and we don’t know how to mitigate them.
That I understand as you believing we don’t have know those measures right now, but you also write
If a company developing an unprecedentedly large AI model with surprising capabilities can’t prove it’s safe, they shouldn’t release it.
That if we agree there’s no way to prove it then you’re pretty much talking about a pause.
If your point is the first one I would disagree with it and I think even OpenAI when they say we don’t know yet how to align a SI.
If your point is the second one, then my problem with that is that I don’t think that would give us close to the same amount of time than a pause. Also it could make most people believe risks from AI, including X-risks, are safeguarded now, and we could lose support because of that. And all of that would lead to more money in the industry that could lead to regulatory capture recursively.
All of that is also related to
it’s closer to what those of us concerned about AI safety ideally want: not an end to progress, but progress that is safe and advances human flourishing.
Which I’m not sure that it’s true. Of course this depends a lot in how much you think that the current work on alignment is close to being enough to make us safe. Are we going parallel enough to the precipice that we’ll be able to reach or steer in time to reach an utopia? Or are we going towards it and we will have some brief progress before falling? Would that be closer to the ideal? Anyways, the ideal is the enemy of the good or the truth or something.
Lastly, after arguing why a pause would be a lot better than other regulations, I’ll give you that of course it would be harder to get/ less “politically palatable” which is arguably the main point of the post. But I don’t how many orders of magnitude. With a pause you win over people who think safer AI isn’t enough or it’s just marketing from the biggest companies and nations. And also, talking about marketing, I think pausing AI is a slogan that can draw a lot more attention, which I think is good given that most people seem to want regulation.
it appears many EAs believe we should allow capabilities development to continue despite the current X-risks.
Where do you get this from?
Also, this:
have < ~ 0.1% chance of X-risk.
means p(doom) <~ 0.001
I think that by far the less intuitive thing about AI X-risk is why AIs would want to kill us instead of doing what they would be “programmed” to do.
I would give more importance to that part of the argument than the “intelligence is really powerful” part.
I think the bigger question is why haven’t we found other species elsewhere in the universe.
Then I see the question about whether they’ll kill us or not as a different one.
I really liked the axis that you presented and the comparision between a version of the community that is more cause oriented vs member oriented.
The only caveat that I have is that I don’t think we can define a neutral point in between them that allows you to classify communities as one type or the other.
Luckily, I think that is unnecesary because even though the objective of EA is to have the best impact in the world and not the greatest number of members, I think we all think the best decision is to have a good balance between cause and member oriented. So the question that we should ask is should EA be MORE big tent, weird, or do we have a good balance right now?
And to achieve that balance we can be more big tent in some aspects, moments and orgs and weirder in others.
I strongly agree with you and, as someone who thinks alignment is extremely hard (not just because of the technical side of the problem, but due the human values side too), I believe that a hard pause and studying how to augment human intelligence is actually the best strategy.
Got it, thanks.
Wait, I don’t understand. Are 63.6% or 76.6% of respondents left-leaning? And 69.58% or 79.81% non-religious?
Have important “crying wolf” cases have happen in real life? About societal issues? I mean, yeah, it is a possibility but the alternatives seem so much worse.
How do we know when we are close enough to the precipice for other people to be able to see it and to ask to stop the race to it? General audiences lately have been talking about how surprised they are about AI so it seems like perfect timing for me.
Also, if people get used to benefit and work in narrow and safe AIs they could put themselves against stopping/ slowing them down.
Even if more people could agree on decelerate in the future it would take more time to stop/ going really slow with more stakeholders going at a higher speed. And of course, after that we would be closer to the precipice that if we started the deceleration before.
I doubt this:
the AGI doom memeplex has, to some extent, a symbiotic relationship with the race toward AGI memeplex
I mean, if you say it could increase the amount of people working in capabilities at first I would agree, but it probably increases a lot more the amount of people working on safety and wanting to slow/ stop capabilities research, which could create legislation and at the end of the day increase time until AGI.
In respect of the other cons of the doom memeplex I kinda agree to a certain extent but I don’t see them come even close to the pros of potentially having lots of people actually taking the problem very seriously.
Why is it considered bad timing?
What if we fundamentally value other things but instrumentally that translates to power?
The challenge isn’t figuring out some complicated, nuanced utility function that “represents human values”; the challenge is getting AIs to do what it says on the tin—to reliably do whatever a human operator tells them to do.
Why do you think this? I infer for what I’ve seen written in other posts and comments that this is a common belief but I don’t find the reasons why.
The fact that there are specific really difficult problems with aligning ML systems doesn’t mean that the original really difficult problem with finding and specifying the objectives that we want for a superintelligence were solved.
I hate it because it makes it seems like alignment is a technical problem that can be solved by a single team and as you put it in your other post we should just race and win against the bad guys.
I could try to envision what type of AI you are thinking of and how would you use it, but I would prefer if you tell me. So, what would you ask your aligned AGI to do and how would it interpret that? And how are you so sure that most alignment researchers would ask it the same things as you?
Cool! I just hope that in the future you crosspost all the posts here. I like the forum and don’t like to have to check different external blogs.
I agree with some points of the post but don’t like at all how it defines the things that cost you the less as “True values” and are recommending people to follow them.
As an agnostic on what is the best way of defining values and how good is for people to do the things they are the most motivated to do, I want to remind people that you could define what your “True values” are as the things you do even tho they cost you more energy; and that doing them could be better.
So I think we agree that it seems good to find the right balance between following costly and not costly values, but calling some of them the True ones seems to imply that you should focus on or do only them.
easily best intro to agi safety
Meat with that label comes from farms that have stricter regulations for mutilation (dehorning, castration, debeaking, tail docking) and better air quality. The animals on those farms have more space and barn enrichments (e.g. toys, animal brushes, hay) and fewer diseases. Suppose meat with this animal welfare label costs 50% more than meat without an animal welfare label, and animal suffering for meat with the animal welfare label is half the amount of animal suffering for meat without a label.
Only half the suffering? With that description I was thinking on lives with a lot less suffering. And even worth living. I don’t know much about animal welfare so where that suffering is coming from?
Take a Minute by K’naan about the value of giving and epistemic humility lol