Ah shucks, I am sorry to hear it. Good luck to you!
Justis
Karma: 926
Hmm. I think reactions to that would vary really widely between researchers, and be super sensitive to when it happened, why, whether it was permanent, and other considerations.
Ah yeah, you’re right—I think basically I put in the percent rather than the probability. So it would indeed be very expensive to be competitive with AMF. Though so is everything else, so that’s not hugely surprising.
As for the numbers, yeah, it does just strike me as really, really unlikely that we can solve AI x-risk right now. 1⁄10,000 does feel about right to me. I certainly wouldn’t expect everyone else to agree though! I think some people would put the odds much higher, and others (like Tyler Cowen maybe?) would put them a bit lower. Probably the 1% step is the step I’m least confident in—wouldn’t surprise me if the (hard to find, hard to execute) solutions that are findable would reduce risk significantly more.
EDIT: tried to fix the math and switched the “relative risk reduction term” to 10%. I feel like among findable, executable interventions there’s probably a lot of variance, and it’s plausible some of the best ones do reduce risk by 10% or so. And 1/1000 feels about as plausible as 1/10000 to me. So, somewhere in there.
(Comment to flag that I looked back over this and just totally pretended 4,000 was equal to 1,000. Whoops. Don’t think it affects the argument very strongly, but I have multiplied the relevant dollar figures by 4.)
Thank you! My perspective is: “figuring out if it’s tractable is at least tractable enough that it’s worth a lot more time/attention going there than is currently”, but not necessarily “working on it is far and away the best use of time/money/attention for altruistic purposes”, and almost certainly not “working on it is the best use of time/money/attention under a wide variety of ethical frameworks and it should dominate a healthy moral parliament”.
To me, “aligned” does a lot of work here. Like yes, if it’s perfectly aligned and totally general, the benefits are mind boggling. But maybe we just get a bunch of AI that are mostly generating pretty good/safe outputs, but a few outputs here and there lower the threshold required for random small groups to wreak mass destruction, and then at least one of those groups blows up the biome.
But yeah given the premise we get AGI that mostly does what we tell it to, and we don’t immediately tell it to do anything stupid, I do think it’s very hard to predict what will happen but it’s gonna be wild (and indeed possibly really good).
Yeah, I share the view that the “Recalls” are the weakest part—I mostly was trying to get my fuzzy, accumulated-over-many-years vague sense of “whoa no we’re being way too confident about this” into a more postable form. Seeing your criticisms I think the main issue is a little bit of a Motte-and-Bailey sort of thing where I’m kind of responding to a Yudkowskian model, but smuggling in a more moderate perspective’s odds (ie. Yudkowsky thinks we need to get it right on the first try, but Grace and MacAskill may be agnostic there).
I may think more about this! I do think there’s something there sort of between the parts you’re quoting, by which I mean yes, we could get agreement to a narrower standard than solving ethics, but even just making ethical progress at all, or coming up with standards that go anywhere good/predictable politically seems hard. Like, the political dimension and the technical/problem specification dimensions both seem super hard in a way where we’d have to trust ourselves to be extremely competent across both dimensions, and our actual testable experiments against either outcome are mostly a wash (ie. we can’t get a US congressperson elected yet, or get affordable lab-grown meat on grocery store shelves, so doing harder versions of both at once seems...I dunno, might hedge my portfolio far beyond that!).
Yeah I think a lot of it is West Coast American culture! I imagine EA would have super different vibes if it were mostly centered in New York.
I think it varies with the merits of the underlying argument! But at the very least we should suppose there’s an irrational presumption toward doom: for whatever reason(s), maybe evopsych or maybe purely memetic, doomy ideas have some kind of selection advantage that’s worth offsetting.
Seconded!
Are there other such posts? I guess because I—erm—left for a while, I may have missed them!
Might be worth thinking about, sure. In my case I don’t think there’d have been much.
Maybe! I dunno. A and B have been pretty easy for me. And C, yeah! But to some degree I think that’s the point—if you’re unhappy with a life trying to make EA your social life + career + sense of meaning, try making a life where it’s none of those things (and donate far more money to charity than anyone else you know thinks is reasonable, sure), and then when you’re fully stable and secure, you can consider returning with a clear mind.
I suppose all this is quite n=1 though! It’s just worked well for me. But mileage may vary.
Some other considerations I think might be relevant:
Are there top labs/research outfits that are eager for top technical talent, and don’t care that much how up to speed that talent is on AI safety in particular? If so, seems like you could just attract eg. math Olympiad finalists or something and give them a small amount of field-specific info to get them started. But if lots of AI safety-specific onboarding is required, that’s pretty bad for movement building.
How deep is the well of untapped potential talent in various ways/various places? Seems like there’s lots and lots of outreach at top US universities right now, arguably even too much for image reasons. There’s probably not enough in eg. India or something—it might be really fruitful to make a concerted effort to find AI Safety Ramanujan. But maybe he ends up at Harvard anyway.
Looking at current top safety researchers, were they as a rule at it for several years before producing anything useful? My impression is that a lot of them came on to the field pretty strong almost right away, or after just a year or so of spinning up. It wouldn’t surprise me if many sufficiently smart people don’t need long at all. But maybe I’m wrong!
The ‘scaling up’ step interests me. How much does this happen? How big of a scale is necessary?
Retention seems maybe relevant too. Very hard to predict how many group participants will stick with the field, and for how long. Introduces a lot of risk, though maybe not relevant for timelines per se.
Yeah! This was the actually the first post I tried to write. But it petered out a few times, so I approached it from a different angle and came up with the post above instead. I definitely agree that “robustness” is something that should be seen as a pillar of EA—boringly overdetermined interventions just seem a lot more likely to survive repeated contact with reality to me, and I think as we’ve moved away from geeking out about RCTs we’ve lost some of that caution as a communtiy.
Yes, I agree it’s a confused concept. But I think that same confused concept gets smuggled into conversations about “impact” quite often.
It’s also relevant for coordination: any time you can be the 100th person that joins a group of 100 that suddenly is able to save lots of lives, there first must have been 99 people who coordinated on the bet they’d be able to get you or someone like you. But how did they make that bet?
My view is that more traditional philanthropic targets make for a much easier sell, so GiveWell style messaging is going to reach/convince way more people than longtermist or x-risk messaging.
So you’ll probably have way, way more people who are interested in EA on the global poverty and health side. I still only donate my pledge money to AMF, plus $100 a month extra to animal welfare, despite being somewhat involved in longtermist/x-risk stuff professionally (and pretty warm on these projects beyond my own involvement).
That being said, for some people EA is their primary social peer group. These people also tend to be highly ambitious. That’s a recipe for people trying really hard to figure out what’s the most prized, and orienting toward that. So there’s lots of buzz around longtermism, despite the absolute numbers in the longtermist direction (people, clicks, eyeballs, money) being lower than those for more traditional, popular interventions.
I’ve found the LessWrong editing service to be a pretty exciting way to provide copyediting, proofreading, feedback etc. to lots and lots of individuals over the last several months. Perhaps an expansion of that model could be valuable? This month there were 32 posts I did copyediting for through the service, which is more than usual but not by too much. That’s way more than I would have even actively trying to promote myself, and I haven’t had to do the promoting (or handle billing with a whole bunch of individuals). If there’s more money for centralized-funding-of-edits, I at least continue to have excess capacity there and find it a lot of fun!
Thanks! Yeah, I think right now I do ~all the feedback requests, and my goal tends to be 24h turnaround time or less (though it does sometimes get closer to 48h).
I’ve failed a few times. My social instincts tried to get me not to post this comment, in case it makes it more likely that I fail again, and failing hurts. I suspect there’s really strong survivorship bias here.
When I was young I felt like “Gosh! When I’m older and have a job, I really should use my power as a globally rich person to help those who are much less well off, because that’s obviously morally obligatory and this Peter Singer guy makes sense.”
When I read Slate Star Codex’s “Everything is Commensurable” I thought “Oh right, I suppose now’s the time for that, I have more money than I need, and 10% seems about right.”
It felt satisfying to be doing something definitive, to have an ironclad excuse for not freaking out about whatever the issue of the day is. “I’m doing my part, anyway.”
Then I learned there was a community, was dazzled by how impressive they all were, overjoyed that they wanted to welcome me, and had a strong emotional reaction to want to be a part of it. It was more excitement about the people than the projects. They felt very much like “my people.”
Now I don’t feel much of anything about it (maybe a touch of pride or annoyance about losing so much money), but I still give my 10% to AMF monthly, and I don’t plan to stop, so I guess the earlier surges of emotions did their job.
Fine, basically. Surprised at the degree of emotional investment people seem to have in the EA community as such, above and beyond doing helpful stuff for the world personally. Sounds unpleasant. If I had feelings anywhere near as negative about a community as the ones I’m seeing routinely expressed, I think I’d just stop engaging and go do something else.