I’m Callum—I founded the ARENA program for technical AI safety upskilling, I’ve also worked on various open source mech interp things. I was in Anthropic’s interp team for ~6 months (working on some of their recent papers) and I’m currently doing interp at DeepMind. I like mathematical art, running, climbing, and most importantly I hold a Guinness World Record for the largest number of clothespegs held in one hand at once (yep, really).
Callum McDougall
Karma: 435
Not forcing myself to be productive when I don’t feel like it. Recognising when I’m feeling like I need to do nothing for a few hours or even a day, and giving myself allowance to do that. Much better to double down when I am feeling productive, than forcing myself to work when I don’t.
(Obviously ymmv, and I’m lucky enough to have a job where I can make this kind of system work for me, I’m aware not everyone does.)
A tourist visa will be enough, I believe. We’ll be helping out participants with this process.
Yep, Excalidraw is great! I also used it to make this post:
https://www.lesswrong.com/posts/TvrfY4c9eaGLeyDkE/induction-heads-illustrated
Sorry for the delay, yep sure! Here’s the link: https://join.slack.com/t/join-asap/shared_invite/zt-1kkzoa53n-ImLZZpiM9L2uoV_bH7Oh2A
and I’ll also update it in the post.
Thanks! Will fix now. I really should set a repeating reminder for myself lol
Yep thanks for mentioning this, it did come up in the discussion on the Slack group and definitely updated me towards Discord. The vote for whether we should use Slack or Discord did end up going in favour of Slack by a margin of 14 votes to 6, so we’ll be sticking with Slack for now, but we might revisit the issue in the future if there’s good reason to (e.g. the 90 day history thing proves a significant inconvenience).
Thanks for commenting! Yep the link seems to have expired, this one should work (and the post is now updated).
Thanks for commenting! Yep the link seems to have expired, this one should work (and the post is now updated).
I’ll copy in my response from EA Groups Slack:
Yeah I’m aware that there are arguments against Slack on these kinds of bases. From my perspective, Discord has 2 main annoyances: (1) worse formatting options in messages (e.g. no links in text) and more importantly (2) no reply threads, which can make messages really cluttered. I’ve mentioned in the slack already that I’d be willing to cover costs if it comes to that
I’d be open to changing my mind on these points though, if I found that Discord had those features or close alternatives
Ah thanks, that looks awesome! Will definitely suggest this in the group
Great! (-:
Thanks for the comment! I first want to register strong agreement with many of your points, e.g. the root of the problem isn’t necessarily technology inherently, but rather our inability to do things like coordinate well and think in a long-term way. I also think that focusing too much on individual risks while avoiding the larger picture is a failure mode that some in the community fall into, and Ord’s book might have done well to spend some time taking this perspective (he does talk about risk factors which is part of the way to a more systemic perspective, but he doesn’t really address the fundamental drivers of many of these risks, which I agree seems like a missed opportunity).
That being said, I think I have a few main disagreements here:
Lack of good opportunities for more general longtermist interventions. I think if there were really promising avenues for advancing along the frontiers you suggest (e.g. trying to encourage cultural philosophical perspective shifts, if I’m understanding your point here correctly) then I’d probably change my mind here. But it still seems imo like these kinds of interventions aren’t as promising as direct work on individual risks, which is still super neglected in cases like bio/AI.
Work on individual risks does (at least partially) generalise. For instance, in the case of work on specific future risks e.g. bio and AI, it doesn’t seem like we can draw useful lessons about what kinds of strategies work (e.g. regulation/slowing research, better public materials and education about the risks, integrating more with the academic community) unless we actually try out these strategies.
Addressing some risks might directly reduce others. For instance, getting AI alignment right would probably be a massive boon for our ability to handle other natural risks. This is pretty speculative though, because we don’t really know what a future where we get AI right looks like.
Yeah +1 to Nandini’s point, I think we should have been made this clearer in the post. I think people have a lot of misconceptions about EA (e.g. lots of people just think EA is about effective charitable giving), and we wanted to emphasise this particular part rather than trying to construct the whole tower of assumptions.
That being said, I do think that the abundance of writing from Ord/Bostrom is something that we could have done a better job of toning down, and different perspectives could have been included. If you have any specific recommendations for reading material you think would positively contribute in any week (or reading material already in the course that you think could be removed), we’d be really grateful!
That’s something we’ve definitely considered, but the idea is for this course to be marketed mainly via CERI, and since they already have existential risks in their name plus define it in a lot of their promo material, we felt like it would probably be more appropriate to stick with that terminology.
“Supercluster”, “Triangulum”, or “Centaurus”?
Keeping with the astronomial theme of “Lightcone” and “Constellation”, while also sounding like they could be the names for gatherings of people
That’s a good point! Although I guess one reply you could have to this is that we shouldn’t expect paradigm shifts to slow down, and indeed I think most of Yudkowsky’s probability mass is on something like “there is a paradigm shift in AI which rapidly unlocks the capabilities for general intellgence”, rather than e.g. continuous scaling from current systems.
Thanks, I really appreciate your comment!
And yep I agree Yudkowsky doesn’t seem to be saying this, because it doesn’t really represent a phase change of positive feedback cycles of intelligence, which is what he expects to happen in a hard takeoff.
I think more of the actual mathematical models he uses when discussing takeoff speeds can be found in his Intelligence Explosion Microeconomics paper. I haven’t read it in detail, but my general impression of this paper (and how it’s seen by others in the field) is that it successfully manages to make strong statements about the nature of intelligence and what it implies for takeoff speeds without relying on reference classes, but that it’s (a) not particularly accessible, and (b) not very in-touch with the modern deep learning paradigm (largely because of an over-reliance on the concept of recursive self-improvement, that now doesn’t seem like it will pan out the way it was originally expected to).
Ah yep, I’d been planning to do that but had forgotten, will do now. Thanks!
Update on the project board thing—I’m assuming that was referring to this website, which looks really awesome!
Thanks Artyom for sending this! Yes, we expect people to be participating full time in the virtual program. We will also try to clearly indicate sections of the material which are more important than others, and structure things in such a way that missing out on one day wouldn’t mean falling behind on the curriculum overall.