Update on the project board thing—I’m assuming that was referring to this website, which looks really awesome!
TheMcDouglas
Karma: 378
Where’s WALY?
How I use Anki: expanding the scope of SRS
Skilling-up in ML Engineering for Alignment: request for comments
MIRI Conversations: Technology Forecasting & Gradualism (Distillation)
Ah yep, I’d been planning to do that but had forgotten, will do now. Thanks!
Thanks, I really appreciate your comment!
And yep I agree Yudkowsky doesn’t seem to be saying this, because it doesn’t really represent a phase change of positive feedback cycles of intelligence, which is what he expects to happen in a hard takeoff.
I think more of the actual mathematical models he uses when discussing takeoff speeds can be found in his Intelligence Explosion Microeconomics paper. I haven’t read it in detail, but my general impression of this paper (and how it’s seen by others in the field) is that it successfully manages to make strong statements about the nature of intelligence and what it implies for takeoff speeds without relying on reference classes, but that it’s (a) not particularly accessible, and (b) not very in-touch with the modern deep learning paradigm (largely because of an over-reliance on the concept of recursive self-improvement, that now doesn’t seem like it will pan out the way it was originally expected to).
That’s a good point! Although I guess one reply you could have to this is that we shouldn’t expect paradigm shifts to slow down, and indeed I think most of Yudkowsky’s probability mass is on something like “there is a paradigm shift in AI which rapidly unlocks the capabilities for general intellgence”, rather than e.g. continuous scaling from current systems.
That’s something we’ve definitely considered, but the idea is for this course to be marketed mainly via CERI, and since they already have existential risks in their name plus define it in a lot of their promo material, we felt like it would probably be more appropriate to stick with that terminology.
Yeah +1 to Nandini’s point, I think we should have been made this clearer in the post. I think people have a lot of misconceptions about EA (e.g. lots of people just think EA is about effective charitable giving), and we wanted to emphasise this particular part rather than trying to construct the whole tower of assumptions.
That being said, I do think that the abundance of writing from Ord/Bostrom is something that we could have done a better job of toning down, and different perspectives could have been included. If you have any specific recommendations for reading material you think would positively contribute in any week (or reading material already in the course that you think could be removed), we’d be really grateful!
Thanks for the comment! I first want to register strong agreement with many of your points, e.g. the root of the problem isn’t necessarily technology inherently, but rather our inability to do things like coordinate well and think in a long-term way. I also think that focusing too much on individual risks while avoiding the larger picture is a failure mode that some in the community fall into, and Ord’s book might have done well to spend some time taking this perspective (he does talk about risk factors which is part of the way to a more systemic perspective, but he doesn’t really address the fundamental drivers of many of these risks, which I agree seems like a missed opportunity).
That being said, I think I have a few main disagreements here:
Lack of good opportunities for more general longtermist interventions. I think if there were really promising avenues for advancing along the frontiers you suggest (e.g. trying to encourage cultural philosophical perspective shifts, if I’m understanding your point here correctly) then I’d probably change my mind here. But it still seems imo like these kinds of interventions aren’t as promising as direct work on individual risks, which is still super neglected in cases like bio/AI.
Work on individual risks does (at least partially) generalise. For instance, in the case of work on specific future risks e.g. bio and AI, it doesn’t seem like we can draw useful lessons about what kinds of strategies work (e.g. regulation/slowing research, better public materials and education about the risks, integrating more with the academic community) unless we actually try out these strategies.
Addressing some risks might directly reduce others. For instance, getting AI alignment right would probably be a massive boon for our ability to handle other natural risks. This is pretty speculative though, because we don’t really know what a future where we get AI right looks like.
Join ASAP (AI Safety Accountability Programme)
Great! (-:
Ah thanks, that looks awesome! Will definitely suggest this in the group
I’ll copy in my response from EA Groups Slack:
Yeah I’m aware that there are arguments against Slack on these kinds of bases. From my perspective, Discord has 2 main annoyances: (1) worse formatting options in messages (e.g. no links in text) and more importantly (2) no reply threads, which can make messages really cluttered. I’ve mentioned in the slack already that I’d be willing to cover costs if it comes to that
I’d be open to changing my mind on these points though, if I found that Discord had those features or close alternatives
Thanks for commenting! Yep the link seems to have expired, this one should work (and the post is now updated).
Thanks for commenting! Yep the link seems to have expired, this one should work (and the post is now updated).
Yep thanks for mentioning this, it did come up in the discussion on the Slack group and definitely updated me towards Discord. The vote for whether we should use Slack or Discord did end up going in favour of Slack by a margin of 14 votes to 6, so we’ll be sticking with Slack for now, but we might revisit the issue in the future if there’s good reason to (e.g. the 90 day history thing proves a significant inconvenience).
Thanks! Will fix now. I really should set a repeating reminder for myself lol
Hey Pablo! These seem really interesting, I love the implementation of music with Anki (even if they haven’t all been successes). Adding audio files was one thing I forgot to mention in the post, I did it when I was learning Chinese and it was pretty useful (although I stopped learning Chinese pretty soon after I started adding them, so they never really caught on—I suspect it would have been too much work to mass-produce).
I like the keyboard shortcuts one! It would be great if Anki had a way of testing whether you’d typed out the right keys, although I agree that without this it seems hard to implement.
And thanks for describing how you create Anki cards for textbooks—it’s always interesting to hear other peoples’ actual card creation process, i.e. how you decide which bits of knowledge to Ankify in whatever you’re reading, rather than just the mechanics (the former is something I didn’t really go into in the post).