Thank you for the correction, I updated the main text to reflect this.
mmKALLL
Karma: 120
Yes, it seems likely that an “interactive message” would have better value for the development cost. It might be worth trying in some cases. However, there’s two main problems with this approach:
The smaller scope and visual impact needed for virality means that the message needs to be greatly simplified. This can more easily lead to misinterpretations, which I’ve understood is highly undesirable for EA related messaging.
There’s still marketing effort involved and most attempts will fail, although it might be easier than with games.
The upside is that any good ideas are cheap to put out there and simply try out. The downside is that the message seems to need particularly careful consideration.
In my experience, most of the virality potential comes from timing and novelty of presentation, and is not so reliant on the message itself. It seems worth a try if any particularly good ideas come up.
As for small games like One Chance—I’m even more pessimistic about their cost-effectiveness. There’s literally hundreds of such games posted on sites like itch.io every day, many of which try their best to deliver a particular message or capture some part of the author’s experience.
Since only about 1/1000 of the games stand out, one shouldn’t rely on luck. Thus marketing effort is needed, but trying to build a following around a tiny game seems like a disproportionately difficult ask. I’m not saying that tiny games are worse as games (I particularly enjoy them), but rather their market is much more competitive (= expensive) to promote in.
Steam games on the other hand enjoy a multitude of benefits—there’s less competition, Steam provides discoverability, it’s easier to build a presence around the game, reviewers are easier to reach and get interested. Yet, even that market seems too expensive for promotion to be worth it.
As far as marketing is concerned, in my opinion it would make more sense to promote existing comprehensive material (like 80,000 Hours, EA introductions, etc). They are already interesting and have potential for virality as it is.
How my view on using games for EA has changed
After looking into other resources, it seems like this is the best overview of what everyone in the field is doing:
This is both very informative and very helpful, thank you for the advice! That does seem like a very reasonable way of thinking about the current situation, and I’m happy to see that there already exist resources that try to compile this information.
I was already referred to AISS in private, but your recommendation helped me take the step of actually applying for their coaching. Looking forward to seeing what comes of it, thanks again!
Thank you for taking the time to provide your thoughts in detail, it was extremely helpful for understanding the field better. It also helped me pinpoint some options for the next steps. For now, I’m relearning ML/DL and decided to sign up for the Introduction to ML Safety course.
I had a few follow-up questions, if you don’t mind:
What are those 100 people going to do? Either start their own org, work independently or try to find some other team.
That seems like a reasonable explanation. My impression was that the field was very talent-constrained, but you make it seem like neither talent nor funding is the bottleneck. What do you think are the current constraints for AI safety work?
My confusion about the funding/private company situation stems from my (likely incorrect) assumption that AI safety solutions are not very monetizable. How would a private company focusing primarily on AI safety make a steady profit?
OpenAI, DeepMind, Anthropic—have the safety of AI systems as an important part of their founding DNA.
I currently view OpenAI and DeepMind more like AI product companies, with “ethics and safety as considerations but not as the focus.” Does this seem accurate to you? Do engineers (in general) have the option to focus mainly on safety-related projects in these companies?
Separately, I’d also like to wish the DeepMind alignment team the best of luck! I was surprised to find that it has grown much more than I thought in recent years.
Yes, and they would have been my number one picks some years ago. However, I’m no longer convinced that they are progressing AI safety measures at the same speed they’re pushing for new capabilities. Intuitively it feels unsustainable (and potentially very dangerous), which is why I’m being more cautious now.
That being said, I’m very glad that both companies are putting effort into growing their safety teams, and hope that they continue to do so.
Thank you for sharing your thoughts! It’s helpful to know that others have been struggling with a similar situation.
Through investigating the need and potential for projects, it seems there are vaguely two main areas for engineers:
Research engineering, it seems to be essentially helping researchers build prototypes and run models as smoothly as possible.
Various meta-projects that grow the talent pool or enable knowledge to be gained and shared more efficiently. The ones in the post you linked fall under this.
It seems like getting (more useful) summaries of papers and blog posts is in very high demand. I wonder if Elicit (https://elicit.org/) is useful enough to somewhat alleviate that problem already.
I also came across this list of engineering project ideas while investigating: https://alignment.dev/
I’m thinking that working on one of these could be a useful entry point. It seems viable to do while studying the field itself.
Thank you for the link, I found several collections of links and more introductory information through it. This was very helpful for finding out about potentially relevant courses and opportunities.
[Question] Concerns about AI safety career change
I struggled with a similar question back when I was a student. What I’ve found out is that people asking this usually want to know how the applicant describes their work and approach, and how confident or passionate a person is about the things they do.
One option could be to talk about the most exciting university project/assignment that you’ve worked on. You could describe something that made it interesting, what you learnt from it, and explain how you handled teamwork or prioritization during it. Interesting results are a plus, but learning experiences also make for a good story.
Other options include some kind of competitive performance, or a hobby project you felt passionate about and dedicated time and energy into. Personally I would even be happy to hear about something nice you did that helped somebody else. Feel free to be open and explain what made the experience special to you.
People asking this question usually understand that new graduates’ achievements don’t necessarily involve work projects. So my advice would be to not worry about the context too much.
Hey Markus, thank you for sharing your thoughts!
Considering that I need to cast a relatively wide net to find even thousands of players, the messaging about Mineralis is not very selective on purpose. I think that the overall demographic will end up being relatively random within the set of people who are gamers and enjoy JRPGs. Whether JRPG fans have a predisposition towards EA—that remains to be seen. :)
I would be super humbled if the game ends up being a success and can be considered useful for the purpose of actually teaching EA concepts. However, I would imagine that a vast majority learns better through reading articles or listening to podcasts.
Setting up a group for discussion sounds like a valuable idea. I added a personal task for that and expect to get to it late next week. I’m thinking Discord might make the most sense, given that many developers already use it anyway.
Cost-effectiveness of making a video game with EA concepts
Saw this article about Patagonia’s future profits being directed to climate change -related charitable activities (on the scale of $100m / year). Working with them to direct funding to CATF etc. might be extremely high-impact?
mmKALLL’s Quick takes
Interesting post with a good angle, thank you for taking the time to write about this!
I’ve also been playing around with the idea of using art or some kind of interactive medium to convey EA concepts. I completely agree that it’s important that ideas are not “pushed” as EA, but rather the work allows room for the viewer to discover the benefits of these ideas for themselves.
One idea I’ve had that I haven’t seen explored elsewhere is a video/board game about welfare and existential risk. For example, the player(s) might step into the shoes of a world leader, drafting policies to limit risks while maximizing global happiness. They would be gradually introduced to various concepts regarding existential risk, while having interesting and challenging game systems to play against.
One of the benefits with this approach is that it doesn’t have to be EA branded at all, yet can introduce difficult concepts with ease and nudge people to study or discuss these topics on their own. It would also be niche enough that market validation is important, but the costs for that could greatly be reduced through building a prototype and running a crowdfunding campaign.
In fact, I had some free time last weekend and built a functional prototype with realistic data. It’s obviously at a very early stage, but could be expanded in a variety of directions. In my opinion the interactivity can bring an immersive extra dimension to a work, and can help users realize new ways to think about the causality of something.
This was a great read, and just the kind of post I have been waiting for! I think that almost all of the principles should be helpful to keep in mind for anyone engaging in this kind of public discourse. In my opinion it is very important to increase the quality of communication and thus quality of knowledge across EA folks, both current and future, both internally and towards the general public; these kinds of posts would seem to help out there.
Of course, I might slightly overvalue this kind of discussion, since I don’t know about the demographics of the EA community, and might be somewhat similar to the earlier self you mentioned in the post, frustrated at not being able to learn and get up to speed about the topics I care about. I don’t know whether there is a demand for these kinds of information in the next generation, although it might be interesting and somewhat relevant to find out. I wonder if anyone has an idea about that?
Either way, thank you very much for taking the time to make these thoughts public, I—as well as many others, it seems—really appreciate it!
Yes, of course, but I haven’t seen spambots in comments so far, and thus would guess that they wouldn’t become a problem on top level either.
Upon revisiting this post and the comments it has garnered, I found myself wondering about another thing I’d like to ask from you:
How would one go about getting involved with the work of CEA or, say, 80,000 Hours? What kinds of skill sets would be essential for having a high impact while doing such meta work? Do you consider the potential impact for doing meta work to be higher than when earning to give?
Also, thanks for the detailed responses so far! I can see why it’s not reasonable to place writing on topics like these on high priority, but it doesn’t exactly give a sense of transparency either. Not that I would know how important or effective giving a sense of transparency would be, though.
I think there definitely have been games that have changed peoples’ moral preferences, or at least provoked thoughts/ideas that have led to shifts in priorities. Usually such a game wouldn’t cause a person to change their views entirely, but I’ve seen many cases where a good game would cause a person to update specific moral values (similar to a good movie or book). I’m not aware of any analysis of the cognitive mechanisms, though.
Just as a personal anecdote, I became more interested/concerned about global welfare after playing games like Cave Story, Final Fantasy 7, and Metal Gear Solid (which have diverse portrayal of strife/conflict). Games like Passage, Mother 3, and Undertale (which are about mortality, family, and friends) caused me to value my interpersonal and family connections much more highly. Yet others made animal welfare more of a concern for me.
In many of the cases, it wasn’t necessarily the content of the game that directly caused an update, but rather the thought process that it provoked, leading to a new conclusion of some kind. This is what I mean by games “inducing” ideas, e.g. by Mineralis having content that “induces EA concepts naturally through gameplay”. Many such occurrences are almost certainly intentional; it could be said to be a way designers add “subtext” into gameplay.
Granted, a lot of the EA-relevant interest provoked by games wasn’t very concrete until I discovered EA. But I think various experiences in games helped “plant the seeds” that led my moral compass to work the way it does today. I’m guessing that people who have experienced many things through movies would say the same about them.