I recently founded Apollo Research: https://www.apolloresearch.ai/
I was previously doing a Ph.D. in ML at the International Max-Planck research school in Tübingen, worked part-time with Epoch and did independent AI safety research.
For more see https://www.mariushobbhahn.com/aboutme/
I subscribe to Crocker’s Rules
No, I think there is a phase where everyone wished they had renewables but they can’t yet get them so they still use fossil fuels. I think energy production will stay roughly constant or increase but the way we produce it will change slower than we would have hoped.
I don’t think we will have a serious decline in energy production.
I think narrow AIs won’t cause mass unemployment but more general AIs will. I also think that objectively that isn’t a problem at this point anymore because AIs can do all the work but I think it will take at least another decade that humans can accept that.
The narrative that work is good because you contribute something to society and so on is pretty deeply engrained, so I guess lots of people won’t be happy after being automated away.
I’m bullish on Solar+storage. But I think that it will take a while to adapt the grid, so I think it will take at least a decade before we can even think about phasing out fossils.
I think narrow AIs won’t cause massive unemployment but the more general they get, the harder it will be to justify using humans instead of ChatGPT++
I think education will have to change a lot because students could literally let their homework be done entirely by ChatGPT and get straight A’s all the time.
I guess it’s something like “until class X you’re not allowed to use a calculator, and then after that you can” but for AI. So it will be normal that you can just print an essay in 5 seconds similar to how you can do complicated math that would usually take hours on paper in 5 seconds on a calculator.
Yeah. I kept the post mostly to AI but I also think that other technological breakthroughs are a possibility. Didn’t want to make it even longer ;)
I think you could write more of these stories for other kinds of disruptions and I’d be interested in reading them.
Thanks for pointing out the mistake. I fixed the “century” occurrences.
Yeah. I thought about it. I wasn’t quite sure how to structure it. I guess I’m not used to writing “story-like” texts. Most of my other writing is just a glorified bullet-point list ;)
Fair. I’ll go over it and explain some of the technical concepts in more detail.
Also, I also expect many people who are familiar with the latest discussions in AI to have longer timelines than this. So the intended audience is not just people who aren’t familiar with the field.
I’m obviously heavily biased here because I think AI does pose a relevant risk.
I think the arguments that people made were usually along the lines of “AI will stay controllable; it’s just a tool”, “We have fixed big problems in the past, we’ll fix this one too”, “AI just won’t be capable enough; it’s just hype at the moment and transformer-based systems still have many failure modes”, “Improvements in AI are not that fast, so we have enough time to fix them”.
However, I think that most of the dismissive answers are based on vibes rather than sophisticated responses to the arguments made by AI safety folks.
I don’t think these conversations had as much impact as you suggest and I think most of the stuff funded by EA funders has decent EV, i.e. I have more trust in the funding process than you seem to have.
I think one nice side-effect of this is that I’m now widely known as “the AI safety guy” in parts of the European AIS community and some people have just randomly dropped me a message or started a conversation about it because they were curious.
I was working on different grants in the past but this particular work was not funded.
I think it’s a process and just takes a bit of time. What I mean is roughly “People at some point agreed that there is a problem and asked what could be done to solve it. Then, often they followed up with ‘I work on problem X, is there something I could do?’. And then some of them tried to frame their existing research to make it sound more like AI safety. However, if you point that out, they might consider other paths of contributing more seriously. I expect most people to not make substantial changes to their research though. Habits and incentives are really strong drivers”.
I have talked to Karl about this and we both had similar observations.
I’m not sure if this is a cultural thing or not but most of the PhDs I talked to came from Europe. I think it also depends on the actor in the government, e.g. I could imagine defense people to be more open to existential risk as a serious threat. I have no experience in governance, so this is highly speculative and I would defer to people with more experience.
Reflects my experience!
The resources I was unaware of were usually highly specific technical papers (e.g. on some aspect of interpretability), so nothing helpful for a general audience.
Probably not in the first conversation. I think there were multiple cases in which a person thought something like “Interesting argument, I should look at this more” after hearing the X-risk argument and then over time considered it more and more plausible.
But like I state in the post, I think it’s not reasonable to start from X-risks and thus it wasn’t the primary focus of most conversations.
I don’t have any strong opinions on that. There is a good chance I’m just uninformed and the IEA is right. My intuition is just that countries don’t like it if their energy gets more expensive, so they’ll keep digging for coal, oil or gas as long as renewables aren’t cheaper.