I used to work in Pharma. There are organisations like the FDA which need to approve a drug before it can be put on the market. We need legislation to create something similar for AI models. And the burden of proof is on the drug-maker—prove that your drug is safe. Same is needed for AI.
I am an engineer. If I build a bridge and it collapses, I am responsible. I will pay millions (or my insurance will) and I will spend the rest of my life in jail, unless I can demonstate (burden of proof is on me) that I did everything exactly as it should be done, and the collapse was due to something I could not have foreseen and the regulations did not foresee. Why not something similar for AI developers—proper liability.
When I work, a company pays me. Let’s say they pay me $100,000 per year. In most countries, the company pays social security on top of this, and other costs, maybe $20,000 per year, to the government. Then I pay taxes and social security, and local taxes. Maybe another $50,000. So for every net $50,000 I earn, the company pays $70,000 to the government in one form or another.
If they use an AI agent, they can just pay $50,000 (or whatever) and the government doesn’t take one penny. If AI’s replace humans as workers, why should humans pay tax, and employers pay tax to hire humans, but AI’s not pay tax, and companies pay not tax to hire AI’s?? If AI “workers” were taxed at the same rate as humans, replacing humans with AI would be a lot less financially interesting.
The government needs to start taxing AI workers in the same way as they tax human workers.
If I commit a crime, the government punishes me. When an AI agent commits a crime, someone must be punished. Let the AI developer and the AI owner/user fight it out in court, but at least one of them must go to jail. Currently they each just deflect blame and nobody pays.
And so on …. there are so many good and very reasonable and fair ways that AI could be managed so that there is a genuine incentive on AI developers to make their AI’s behave.
Denis
I love this post.
When I worked at a multi-national, I learned (too late) a concept called PIE, which was eye-opening.
P = Performance—this is what you actually do, the contributions you make.
I = Image—this is how you and your performance are perceived.
E = Exposure—this is who sees your work.
The point of this was that many of us work on the P only, we assume that the I and E will follow. But often they don’t unless we actively manage them. A lot of corporate frustration is about people who focus only on P angriliy complaining about people who get ahead while they don’t—and often those who get ahead are people who focus on I and E—but also P.
The trainer’s central thesis was this: YOU are responsible for your P, your I and your E. If you focus only on your Performance, don’t blame anyone else when you don’t get promoted. You need to take ownership for how you are perceived and who perceives you.
Here’s where it got interesting. People pushed back: that isn’t fair! I focus on doing my job, I do it extremely well, why should “image” matter? Why should exposure matter?
And the response was not what they expected. It was not defensive at all. The trainer said: “That is not true. If you are not focused on image and exposure, you are not doing your job as well as you could be. Because you are not setting an example that people follow. You are not sharing your ideas for others to learn from. You are not Influencing enough.
I sometimes think this is a great concept to share within EA. Because within EA, there are some people who do a fantastic job of getting their ideas out there, about getting talked about, about moving the Overton window. And there are others who also do brilliant work, but much of it stays within the EA community.
And in the end, if we’re looking at impact, we have to ask: who has the most impact? Not: who wrote the best articles? Not: who put in the most hours? But the impact.
This week, one guy did more for AI Safety than 1000 people have done in 10 years. He resigned (P), but unlike many before him, he somehow grasped the media moment, came across as a very normal, not nerdy guy, who truly cared about humanity (I). And got the exposure via his posts and major interviews. (E) He’s surely not the “best” AI Safety person ever, but he had a huge impact by getting the image and exposure right.
I know there was also a lot that was out of his control, the timing, the background and so on. But it should be an example to us.
Hey, I have experience with the preliminary parts of this—chemical engineering, but not in the context of cultivated meat. Happy to help / advise.
I have run industrial workshops where we’d get everyone in a room, start with a blank sheet of paper and end up with an 80⁄20 process flow diagram, highlighting the certainties and the open questions and the opportunities for further research, and then move on from this (with a different group sometimes) to build an 80⁄20 TEA.To be clear, what I have run was not as computationally intense as you propose. The model with Monte-Carlo syntheses etc would typically be a secondary step. The first step would be to get the right information into a flowsheet, then into a TEA, and for each parameter to have a clear idea of the sources of variability and the uncertainties. Then you can even run the Monte Carlo simulation on the TEA itself if it’s simple enough.
It’s not rocket-surgery, as W would say. Just structure and getting the right people, the right knowledge and the right mentality in the room, and getting the right pre-work done.
Since I don’t work in this field, I can’t really say to what extent the bit I could help with has already been done … if not, I’d be happy to help, time permitting. No interest in being paid to help, at least for the first iteration.
I think all your comments are valid. I definitely did not agree with the conclusions or the negativity towards utilitarianism as the vast majority of EA’s see it.
My motive for sharing this was: in previous posts I’ve shared examples of where EA’s are unjustly considered, basically, to weird to be relevant, without any clear justification of why. In this article, at least the allegedly flawed assumptions of EA’s were clearly outlined, and it was not outrageous to argue that, yes, some EA’s do think like that sometimes.
So, IMHO we all need to be ready with arguments like what you’ve outlined, to say “I can see why you think that—but here is a better way to think about it, and I think it will reassure you that most EA’s, even those who call themselves utilitarians, do not have these crazy ideas.”
100% feel the same way. Thank you for stating it so clearly.
It is sickening to watch our leaders do exactly nothing, as if this was some sort of joke.
It makes me incredibly angry that, at what may be the most important time in human history, at a time when the world desperately needs good leadership, the most powerful man in the world is such a vile, ignorant moron without a care for anything but his own ego and his bank account. This is what makes me fear that it really could get to extinction, because he is going to let humanity expire rather than listen to advice that he doesn’t agree with. And those spineless Republicans who refuse to stop him. And the pathetic EU leadership who talk a good game but ultimately care more about profit. It is sad, but humanity depends on the wisdom and decency of Xi Jinping—normally we’d be hoping we could persuade him to be reasonable, now it’s likely that he will be the reasonable one.
It makes me angry that so many people are stating so openly, factually, that humanity may be destroyed. We know they are the experts, but, as Al Gore would say, believing them would be inconvenient for us, so we just ignore them.
It makes me angry that we have a world in which experts are no longer trusted. Yes, we understand how this has been achieved and for what ends, but at times like this the world needs people who can inspire universal trust, and instead anyone who disagrees is labelled a fraud or a traitor or naive …
I suppose writing this post has made me realise that I don’t feel lonely so much as angry.
Utilitarianism / Extinction / EA’s
I have to agree that the motivation and talent of AIM staff is pretty unique!
I would counter that EA / Rationalism is not a set of ideas but a way of thinking, analogous to the scientific method. You can disagree with the ideas that some EA’s or Rationalists have, but you cannot argue that approaching a topic using data and logic and science is flawed. That is just plain wrong.
For years now, we have seen a constant attack on the scientific method, on science itself. And IMHO we have not pushed back correctly, and specifically, we have not pushed back hard enough and bluntly enough.By trying to be reasonable with people who are not being reasonable, we have only lent credibility to their false ideas. And the result is that the most advanced democracy in the world officially denies climate change and has a Secretary of Health who trusts tictoc influencers more than doctors.
We need to call bullshit on arguments like this. It is bullshit. If you disagree with the conclusion, sure, argue against that based on logic or data. But you don’t get to dismiss it just because the people who reached that conclusion used logic and data and science to reach it. You just don’t. It’s not an “everyone is entitled to their opinion” case—it’s just plain wrong.
We need to be willing to stand behind our conclusions, and call out those who claim to refute them with ad hominem arguments.
I’m not sure what to think. It feels like EA’s are cool with being seen as weird. And I’m not cool with that, because IMHO it dramatically reduces our potential for impact. This being an example, where arguments which make sense about AI safety get rejected for no reason other than that they come from “Rationalists.”
Ultimately do we want to feel good about ourselves, reassured that we’re right even as everyone else is wrong, or do we actually want to make sure that the right decisions get taken to prevent AI catastrophe?
My problem is that it seems like ALL the coverage that EA’s get is like this.
I do take your point that EA’s may seem weird, but most of the EA’s I know are mostly weird in a good way—more caring, more focused on helping, more concerned with the victims, … - but we allow all this to be overshadowed by the perceived weirdness of using logic and data to help people more, or by stereotypes based on more controversial ideas like longtermism taken to its logical extreme.
And I’m not worried about its effect on our delicate egos .. I’m worried by how this limits our potential to impact the vital debates of our time. It frustrates me that EA arguments can be dismissed by ad hominem arguments. This article basically says :`”we can ignore those concerns, they only come from those weird EA’s.”
Absolutely. I agree. I just think it’s sad—for us but also for them. My impression is that for every “successful” academic, there are 10 frustrated academics who feel they have been undervalued, disrespected, unlucky, … and even the successful ones frequently have very high stress levels—whereas those who focus on the subject rather than the prestige tend to be happier. Of course some achieve both and that is the ideal.
Fair push-back, indeed there is no need to exaggerate when the facts are already so strong!
Stereotyping EA’s as “weird”
I hope you’re right. To be crystal clear, I fully support giving staff a good living wage. And I have no doubt that the aim team are extremely self-motivated. But I just fear that you are betting against decades of experience and research about motivation. I would love to be wrong.
I’m curious why you think this was AI/LLM reasoning.
I’m not sure what precise objection you have to the article. There is a clear, tangible problem: what we are doing now is not working. The majority of people are not engaged in AI Safety. When people are not engaged in a critical current topic, that is almost by definition a communication problem.
You didn’t ask me (I think), but yes, I believe that if there is no alternative funding available, then we need to take funding from other critical EA work and use it to get the general public engaged in AI. I believe that if we do the calculation accurately, the impact per $ of getting the American and EU voters actively considering AI when they vote is enormous, as it will cause politicians to take the issue, and safety concerns, far more seriously than they are currently, and lead to sensible regulation.
One data-point to put this in context: multinational companies (Unilever, J&J, Anheiser-Busch, etc) regularly spend more money on advertising (short-form communication aimed at the general public) than they spend on producing the products they sell. This is because it is proven to make a difference, and they can measure the impact per dollar precisely.It sounds like you either do not believe this is valuable, or do not believe it can be done. These companies, and the AI labs and the climate-deniers would not be spending so much money on communication if it wasn’t effective.
The AI Safety movement does not need to spend a lot of money on this, but we do need to do the bare minimum, or we’re just letting the other side dominate the public debate.
100% spot on. I have a draft post I’d been meaning to share about this:
I received 9 AI-Safety related mails in my inbox this week. The average length is 2699 words, or about 10 pages of a Word document. Most of these are full of links to much longer articles for anyone who is serious about the topic. One includes a “very strong recommendation” to listen to a 6-hour Dwarkesh podcast. And this is the short-content stuff, not the in-depth predictions like AI 2027 and Europe 2031. The content is outstanding, thoughtful, thorough and deeply researched, and I absolutely would not change it.
But in parallel to this we need to start creating sharp, CONCISE, messaging that will be read by the 99% of people who might only have a few minutes to learn about the topic and may not see why they should spend even that. These people vote too.
This requires a totally different skillset (that I do not have either). The vital soundbite. The one line, one data point that shocks people—delivered in isolation, not cluttered with caveats or even justification.
We can learn from advertisers whose job is to communicate a complex idea in 30 seconds in an engaging way. There is a science and an art to this, and those of us who enjoy long, complex arguments usually do not even realise this, let alone possess it …
Or we can learn from our enemies. Donald Trump is a master of this—his statement that “Canada has been ripping us off for decades” is a great example. Even though we know he’s a shameless liar, we’re compelled to think about it—it engages us, it enables otherwise rational people to ask “but has Canada beeb screwing us all these years?” …
In the field of AI Safety, the statement “If anyone builds it, everybody dies!” was a great example. It could have changed the game. It was the kind of idea that got people’s attention.
But instead of doubling down and getting an active public debate on AI Safety on the front pages, we (the self-styled AI Safety community!!) killed all the momentum with our desire to be precise and calibrated and not to exaggerate. We watered it down. We said it wasn’t as simple as that. We mentioned multiple scenarios, many of which turn out OK. We presented complex scenarios and analyses that served only to tell 99% of people that this was a very complex topic and probably not as worrying as it had seemed, and that they would never understand the arguments well enough.
We have to choose. Engaging content tends to shock. It exaggerates. It oversimplifies. But it gets people thinking. We need to realise that this is a separate but necessary step to having the AI Safety discussion that the world needs to have.
Meanwhile Donald Trump invents a lie that has 60 million people believing they understand international tariffs, which he doesn’t even understand himself. We re right, but he is a far more effective communicator.
It sickens me to se this again and again—on the climate, on science, on AI, … ; the good guys are right but the bad guys win the public debate because they are waaaaaaaay better at communicating to the general public.
When IABIED came out, we needed everyone to be joining in the chorus that this was massively scary, that yes, it was likely we’d all die, that stopping AI was the single most urgent thing that humanity needed to do. We needed to make it go viral. Because that’s how you get the topic into the public eye. That’s how you get voters and influencers and even politicians engaged. Once we get the engagement, once everything from pausing AI to shutting down AI labs globally is in the Overton window, then we can start hedging a bit, sharing more complex ideas. Because then people will realise that they need to engage.
But it’s a 2-stage process. The first stage is to get Joe and Jane Public to get a visceral feeling that this is scary, that they need to do something about it. There are people out there who could do this, but almost all of them are being paid by the AI labs to reassure the public that there’s nothing to worry about …
Fantastic! I love this idea and your plans.
I don’t know if you’re connected into the global EG ecosystem, if not, talk to Luke Moore, he knows everyone and everything.
Some of your challenges have been solved or are being worked on in different countries, in different ways. E.g. many European EGI’s had to translate material and adapt for local consumption. Several (including us in Ireland) have struggled with tax-policies which discourage foreign donations.
For funding, have you talked to Meta-Charity Funders?
Great post, just seeing it now.
I work with a lot of PhD’s and post-docs in academia and industry. My sense is that, while the incentives are nominally what they are, there is a deeper problem with using them as an excuse. If you’re in an academic field to “succeed”, you are in the wrong field. Academia, PhD’s, post-docs and all that are about spending irrational amounts of energy and going to great depth in a subject because you want to, driven by intellectual curiosity. Some people, a small percentage, turn this into a highly paid career, but these are the exceptions, and there is a lot of luck involved—the theory you guess turns out to be right, you advisor picks a topic that turns out to be hot 10 years later, etc. You cannot control these things. But you can have a very satisfying, if not super well-paid, life. It feels like getting paid for stuff you’d happily do in your free time, arguing about to[pics you’re passionate about. Once you deviate from the path of seeking the truth, you lose this. You are no longer being true to yourself. You stress about perception and luck and so much that you can’t control. You envy rather than celebrate other researchers and new discoveries, It becomes psychologically stressful, you will not be happy. If you’re smart enough to do a PhD but more interested in getting ahead, just become a CEO.
Thanks, this is some great perspective. It is true that I’m based in Europe and most of my direct interventions with EAs have been with Effective Giving and Animal Rights people, maybe not typical, at least in style. I do love reading EA’s like Yudkowsky and MacAskill too, and often I agree with them, but I can see why many might not.
However where I would push back is on the idea that EA does not belong in the centre of the Overton window. IMHO there is hope for a post-fascist world where truth-seeking becomes again a virtue in itself. EA’s themselves disagree on many things, so I’m not pushing one particular political position, but rather the idea that, when faced with challenging problems, EA-type analysis would be part of the discussion, part of the truth-seeking, with people like Scott Alexander or Elizier Yudkowsky or Will MacAskill playing the role that French intellectuals used to play—helping people to tease through the details, popularising ideas even if they can’t fit in a 3-word slogan, …… and this becomes much easier if we can bring the concept of EA into mainstream dialogue in a positive context. Maybe we could get Billie Eilish to talk about factory farms …
Great, I really applaud this initiative, and it feels to me that it can be a real credibility builder. IMHO there is no question at all that artificial protein will dominate our diets by the end of the century, and hopefully a lot soon. It is difficult, but the current system is so bad, on so many levels, that even imperfect solution will be just so much better, cheaper, safer, more ethical …
I had one additional though: this week I did a TEA for a new process. I knew quite a bit about the process itself but hadn’t done a TEA before. I asked Claude Pro to help, and it was amazing. Both for the insights and for the formatting—it gave me a 10-page Excel with a very flexible TEA.
What I particularly liked was a couple of places where it caught me out (“you didn’t include the energy consumption of Step X, which can be significant”) or questioned my assumptions (“if you make product X, your TEA will be dominated by Cost Y, which is not realistic, because in that case you would change Condition Z” … really insightful things).
I would never use an AI-generated TEA as a final model, but it is a great way to get an 80⁄20 model that people can use as a starting point.
(if this seems useful, happy to go into more detail about this, but my guess is that you can do it yourselves, it’s very intuitive once you’re very clear about what you want and what you do not want Claude to assume).
(I’m all for slowing down AI, but also astonished by what the AI we have already can do!)