Brilliant. âmaintaining access to the frontier of animal sufferingâ is so cursed it made me laugh out loud. So many gems here.
I believe Anthropic engages in quite aggressive anti-regulation lobbying with one side of itâs mouth even as itâs all like âoh no, our products are delivering too much value, weâre so scaredâ with the other. And ya, literally pushing forward the frontier all the time to attract investment is suspiciously similar to what an evil company would do, so thank god we know theyâre on our side.
I find the âmake money doing evil stuff, but be slightly less evil than the imagined counterfactualâ theory of change sort of mesmerizing. I can kind of imagine versions of it that work on paper maybe in a sort of trolley problem way, but it smells too clever by half.
In the actual case of AI companies:
It is not obvious how much the existence of Anthropic is zero-sum relative to the rest of the capabilities sector. Part of the case has to be that they exist more or less instead of some other actor, but they might just be increasing the total supply of digital minds. Even if their specific competitors today donât like them, they could still be contributing to the power of the industry through their contribution to eg. tool use through MCP, to the general lobbying accelerationist effort, to the talent pool /â salaries of capabilities work, creating demand for LLM products, attracting investment etc. Even if eventually there is only room for a few AI firms, they are all unprofitable money pits right now anyway, so it probably isnât easy to displace someone else and they could basically be growing the space more than displacing nefarious actors.
It is not obvious that Anthropic produces lower x-risk per dollar invested or token produced or whatever metric than their competitors. I mean, maybe. They write some thoughtful blog posts and some insane nationalist ones, I like some of their âsafetyâ research. The bar is really low, I kind of hate Google and Sam Altman is a known abuser (employees have said so, he lies constantly, and his sister credibly claims he sexually assaulted her). But, if you entertain the perspective that actually ~all of their production ready âsafetyâ work is unserious in the face of the alignment /â disempowerment problems at stake, then it could easily just be a wash. Like, buying from a factory farm with 20% more thumb twiddling, but essentially the same approach to mass torture.
And between those two objections, thereâs nothing really left to be said for the âmake money doing evil stuff, but be slightly less evil than the imagined counterfactualâ theory of change. I will caveat that I personally lack strong confidence on how this nets out per say⌠generally I think the âitâs bad to do evil stuffâ case wins on simplicity. Maybe, like, a true fast follower company that mostly just distilled and did for-profit safety and security work would be cool.
I tend to think safety-washing or moral cover arenât necessarily a huge deal for the companies because EAâs brand isnât that important to most people and AI Safety isnât that important to most people. Like, I donât know that the talent-attracting boost from not being evil is such a big deal. Nihilistic companies do evil stuff all the time in my opinion. Meta was using their LLMs to sensually chat up kids (fully automated grooming! robo-pedos!) and it only made their hiring a little bit harder. Iâm sure there are plenty of creeps who know ML or schmucks who can be made to look the other way for a check.
But I will say, I think the whole âgood guy with an ASI companyâ line of thinking and the for-profit interests promoting it may have significantly corroded the EA AI Safety scene itself and itâs ability to judge right from wrong. 80,000 Hours consultation recommended that I just get any âopsâ job at Google s long as it vaguely related to AI, like wtf. So much for careful philosophy, yâknow, just get right up to the finish line and say âya, idk, I guess just try to be a lab aid for who-ever is making the deadliest humanoid virusesâ. Look at how much hate Holly Elmore got for promoting moratorium advocacy as a cause area. And the level of conflict of interest going all the way to the very top of eg. Coefficient Giving, CEA was never âepistemically virtuousâ shall we say; seems sort of corrupt.
Brilliant. âmaintaining access to the frontier of animal sufferingâ is so cursed it made me laugh out loud. So many gems here.
I believe Anthropic engages in quite aggressive anti-regulation lobbying with one side of itâs mouth even as itâs all like âoh no, our products are delivering too much value, weâre so scaredâ with the other. And ya, literally pushing forward the frontier all the time to attract investment is suspiciously similar to what an evil company would do, so thank god we know theyâre on our side.
I find the âmake money doing evil stuff, but be slightly less evil than the imagined counterfactualâ theory of change sort of mesmerizing. I can kind of imagine versions of it that work on paper maybe in a sort of trolley problem way, but it smells too clever by half.
In the actual case of AI companies:
It is not obvious how much the existence of Anthropic is zero-sum relative to the rest of the capabilities sector. Part of the case has to be that they exist more or less instead of some other actor, but they might just be increasing the total supply of digital minds. Even if their specific competitors today donât like them, they could still be contributing to the power of the industry through their contribution to eg. tool use through MCP, to the general lobbying accelerationist effort, to the talent pool /â salaries of capabilities work, creating demand for LLM products, attracting investment etc. Even if eventually there is only room for a few AI firms, they are all unprofitable money pits right now anyway, so it probably isnât easy to displace someone else and they could basically be growing the space more than displacing nefarious actors.
It is not obvious that Anthropic produces lower x-risk per dollar invested or token produced or whatever metric than their competitors. I mean, maybe. They write some thoughtful blog posts and some insane nationalist ones, I like some of their âsafetyâ research. The bar is really low, I kind of hate Google and Sam Altman is a known abuser (employees have said so, he lies constantly, and his sister credibly claims he sexually assaulted her). But, if you entertain the perspective that actually ~all of their production ready âsafetyâ work is unserious in the face of the alignment /â disempowerment problems at stake, then it could easily just be a wash. Like, buying from a factory farm with 20% more thumb twiddling, but essentially the same approach to mass torture.
And between those two objections, thereâs nothing really left to be said for the âmake money doing evil stuff, but be slightly less evil than the imagined counterfactualâ theory of change. I will caveat that I personally lack strong confidence on how this nets out per say⌠generally I think the âitâs bad to do evil stuffâ case wins on simplicity. Maybe, like, a true fast follower company that mostly just distilled and did for-profit safety and security work would be cool.
I tend to think safety-washing or moral cover arenât necessarily a huge deal for the companies because EAâs brand isnât that important to most people and AI Safety isnât that important to most people. Like, I donât know that the talent-attracting boost from not being evil is such a big deal. Nihilistic companies do evil stuff all the time in my opinion. Meta was using their LLMs to sensually chat up kids (fully automated grooming! robo-pedos!) and it only made their hiring a little bit harder. Iâm sure there are plenty of creeps who know ML or schmucks who can be made to look the other way for a check.
But I will say, I think the whole âgood guy with an ASI companyâ line of thinking and the for-profit interests promoting it may have significantly corroded the EA AI Safety scene itself and itâs ability to judge right from wrong. 80,000 Hours consultation recommended that I just get any âopsâ job at Google s long as it vaguely related to AI, like wtf. So much for careful philosophy, yâknow, just get right up to the finish line and say âya, idk, I guess just try to be a lab aid for who-ever is making the deadliest humanoid virusesâ. Look at how much hate Holly Elmore got for promoting moratorium advocacy as a cause area. And the level of conflict of interest going all the way to the very top of eg. Coefficient Giving, CEA was never âepistemically virtuousâ shall we say; seems sort of corrupt.
Just a few thoughts. Really good satire. Thanks!