I am a lawyer and policy researcher interested in improving the governance of artificial intelligence. I currently work as Director of Research at the Institute for Law & AI. I previously worked in various legal and policy roles at OpenAI.
I am also a Research Affiliate with the Centre for the Governance of AI and a VP at the OāKeefe Family Foundation.
My research focuses on the law, policy, and governance of advanced artificial intelligence.
You can share anonymous feedback with me here.
I upvoted and didnāt disagree-vote, because I generally agree that using AI to nudge online discourse in more productive directions seems good. But if I had to guess where disagree votes come from, it might be a combination of:
It seems like we probably want politeness-satisficing rather than politeness-maximizing. (This could be consistent with some versions of the mechanism you describe, or a very slightly tweaked version).
Thereās a fine line between politiness-moderating and moderating the substance of ideas that make people uncomfortable. Historically, it has been hard to police this line, and given the empirically observable political preferences of LLMs, itās reasonable for people who donāt share those preferences to worry that this will disadvantage them (though I expect this bias issue to get better over time, possibly very soon)
There is a time and place for spirited moral discourse that is not āpolite,ā because the targets of the discourse are engaging in highly morally objectionable action, and it would be bad to always discourage people from engaging in such discourse.*
*This is a complicated topic that I donāt claim to have either (a) fully coherent views on, or (b) have always lived up to the views I do endorse.
Both Sam and Dario saying that they now believe they know how to build AGI seems like an underrated development to me. To my knowledge, they only started saying this recently. I suspect they are overconfident, but still seems like a more significant indicator than many people seem to be tracking.
I also have very wide error bars on my $1B estimate; I have no idea how much equity early employees would normally retain in a startup like Anthropic. That number is also probably dominated by the particular compensation arrangements and donation plans of ~5ā10 key people and so very sensitive to assumptions about them individually.
Indeed, though EAs were less well-represented at the senior managerial and executive levels of OpenAI, especially after the Anthropic departures.
One factor this post might be failing to account for: the wealth of Anthropic founders and early-stage employees, many of whom are EAs, EA-adjacent, or at minimum very interested in supporting existential AI safety. I donāt know how much equity they have collectively, how liquid it is, how much they plan to donate, etc. But if I had to guess, thereās probably at least $1B earmarked for EA projects there, at least in NPV terms?
(In general, this topic seems under-discussed.)
Iām obviously not a bankruptcy lawyer, but I was surprised to read this because I assumed that the statute of limitations depended on the jurisdiction of the bankruptcy estate, not the clawback claimee. Am I wrong? Or are there parallel proceedings in other jurisdictions?
Thank you for all your contributions, Luke! GWWC made tremendous progress under your leadership.
On the allocative efficiency front, the Harris campaign has pledged to impose nation-wide rent controls, an idea first floated by President Biden. Under the proposal, ācorporate landlordsā with 50+ units would have to āeither cap rent increases on existing units to no more than 5% or lose valuable federal tax breaks,ā referring to depreciation write-offs. This would be a disastrously bad policy for the supply-side of housing, and an example of the sort of destructive economic populism normally ascribed to Trump.
Harrisās terrible housing policy can be discounted insofar as it would require an Act of Congress. That said, the impending expiration of key TJCA provisions creates a real opportunity for a version of this idea to be advanced via tax negotiations. As a senator, Harris introduced the Rent Relief Act in 2018, which would have offered ātax credits to renters who earn below $100,000 and spend more than 30% of their income on rent and utilities.ā This tracks with her record as attorney general, where she drafted and helped pass the California Homeowner Bill of Rights while supporting a number of other dubious āaffordable housingā initiatives. Her policy instincts are thus consistent with the worst āsubsidize demand, restrict supplyā form of lawyerly progressivism.
I think rent control is bad policy, but I think itās intellectually indefensible to not also note the various housing-restrictive things Trump has championed: e.g. https://āāwww.donaldjtrump.com/āāagenda47/āāagenda47-ending-bidens-war-on-the-suburbs-that-pushes-the-american-dream-further-from-reach and https://āātrumpwhitehouse.archives.gov/āābriefings-statements/āāpresident-donald-j-trump-protecting-suburbs-preserving-american-dream-americans/āā. Vance has also framed this as a demand issue due to illegal immigrants rather than as a supply side issue, which is obvious nonsense and also in deep tension with his pronatalist position. You also fail to consider how a 10% tariff on goods imported into the country would affect construction costs (my guess: not good) or how mass deportation of an enormous proportion of the construction labor force would affect it (my guess: also not good).
I would also note that, for all the flaws of the BidenāHarris administration on housing, more houses are being built during their administration than during Trumpās.
In general I find that a lot of your arguments are extremely one-sided in that they ignore very obvious counterarguments and fail to make the relevant comparisons on the same issue.
For example, on innovation policy, I think itās fair to praise the Trump administration for Operation Warp Speed, though this was a bipartisan effort that was enacted while Harris was a Senator, so she should also get credit for it. On the other hand, the Trump administration would do a lot of things that would be awful for innovation policy, including restricting immigration, making the US more culturally hostile to highly educated immigrants, increasing costs via 10% across-the-board tariffs, and reducing the fiscal resources available to subsidize innovation.
Strong downvoted on this basis.
Title should be āAI companies . . .ā
I agree he shouldnāt have his past donations held against him, and that his past generosity should be praised.
At the same time, heās not simply āstopping giving.ā His prior plan was that his estate would go to BMGF. Letās assume that that was reflected in his estate planning documents. He would have had to make an affirmative change to effect this new plan. So with this specific action he is not āstopping giving,ā he is actively altering his plan to be much worse.
I think many people are tricking themselves into being more intellectually charitable to Hanania than warranted.
I know relatively little about Hanania other than stuff that has been brought to my attention through EA drama and some basic āknow thy enemyā reading I did on my own initiative. I feel pretty comfortable in my current judgment that his statements on race are not entitled charitable readings in cases of ambiguity.
Hanania by his own admission was deeply involved in some of the most vilely racist corners of the internet. He knows what sorts of messages appeal to and mobilize those people, and how such racists would read his messages. He āknow[s] how it looksā not just to left-wing people but to racists.
More recently, he has admitted that he harbors irrational animus (mostly anti-LGBT stuff from what I know) that seems like a much better explanation for his policy positions rather than any attempt at beneficence from egalitarian first principles. If you just read his recent policy stances on racial issues, they are shot through with an underlying contempt, lack of empathy, and broad-strokes painting that are all consistent with what I think can fairly be called a racist disposition towards Black people in particular.
Charitable interpretation of statements can be a sensible disposition in many settings. But giving charitable interpretations to people with this sort of history seems both morally and epistemically unwise.
The prior on āperson with a white supremacist history still engaged in right wing racial politics still has a racist underlying psychologyā should be very high. Right-wing racists also frequently engage in dogwhistles to signal to each other while maintaining plausible deniability. Reading that statement (and others of his) with those priors+facts in mind, I feel very comfortable not giving Hanania any benefit of the doubt here.
Thereās also a textual case that I think supports the racist reading. Woke people walking around āin suitsā is not at all a common tropeāIāve literally never heard of someone talking about a woke person wearing a suit as some sort of significant indicator of anything. But racists judging Black people by what they wearāe.g., purporting to be willing to be nicer to Black people if only they dressed more appropriatelyāis a huge trope in American race discourse. This sort of congruence between racist tropes and Hananiaās language similarly applies to āin subwaysā and āanimals.ā These are racist tropes consistently used about Black people, not woke people.
Yeah fair, should have considered that more duh
Example: They crammed three cosmonauts into a capsule initially designed for one person. But due to the size constraints, the cosmonauts couldnāt wear proper spacesuits; they had to wear leisure suits!
Pretty wild discussion in this podcast about how aggressively the USSR cut corners on safety in their space program in order to stay ahead of the US. In the authorās telling of the history, this was in large part because Khrushchev wanted to rack up as many āfirstsā (e.g., first satellite, first woman in space) as possible. This seems like it was most proximately for prestige and propaganda rather than any immediate strategic or technological benefit (though of course the space program did eventually produce such bigger benefits).
Evidence of the following claim for AI: people may not need a reason to cut corners on safety because the material benefits are so high. They may do so just because of the prestige and glory of being first.
It could be the case that the board would reliably fail in all nearby fact patterns but that market participants simply did not know this, because there were important and durable but unknown facts about e.g. the strength of the MSFT relationship or playersā BATNAs.
I agree this is an alternative explanation. But my personal view is also that the common wisdom that it was destined to fail ab initio is incorrect. I donāt have much more knowledge than other people do on this point, though.
I think it would be fair to describe some Presidents as being effectively powerless with regard their veto yes, if the other party control a super-majority of the legislature and have good internal discipline.
(Emphasis added.) I think this is the crux of the argument. I agree that the OpenAI board may have been powerless to accomplish a specific result in a specific situation. Similarly, in this hypo, the President may be powerless powerless to accomplish a specific result (vetoing legislation) in a specific situation.
But I think this is very far away from saying a specific institution is āpowerlessā simpliciter, which is what I disagreed with Zachās headline. (And so similarly would disagree that the President was āpowerlessā simpliciter in your hypo.)
An institutionās powers will almost always be constrained significantly by both law and politics, so showing significant constraints on an institutionās ability to act unilaterally is very far from showing it overall completely lacks power.
I agree this would be appealing to intellectually consistent conservatives, but this seems like a bad meme to be spreading/āstrengthening for animal welfare. Maybe local activists should feel free to deploy it if they think they can flip some conservativeās position, but they will be setting themselves up for charges of hypocrisy if they later want to e.g. ban eggs from caged chickens.
How are you defining āpowerlessā? See my previous comment: I think the common meaning of āpowerlessā implies not just significant constraints on power but rather the complete absence thereof.
I would say that the LTBT is powerless iff it can be trivially prevented from accomplishing its primary functionāoverriding the financial interests of the for-profit Anthropic investorsāby those investors, such as with a simple majority (which is the normal standard of corporate control). I think this is very unlikely to be true, p<5%.
I definitely would not say that the OpenAI Board was powerless to remove Sam in general, for the exact reason you say: they had the formal power to do so, but it was politically constrained. That formal power is real and, unless it can be trivially overruled in any instance in which it is exercised for the purpose for which it exists, sufficient to not be āpowerless.ā
It turns out that they were maybe powerless to remove him in that instance and in that way, but I think there are many nearby fact patterns on which the Sam firing could have worked. This is evident from the fact that, in the period of days after November 17, prediction markets gave much less than 90% oddsāand for many periods of time much less than 50%āthat Sam would shortly come back as CEO.
As an intuition pump: Would we say that the President is powerless just because the other branches of government can constrain her (e.g., through the impeachment power or ability to override her veto) in many cases? I think not.
āPowerlessā under its normal meaning is a very high bar, meaning completely lacking power. Taking all of Anthropicās statements as true, I think we have evidence that the LTBT has significant powers (the ability to appoint an increasing number of board members), with unclear but significant legal and (an escalating supermajority requirement) and political constraints on those powers. I think itās good to push for both more transparency on what those constraints are and for more independence. But unless a simple majority of shareholders are able to override the LTBTāwhich seems to be ruled out by the evidenceāI would not describe them as powerless.
A warm welcome to the forum!
I donāt claim to speak authoritatively, or to answer all of your questions, but perhaps this will help continue your exploration.
Thereās an āoldā (by EA standards) saying in EA, that EA is a Question, Not an Ideology. Most of what connects the people on this forum is not necessarily that they all work in the same cause area, or share the same underlying philosophy, or have the same priorities. Rather, what connects us is rigorous inquiry into the question of how we can do the most good for others with our spare resources. Because many of these questions are philosophical, people who start from that same question can and do disagree.
Accordingly, people in EA fall on both sides of many of the questions you ask. There are definitely people in EA that donāt think that we should prioritize future lives over present lives. There are definitely people who are skeptical about AI safety. There are definitely people who are concerned about the āmoral licensingā effects of earning-to-give.
So I guess my general answer to your closing question is: you are not missing anything; on the contrary, you have identified a number of questions that people in EA have been debating for the past ~20 years and will likely continue doing so. If you share the general goal of effectively doing good for the world (as, from your bio, it looks like you do), I hope you will continue to think about these questions in an open-minded and curious way. Hopefully discussions and interactions with the EA community will provide you some value as you do so. But ultimately, what is more important than your agreement or disagreement with the EA community about any particular issue is your own commitment to thinking carefully about how you can do good.